Error: "The process 'xxx' not found" occurs after executing parallel python code

Question

I’m running a parallel python code using a port of pp (http://www.parallelpython.com//) for python 3 (https://pypi.python.org/pypi/ppft) and I’m getting this error right after the code finishes executing:

ERROR: The process "12492" not found.

I get one error for each processor used for the parallel program, each with different numbers and the numbers change each time I run the program. If I specify the number of workers or cpus to use then I also get this error:

ERROR: The process with PID 13440 could not be terminated.
Reason: The operation attempted is not supported.

Again the numbers after PID change each time the program is run. This does not occur if the program is not run in parallel. This error occurs after the program is finished running so it does not affect its functionality.

Any ideas how to avoid this error?

EDIT: here is a simple program that causes the error

import ppft

def main(x): return x

#Parallel set-up
ppservers = ('*',)      
job_server = ppft.Server(ppservers = ppservers)
ncpus = job_server.get_active_nodes()['local']
print('Starting Parallel Process with', ncpus, 'Workers')

jobs = [(input, job_server.submit(main,(input,), (), ())) for input in range(ncpus)]
#Run parallel jobs
for input, job in jobs:
    print(job())

This is the output:

C:UsersMarkDropboxVisual Studio ProjectsTestModuleTestModule>TestModule.py
Starting Parallel Process with 4 Workers
0
1
2
3

C:UsersMarkDropboxVisual Studio ProjectsTestModuleTestModule>ERROR: The process "10804" not found.
ERROR: The process "14300" not found.
ERROR: The process "3932" not found.
ERROR: The process "4548" not found.

Asked By: DivideByZero

||

Source

Answer 1

Credit for this answer goes to wprins from the Parallel Python Forum. I am posting it here as this webpage currently shows up first on google. From wprins’ answer:

PP works well with pypy (on Windows), however on shutdown, you get (harmless) failure messages from the invocation of ‘TASKKILL’ in pp.py line 823 (version 1.6.5) due to the child processes already being dead and absent at the time when the TASKKILL is called. This does not occur using normal CPython and I’ve not investigated why this is different between CPython and PyPy.

Nonetheless, since it is harmless, and the child processes are already gone, I’d like to suggest that the TASKKILL line is modified as follows to suppress the spurious error output:

The TASKKILL line causing this error message is in the file pp.py. In version 1.6.5 of pp it is line 823. In version 1.6.4.7.1 of ppft it is in line 874 (near the end of the file). The file pp.py in my case was in the directory “Python36Libsite-packagesppft-1.6.4.7.1-py3.6.eggpp.py”

To suppress the error message, change the line from:

os.popen('TASKKILL /PID '+str(worker.pid)+' /F')

to:

os.popen('TASKKILL /PID '+str(worker.pid)+' /F 2>NUL')

Answered By: DivideByZero

Error: "The process 'xxx' not found" occurs after executing parallel python code

Question:

Answers: