Python multiprocessing example not working

It works.

I’ve marked the changes needed to make your sample run using comments:

from multiprocessing import Process

def f(name):
print 'hello', name #indent

if __name__ == '__main__':
    p = Process(target=f, args=('bob',))
    p.start()
    p.join()` # remove ` (grave accent)

result:

from multiprocessing import Process

def f(name):
    print 'hello', name

if __name__ == '__main__':
    p = Process(target=f, args=('bob',))
    p.start()
    p.join()

Output from my laptop after saving it as ex1.py:

reuts@reuts-K53SD:~/python_examples$ cat ex1.py 
#!/usr/bin/env python
from multiprocessing import Process

def f(name):
    print 'hello', name

if __name__ == '__main__':
    p = Process(target=f, args=('bob',))
    p.start()
    p.join()
reuts@reuts-K53SD:~/python_examples$ python ex1.py 
hello bob

Most likely your main process exits before sysout is flushed. Try this:

from multiprocessing import Process
import sys

def f(name):
    print 'hello', name

if __name__ == '__main__':
    p = Process(target=f, args=('bob',))
    p.start()
    p.join()
    # make sure all output has been processed before we exit
    sys.stdout.flush()

If this doesn’t work, try adding time.sleep(1) as the last statement.

I had the issue that multiprocessing did not work on Spyder, and always landed here. I solved it by using threading instead of multiprocessing. as described here: https://pymotw.com/2/threading/

import threading
def worker(num):
    """thread worker function"""
    print 'Worker: %s' % num
    return
threads = []
for i in range(5):
    t = threading.Thread(target=worker, args=(i,))
    threads.append(t)
    t.start()

The problem is not IDLE. The problem is trying to print to sys.stdout in a process that has no sys.stdout. That is why Spyder has the same problem. Any GUI program on Windows is likely to have the same problem.

On Windows, at least, GUI programs are usually run in a process without stdin, stdout, or stderr streams. Windows expects GUI programs to interact with users through widgets that paint pixels on the screen (the G in Graphical) and receive key and mouse events from Windows event system. That is what the IDLE GUI does, using the tkinter wrapper of the tcl tk GUI framework.

When IDLE runs user code in a subprocess, idlelib.run runs first, and it replaces None for the standard streams with objects that interact with IDLE itself through a socket. Then it exec()s user code. When the user code runs multiprocessing, multiprocessing starts further processes that have no std streams, but never get them.

The solution is to start IDLE in a console: python -m idlelib.idle (the .idle is not needed on 3.x). Processes started in a console get std streams connect to the console. So do further subprocesses. The real stdout (as opposed to the sys.stdout) of all the processes is the console. If one runs the third example in the doc,

from multiprocessing import Process
import os

def info(title):
    print(title)
    print('module name:', __name__)
    print('parent process:', os.getppid())
    print('process id:', os.getpid())

def f(name):
    info('function f')
    print('hello', name)

if __name__ == '__main__':
    info('main line')
    p = Process(target=f, args=('bob',))
    p.start()
    p.join()

then the ‘main line’ block goes to the IDLE shell and the ‘function f’ block goes to the console.

This result shows that Justin Barber’s claim that the user file run by IDLE cannot be imported into processes started by multiprocessing is not correct.

EDIT: Python saves the original stdout of a process in sys.__stdout__. Here is the result in IDLE’s shell when IDLE is started normally on Windows, as a pure GUI process.

>>> sys.__stdout__
>>> 

Here is the result when IDLE is started from CommandPrompt.

>>> import sys
>>> sys.__stdout__
<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>
>>> sys.__stdout__.fileno()
1

The standard file numbers for stdin, stdout, and stderr are 0, 1, 2. Run a file with

from multiprocessing import Process
import sys

def f(name):
    print('hello', name)
    print(sys.__stdout__)
    print(sys.__stdout__.fileno())
if __name__ == '__main__':
    p = Process(target=f, args=('bob',))
    p.start()
    p.join()

in IDLE started in the console and the output is the same.

Try using this code (from standard manual). Works for me on windows. Another one did not work for me either 🙂

import multiprocessing as mp

    def foo(q):
        q.put('hello')
    
    if __name__ == '__main__':
        mp.set_start_method('spawn')
        q = mp.Queue()
        p = mp.Process(target=foo, args=(q,))
        p.start()
        print(q.get())
        p.join()