Global variable not accessed inside pool(multiprocessing) in python

Question

The global variable defined in main is accessed inside normal function but its not accessed in Pool()

from multiprocessing import Pool    
import functools


def inc(x):
    print( x + 1)
    
    print(headers) # GETTING ERROR HERE

def dec(x):
    print (x - 1)
    

def add(x, y):
    print (x + y)
    
    
    
def a(f):
    f()
    


def main():
    print(headers)
    
    f_inc = functools.partial(inc, 4)
    f_dec = functools.partial(dec, 2)
    f_add = functools.partial(add, 3, 4)

    
    with Pool() as pool:
        res = pool.map(a, [f_inc, f_dec, f_add])

    print(res)


if __name__ == '__main__':
    global headers
    headers = {'Accept': 'application/json'}
    main()

Expected output is

{'Accept': 'application/json'}
5
{'Accept': 'application/json'}
1
7
None

But the output i get is

{'Accept': 'application/json'}
    5

NameError: name 'headers' is not defined

The global variable is not getting accessed inside pool in multiprocessing.

Asked By: Madan Raj

||

Source

Answer 1

You can’t access it in separate pool processes because each process has started its own Python interpreter, and the global variable isn’t in it.

You need to pass the variables you need to each pool process. There are a number of ways to communicate between Processes. One of them is a multiprocessing.Queue(), and there are other ways too. There are many examples in the Python.org docs that show how to do this here.

Answered By: akaButters

Global variable not accessed inside pool(multiprocessing) in python

Question:

Answers: