How do I use multprocessing.Value (or other shared resources) on Mac with Spawn?

Question

Example:

import multiprocessing as mp

counter = mp.Value('i', 0)

def test_fun(i):
    global counter
    with counter.get_lock():
        counter.value += 1

def main():
    global counter
    with mp.Pool(4) as p:
        result = p.map(test_fun, range(4))
    print(counter.value)

if __name__ == "__main__":
    main()

The expected output is 4, since the value is shared but outputs 0 on Mac.

It works when using linux or using fork but I’d like it work with Spawn.

Asked By: thc

||

Source

Answer 1

just move counter declaration to your main function, then set it as a global variable inside the children initializer, same way it’s done on windows.

import multiprocessing as mp

def setup_fn(value):
    global counter
    counter = value

def test_fun(i):
    global counter
    with counter.get_lock():
        counter.value += 1

def main():
    counter = mp.Value('i', 0)
    with mp.Pool(4, initializer=setup_fn, initargs=(counter,)) as p:
        result = p.map(test_fun, range(4))
    print(counter.value)

if __name__ == "__main__":
    main()

Answered By: Ahmed AEK

How do I use multprocessing.Value (or other shared resources) on Mac with Spawn?

Question:

Answers: