multiprocessing PicklingError in Python 3.6.5 that doesn't occur in 3.6.1
Question:
The following code works in Python 3.6.1 … but returns this error in Python 3.6.5:
Traceback (most recent call last):
File "G:/GOOD/Coding/Deepthroat/Deepthroat2/Bin/Testing/gh.py", line 36, in <module>
loops.start()
File "C:Program FilesPython36libmultiprocessingprocess.py", line 105, in start
self._popen = self._Popen(self)
File "C:Program FilesPython36libmultiprocessingcontext.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "C:Program FilesPython36libmultiprocessingcontext.py", line 322, in _Popen
return Popen(process_obj)
File "C:Program FilesPython36libmultiprocessingpopen_spawn_win32.py", line 65, in __init__
reduction.dump(process_obj, to_child)
File "C:Program FilesPython36libmultiprocessingreduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function loops at 0x0000019D3E30B598>: it's not the same object as __main__.loops
Here is the code:
from multiprocessing import Process
import time
def timeout(mm):
for i in range(6):
time.sleep(0.1)
print('stop all bla loops now')
mm.terminate()
return
def loops():
for i in range(5):
time.sleep(0.1)
print('loop1')
billy = 'loop1 done'
for i in range(5):
time.sleep(0.1)
print('loop2')
billy = 'loop2 done'
for i in range(5):
time.sleep(0.1)
print('loop3')
billy = 'loop3 done'
billy = 'cool'
loops = Process(target=loops)
loops.start()
timeout = Process(target=timeout(loops))
timeout.start()
Answers:
This is actually just an error by you because you are overwriting loops
, which means it can’t be pickled (pickle just stores a reference to the function). If you say this error didn’t occur in python3.6.1, than probably there are test missing in python3.6.1
If the target and variable both having same name like loops = Process(target=loops) then this error returns.
I could solve my problem using the explanations by Medalng and comments of Rhys in above answer. Re-iterated it again to help others as I could catch Rhys’s comment in third visit.
The following code works in Python 3.6.1 … but returns this error in Python 3.6.5:
Traceback (most recent call last):
File "G:/GOOD/Coding/Deepthroat/Deepthroat2/Bin/Testing/gh.py", line 36, in <module>
loops.start()
File "C:Program FilesPython36libmultiprocessingprocess.py", line 105, in start
self._popen = self._Popen(self)
File "C:Program FilesPython36libmultiprocessingcontext.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "C:Program FilesPython36libmultiprocessingcontext.py", line 322, in _Popen
return Popen(process_obj)
File "C:Program FilesPython36libmultiprocessingpopen_spawn_win32.py", line 65, in __init__
reduction.dump(process_obj, to_child)
File "C:Program FilesPython36libmultiprocessingreduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function loops at 0x0000019D3E30B598>: it's not the same object as __main__.loops
Here is the code:
from multiprocessing import Process
import time
def timeout(mm):
for i in range(6):
time.sleep(0.1)
print('stop all bla loops now')
mm.terminate()
return
def loops():
for i in range(5):
time.sleep(0.1)
print('loop1')
billy = 'loop1 done'
for i in range(5):
time.sleep(0.1)
print('loop2')
billy = 'loop2 done'
for i in range(5):
time.sleep(0.1)
print('loop3')
billy = 'loop3 done'
billy = 'cool'
loops = Process(target=loops)
loops.start()
timeout = Process(target=timeout(loops))
timeout.start()
This is actually just an error by you because you are overwriting loops
, which means it can’t be pickled (pickle just stores a reference to the function). If you say this error didn’t occur in python3.6.1, than probably there are test missing in python3.6.1
If the target and variable both having same name like loops = Process(target=loops) then this error returns.
I could solve my problem using the explanations by Medalng and comments of Rhys in above answer. Re-iterated it again to help others as I could catch Rhys’s comment in third visit.