After thread and process close, the MainThread still running
Question:
I’m a begineer in Python Parallel Programming, my problem is why my code still running after the thread and process close.
I’m try use this two function to check which is still alive :
- print(f"thread : {threading.enumerate()}")
- print(f"process : {multiprocessing.active_children()}")
and it return
thread : [<_MainThread(MainThread, started 15848)>, <Thread(QueueFeederThread, started daemon 5336)>]
process : []
But I don’t know how to close these two threads
here in my code:
import threading
import multiprocessing
import queue
import torch
import cv2
import time
class read_video(threading.Thread):
def __init__(self, cv, lock, sync_lock, display_frame_queue, predict_frame_queue):
super().__init__(name="SHOW_VIDEO")
self.daemon = True
self.cv = cv2.VideoCapture(cv)
self.display_frame_queue = display_frame_queue
self.lock = lock
self.sync_lock = sync_lock
self.predict_frame_queue = predict_frame_queue
def read(self):
self.sync_lock.wait()
while True:
if self.display_frame_queue.qsize() <= 3:
ret, frame = self.cv.read()
if ret:
frame = cv2.resize(frame, (1280, 720))
self.display_frame_queue.put(frame)
self.predict_frame_queue.put(frame)
else:
break
if self.lock.is_set():
self.cv.release()
break
return
def run(self):
self.read()
class display_video(threading.Thread):
def __init__(self, lock, sync_lock, display_frame_queue, predict_result_queue):
super().__init__(name="READ_VIDEO")
self.lock = lock
self.daemon = True
self.sync_lock = sync_lock
self.display_frame_queue = display_frame_queue
self.predict_result_queue = predict_result_queue
def show(self):
color = [(0, 255, 0), (255, 255, 0), (0, 255, 255), (0, 140, 255)]
class_name = ['Car', 'Motorcycle', 'Person', 'Truck']
self.sync_lock.wait()
while True:
frame = self.display_frame_queue.get()
result = self.predict_result_queue.get()
for i in result:
object_type = int(i[-1])
cv2.rectangle(frame, (int(i[0]), int(i[1])), (int(i[2]), int(i[3])), color[object_type], 1)
cv2.putText(frame, f"{format(i[-2], '.2f')} {class_name[object_type]}", (int(i[0]), int(i[1] - 10)),
cv2.FONT_HERSHEY_SIMPLEX, 0.6, color[object_type], 2)
cv2.imshow("vnaoisdf", frame)
if cv2.waitKey(1) & 0xff == ord('c'):
self.lock.set()
cv2.destroyAllWindows()
break
return
def run(self):
self.show()
class Model(multiprocessing.Process):
def __init__(self, model_path, lock, sync_lock, predict_frame_queue, predict_result_queue):
super().__init__(name="MODEL")
self.daemon = True
self.model_path = model_path
self.lock = lock
self.sync_lock = sync_lock
self.predict_frame_queue = predict_frame_queue
self.predict_result_queue = predict_result_queue
def predict(self):
model = torch.hub.load('ultralytics/yolov5', 'custom', path=self.model_path)
model.cuda()
self.sync_lock.wait()
while True:
if not self.predict_frame_queue.empty():
frame = self.predict_frame_queue.get()
predict = model(frame).xyxy[0].cpu().numpy()
self.predict_result_queue.put(predict)
if self.lock.is_set():
break
self.close()
return
def run(self):
self.predict()
if __name__ == "__main__":
lock = multiprocessing.Event()
sync_lock = multiprocessing.Barrier(3)
display_queue = queue.Queue()
predict_frame_queue = multiprocessing.Queue()
predict_result_queue = multiprocessing.Queue()
a = read_video("testingv.mp4", lock, sync_lock, display_queue, predict_frame_queue)
b = display_video(lock, sync_lock, display_queue, predict_result_queue)
c = Model("best200.pt", lock, sync_lock, predict_frame_queue, predict_result_queue)
a.start()
b.start()
c.start()
a.join()
b.join()
c.join()
predict_frame_queue.close()
predict_result_queue.close()
c.terminate()
print(f"thread : {threading.enumerate()}")
print(f"process : {multiprocessing.active_children()}")
Can anyone tell me what the problem is and how to fix it
Answers:
TL;DR
First some comments:
- You have made the
read_video
thread, the display_video
thread and the Model
process all daemon yet your main process waits for their completion by calling join
. Then nothing is gained by making the threads and process dameon. When a thread or process puts items on a multiprocessing.Queue
instance a hidden thread is started that manages the actual putting of the items on the queue, which is built from a multiprocessing.Pipe
that has a limited capacity. This prevents the writing process or thread from blocking on a put
call (it will be this hidden thread that will block instead).
- Following the call to
c.join()
you execute c.terminate()
. But the process, by having already called join
, has already been terminated. So this seems pointless.
- Your main process issues calls to
close
on the multiprocessing.Queue
instances. This call is normally made by "writers" to the queue to signify that no more items will be placed on the queue. But your main process is not the one putting items on these queues and so calling close
should not be its responsibility. It is also a call that no thread or process really needs to make. It just allows the (hidden) thread that is managing the queue to terminate once the queue becomes empty. This will occur anyway as soon as the queue is garbage collected.
- In my opinion (others may disagree), it is better in general not to subclass the
threading.Thread
and multiprocessing.Process
classes since the logic you have implemented in those subclasses are now too tightly coupled to threads and processes. I think it is better, for example, to have Model
just be a subclass of object
(implicitly or explicitly) with the run
method removed and to then do m = Model(...); c = Process(target=m.predict)
.
As for the problem you may be having: When a thread or process has put items on a multiprocessing.Queue
instance, it cannot terminate until those items have been retrieved by some reading thread or process. So you must never try to join
such a writer thread/process until the queue it has written to is empty or will be empty.
The read_video
queue is a "writer" to the predict_frame_queue
and the Model
process is a writer to the predict_result_queue
. The display_video
thread is the "reader" of both of these queues. If ‘c’ is typed in the video window then the display_video.show
calls self.lock.set()
to signal that it is finished and then it returns. But has it possibly left any items on either of the multiprocessing queues? If so, this is a problem because the main process is attempting to join
the Model
process and read_video
thread but these will not terminate as long as there are unread items on them. Likewise, method Model.predict
returns when it sees that self.lock
has been set and it, too, may have not emptied the predict_frame_queue
. It is true that it does not check self.lock
until the call self.predict_frame_queue_empty()
returns True
. But if you read the documentation on the queue_empty
method you will see that this call is unreliable. Moreover, even if it were reliable, you would have a race condition arising the the period of time that exists between finding the queue empty and returning for in that small window of time it is possible that another item has been placed on the queue.
Now I cannot be sure that this is what is causing your hang. You may have another issue that I have not spotted. But at the very least, your multiprocessing queue readers must ensure that they do not return until they are certain they have read all the messages if you will be joining these threads and process in the main process.
Try This
Keep your threads and process as daemon, but do not do join
calls on these in the main process. Instead, the main process also waits on the termination event:
if __name__ == "__main__":
lock = multiprocessing.Event()
sync_lock = multiprocessing.Barrier(3)
display_queue = queue.Queue()
predict_frame_queue = multiprocessing.Queue()
predict_result_queue = multiprocessing.Queue()
a = read_video("testingv.mp4", lock, sync_lock, display_queue, predict_frame_queue)
b = display_video(lock, sync_lock, display_queue, predict_result_queue)
c = Model("best200.pt", lock, sync_lock, predict_frame_queue, predict_result_queue)
a.start()
b.start()
c.start()
# Wait for this event:
lock_event.wait()
# Threads and process will still be running but will terminate
# when we do.
Alternative
You can remove the daemon
attribute from your process and threads and then make sure that no "readers" of a multiprocessing.Queue (the Model
process doing get
calls on predict_frame_result
and the display_video
thread doing get
calls on predict_result_queue
) are terminating before first getting all items from the queue. The safest way to do this is to have the writers of these queues (read_video
and Model
respectively) put a special sentinel item (None
can be used if None
cannot be a "normal" data item) indicating that this is the last item that will ever be put on the queue. Then the readers enter a loop getting items from the queue until it sees the sentinel item.
Model.predict
should be:
def predict(self):
...
frame = '' # anything other than a sentinel value
while not self.lock_is_set():
frame = self.predict_frame_queue.get()
# Sentinel?
if frame is None:
break
predict = model(frame).xyxy[0].cpu().numpy()
self.predict_result_queue.put(predict)
# Put a sentinel item:
predict_result_queue.put(None)
# Empty the queue if we haven't seem the sentinel:
while frame is not None:
frame = self.predict_frame_queue.get()
#self.close() # What is this?
return
And read_video.read
should be:
def read(self):
...
# Put sentinels:
self.predict_frame_result.put(None)
self.result_frame_result.put(None)
return
And display_video.show
should be:
def show(self):
...
if cv2.waitKey(1) & 0xff == ord('c'):
self.lock.set()
cv2.destroyAllWindows()
# Empty queues:
while display_frame_queue.get() is not None:
pass
while predict_result_queue.get() is not None:
pass
break
return
I’m not sure about the first answer. for me I managed to terminate the queue thread by calling these lines of code to ensure the queue’s thread is terminated
while self.images_queue.qsize() > 0:
self.images_queue.get()
del self.images_queue
This is the only thing that worked for me. emptying the queue alone didn’t work and also del the queue alone didn’t
I’m a begineer in Python Parallel Programming, my problem is why my code still running after the thread and process close.
I’m try use this two function to check which is still alive :
- print(f"thread : {threading.enumerate()}")
- print(f"process : {multiprocessing.active_children()}")
and it return
thread : [<_MainThread(MainThread, started 15848)>, <Thread(QueueFeederThread, started daemon 5336)>]
process : []
But I don’t know how to close these two threads
here in my code:
import threading
import multiprocessing
import queue
import torch
import cv2
import time
class read_video(threading.Thread):
def __init__(self, cv, lock, sync_lock, display_frame_queue, predict_frame_queue):
super().__init__(name="SHOW_VIDEO")
self.daemon = True
self.cv = cv2.VideoCapture(cv)
self.display_frame_queue = display_frame_queue
self.lock = lock
self.sync_lock = sync_lock
self.predict_frame_queue = predict_frame_queue
def read(self):
self.sync_lock.wait()
while True:
if self.display_frame_queue.qsize() <= 3:
ret, frame = self.cv.read()
if ret:
frame = cv2.resize(frame, (1280, 720))
self.display_frame_queue.put(frame)
self.predict_frame_queue.put(frame)
else:
break
if self.lock.is_set():
self.cv.release()
break
return
def run(self):
self.read()
class display_video(threading.Thread):
def __init__(self, lock, sync_lock, display_frame_queue, predict_result_queue):
super().__init__(name="READ_VIDEO")
self.lock = lock
self.daemon = True
self.sync_lock = sync_lock
self.display_frame_queue = display_frame_queue
self.predict_result_queue = predict_result_queue
def show(self):
color = [(0, 255, 0), (255, 255, 0), (0, 255, 255), (0, 140, 255)]
class_name = ['Car', 'Motorcycle', 'Person', 'Truck']
self.sync_lock.wait()
while True:
frame = self.display_frame_queue.get()
result = self.predict_result_queue.get()
for i in result:
object_type = int(i[-1])
cv2.rectangle(frame, (int(i[0]), int(i[1])), (int(i[2]), int(i[3])), color[object_type], 1)
cv2.putText(frame, f"{format(i[-2], '.2f')} {class_name[object_type]}", (int(i[0]), int(i[1] - 10)),
cv2.FONT_HERSHEY_SIMPLEX, 0.6, color[object_type], 2)
cv2.imshow("vnaoisdf", frame)
if cv2.waitKey(1) & 0xff == ord('c'):
self.lock.set()
cv2.destroyAllWindows()
break
return
def run(self):
self.show()
class Model(multiprocessing.Process):
def __init__(self, model_path, lock, sync_lock, predict_frame_queue, predict_result_queue):
super().__init__(name="MODEL")
self.daemon = True
self.model_path = model_path
self.lock = lock
self.sync_lock = sync_lock
self.predict_frame_queue = predict_frame_queue
self.predict_result_queue = predict_result_queue
def predict(self):
model = torch.hub.load('ultralytics/yolov5', 'custom', path=self.model_path)
model.cuda()
self.sync_lock.wait()
while True:
if not self.predict_frame_queue.empty():
frame = self.predict_frame_queue.get()
predict = model(frame).xyxy[0].cpu().numpy()
self.predict_result_queue.put(predict)
if self.lock.is_set():
break
self.close()
return
def run(self):
self.predict()
if __name__ == "__main__":
lock = multiprocessing.Event()
sync_lock = multiprocessing.Barrier(3)
display_queue = queue.Queue()
predict_frame_queue = multiprocessing.Queue()
predict_result_queue = multiprocessing.Queue()
a = read_video("testingv.mp4", lock, sync_lock, display_queue, predict_frame_queue)
b = display_video(lock, sync_lock, display_queue, predict_result_queue)
c = Model("best200.pt", lock, sync_lock, predict_frame_queue, predict_result_queue)
a.start()
b.start()
c.start()
a.join()
b.join()
c.join()
predict_frame_queue.close()
predict_result_queue.close()
c.terminate()
print(f"thread : {threading.enumerate()}")
print(f"process : {multiprocessing.active_children()}")
Can anyone tell me what the problem is and how to fix it
TL;DR
First some comments:
- You have made the
read_video
thread, thedisplay_video
thread and theModel
process all daemon yet your main process waits for their completion by callingjoin
. Then nothing is gained by making the threads and process dameon. When a thread or process puts items on amultiprocessing.Queue
instance a hidden thread is started that manages the actual putting of the items on the queue, which is built from amultiprocessing.Pipe
that has a limited capacity. This prevents the writing process or thread from blocking on aput
call (it will be this hidden thread that will block instead). - Following the call to
c.join()
you executec.terminate()
. But the process, by having already calledjoin
, has already been terminated. So this seems pointless. - Your main process issues calls to
close
on themultiprocessing.Queue
instances. This call is normally made by "writers" to the queue to signify that no more items will be placed on the queue. But your main process is not the one putting items on these queues and so callingclose
should not be its responsibility. It is also a call that no thread or process really needs to make. It just allows the (hidden) thread that is managing the queue to terminate once the queue becomes empty. This will occur anyway as soon as the queue is garbage collected. - In my opinion (others may disagree), it is better in general not to subclass the
threading.Thread
andmultiprocessing.Process
classes since the logic you have implemented in those subclasses are now too tightly coupled to threads and processes. I think it is better, for example, to haveModel
just be a subclass ofobject
(implicitly or explicitly) with therun
method removed and to then dom = Model(...); c = Process(target=m.predict)
.
As for the problem you may be having: When a thread or process has put items on a multiprocessing.Queue
instance, it cannot terminate until those items have been retrieved by some reading thread or process. So you must never try to join
such a writer thread/process until the queue it has written to is empty or will be empty.
The read_video
queue is a "writer" to the predict_frame_queue
and the Model
process is a writer to the predict_result_queue
. The display_video
thread is the "reader" of both of these queues. If ‘c’ is typed in the video window then the display_video.show
calls self.lock.set()
to signal that it is finished and then it returns. But has it possibly left any items on either of the multiprocessing queues? If so, this is a problem because the main process is attempting to join
the Model
process and read_video
thread but these will not terminate as long as there are unread items on them. Likewise, method Model.predict
returns when it sees that self.lock
has been set and it, too, may have not emptied the predict_frame_queue
. It is true that it does not check self.lock
until the call self.predict_frame_queue_empty()
returns True
. But if you read the documentation on the queue_empty
method you will see that this call is unreliable. Moreover, even if it were reliable, you would have a race condition arising the the period of time that exists between finding the queue empty and returning for in that small window of time it is possible that another item has been placed on the queue.
Now I cannot be sure that this is what is causing your hang. You may have another issue that I have not spotted. But at the very least, your multiprocessing queue readers must ensure that they do not return until they are certain they have read all the messages if you will be joining these threads and process in the main process.
Try This
Keep your threads and process as daemon, but do not do join
calls on these in the main process. Instead, the main process also waits on the termination event:
if __name__ == "__main__":
lock = multiprocessing.Event()
sync_lock = multiprocessing.Barrier(3)
display_queue = queue.Queue()
predict_frame_queue = multiprocessing.Queue()
predict_result_queue = multiprocessing.Queue()
a = read_video("testingv.mp4", lock, sync_lock, display_queue, predict_frame_queue)
b = display_video(lock, sync_lock, display_queue, predict_result_queue)
c = Model("best200.pt", lock, sync_lock, predict_frame_queue, predict_result_queue)
a.start()
b.start()
c.start()
# Wait for this event:
lock_event.wait()
# Threads and process will still be running but will terminate
# when we do.
Alternative
You can remove the daemon
attribute from your process and threads and then make sure that no "readers" of a multiprocessing.Queue (the Model
process doing get
calls on predict_frame_result
and the display_video
thread doing get
calls on predict_result_queue
) are terminating before first getting all items from the queue. The safest way to do this is to have the writers of these queues (read_video
and Model
respectively) put a special sentinel item (None
can be used if None
cannot be a "normal" data item) indicating that this is the last item that will ever be put on the queue. Then the readers enter a loop getting items from the queue until it sees the sentinel item.
Model.predict
should be:
def predict(self):
...
frame = '' # anything other than a sentinel value
while not self.lock_is_set():
frame = self.predict_frame_queue.get()
# Sentinel?
if frame is None:
break
predict = model(frame).xyxy[0].cpu().numpy()
self.predict_result_queue.put(predict)
# Put a sentinel item:
predict_result_queue.put(None)
# Empty the queue if we haven't seem the sentinel:
while frame is not None:
frame = self.predict_frame_queue.get()
#self.close() # What is this?
return
And read_video.read
should be:
def read(self):
...
# Put sentinels:
self.predict_frame_result.put(None)
self.result_frame_result.put(None)
return
And display_video.show
should be:
def show(self):
...
if cv2.waitKey(1) & 0xff == ord('c'):
self.lock.set()
cv2.destroyAllWindows()
# Empty queues:
while display_frame_queue.get() is not None:
pass
while predict_result_queue.get() is not None:
pass
break
return
I’m not sure about the first answer. for me I managed to terminate the queue thread by calling these lines of code to ensure the queue’s thread is terminated
while self.images_queue.qsize() > 0:
self.images_queue.get()
del self.images_queue
This is the only thing that worked for me. emptying the queue alone didn’t work and also del the queue alone didn’t