After thread and process close, the MainThread still running

Question:

I’m a begineer in Python Parallel Programming, my problem is why my code still running after the thread and process close.

I’m try use this two function to check which is still alive :

  • print(f"thread : {threading.enumerate()}")
  • print(f"process : {multiprocessing.active_children()}")
    and it return
    thread : [<_MainThread(MainThread, started 15848)>, <Thread(QueueFeederThread, started daemon 5336)>]
    process : []
    But I don’t know how to close these two threads

here in my code:

import threading
import multiprocessing
import queue
import torch
import cv2
import time


class read_video(threading.Thread):
    def __init__(self, cv, lock, sync_lock, display_frame_queue, predict_frame_queue):
        super().__init__(name="SHOW_VIDEO")
        self.daemon = True
        self.cv = cv2.VideoCapture(cv)
        self.display_frame_queue = display_frame_queue
        self.lock = lock
        self.sync_lock = sync_lock
        self.predict_frame_queue = predict_frame_queue

    def read(self):

        self.sync_lock.wait()
        while True:
            if self.display_frame_queue.qsize() <= 3:
                ret, frame = self.cv.read()
                if ret:
                    frame = cv2.resize(frame, (1280, 720))
                    self.display_frame_queue.put(frame)
                    self.predict_frame_queue.put(frame)
                else:
                    break
            if self.lock.is_set():
                self.cv.release()
                break

        return

    def run(self):
        self.read()


class display_video(threading.Thread):
    def __init__(self, lock, sync_lock, display_frame_queue, predict_result_queue):
        super().__init__(name="READ_VIDEO")
        self.lock = lock
        self.daemon = True
        self.sync_lock = sync_lock
        self.display_frame_queue = display_frame_queue
        self.predict_result_queue = predict_result_queue

    def show(self):

        color = [(0, 255, 0), (255, 255, 0), (0, 255, 255), (0, 140, 255)]
        class_name = ['Car', 'Motorcycle', 'Person', 'Truck']
        self.sync_lock.wait()
        while True:
            frame = self.display_frame_queue.get()
            result = self.predict_result_queue.get()
            for i in result:
                object_type = int(i[-1])
                cv2.rectangle(frame, (int(i[0]), int(i[1])), (int(i[2]), int(i[3])), color[object_type], 1)
                cv2.putText(frame, f"{format(i[-2], '.2f')} {class_name[object_type]}", (int(i[0]), int(i[1] - 10)),
                            cv2.FONT_HERSHEY_SIMPLEX, 0.6, color[object_type], 2)
            cv2.imshow("vnaoisdf", frame)
            if cv2.waitKey(1) & 0xff == ord('c'):
                self.lock.set()
                cv2.destroyAllWindows()
                break

        return

    def run(self):
        self.show()


class Model(multiprocessing.Process):
    def __init__(self, model_path, lock, sync_lock, predict_frame_queue, predict_result_queue):
        super().__init__(name="MODEL")
        self.daemon = True
        self.model_path = model_path
        self.lock = lock
        self.sync_lock = sync_lock
        self.predict_frame_queue = predict_frame_queue
        self.predict_result_queue = predict_result_queue

    def predict(self):

        model = torch.hub.load('ultralytics/yolov5', 'custom', path=self.model_path)
        model.cuda()
        self.sync_lock.wait()
        while True:
            if not self.predict_frame_queue.empty():
                frame = self.predict_frame_queue.get()
                predict = model(frame).xyxy[0].cpu().numpy()
                self.predict_result_queue.put(predict)
            if self.lock.is_set():
                break
        self.close()

        return

    def run(self):
        self.predict()


if __name__ == "__main__":
    lock = multiprocessing.Event()
    sync_lock = multiprocessing.Barrier(3)
    display_queue = queue.Queue()
    predict_frame_queue = multiprocessing.Queue()
    predict_result_queue = multiprocessing.Queue()

    a = read_video("testingv.mp4", lock, sync_lock, display_queue, predict_frame_queue)
    b = display_video(lock, sync_lock, display_queue, predict_result_queue)
    c = Model("best200.pt", lock, sync_lock, predict_frame_queue, predict_result_queue)
    a.start()
    b.start()
    c.start()

    a.join()
    b.join()
    c.join()

    predict_frame_queue.close()
    predict_result_queue.close()
    c.terminate()

    print(f"thread : {threading.enumerate()}")
    print(f"process : {multiprocessing.active_children()}")

Can anyone tell me what the problem is and how to fix it

Asked By: BON

||

Answers:

TL;DR

First some comments:

  1. You have made the read_video thread, the display_video thread and the Model process all daemon yet your main process waits for their completion by calling join. Then nothing is gained by making the threads and process dameon. When a thread or process puts items on a multiprocessing.Queue instance a hidden thread is started that manages the actual putting of the items on the queue, which is built from a multiprocessing.Pipe that has a limited capacity. This prevents the writing process or thread from blocking on a put call (it will be this hidden thread that will block instead).
  2. Following the call to c.join() you execute c.terminate(). But the process, by having already called join, has already been terminated. So this seems pointless.
  3. Your main process issues calls to close on the multiprocessing.Queue instances. This call is normally made by "writers" to the queue to signify that no more items will be placed on the queue. But your main process is not the one putting items on these queues and so calling close should not be its responsibility. It is also a call that no thread or process really needs to make. It just allows the (hidden) thread that is managing the queue to terminate once the queue becomes empty. This will occur anyway as soon as the queue is garbage collected.
  4. In my opinion (others may disagree), it is better in general not to subclass the threading.Thread and multiprocessing.Process classes since the logic you have implemented in those subclasses are now too tightly coupled to threads and processes. I think it is better, for example, to have Model just be a subclass of object (implicitly or explicitly) with the run method removed and to then do m = Model(...); c = Process(target=m.predict).

As for the problem you may be having: When a thread or process has put items on a multiprocessing.Queue instance, it cannot terminate until those items have been retrieved by some reading thread or process. So you must never try to join such a writer thread/process until the queue it has written to is empty or will be empty.

The read_video queue is a "writer" to the predict_frame_queue and the Model process is a writer to the predict_result_queue. The display_video thread is the "reader" of both of these queues. If ‘c’ is typed in the video window then the display_video.show calls self.lock.set() to signal that it is finished and then it returns. But has it possibly left any items on either of the multiprocessing queues? If so, this is a problem because the main process is attempting to join the Model process and read_video thread but these will not terminate as long as there are unread items on them. Likewise, method Model.predict returns when it sees that self.lock has been set and it, too, may have not emptied the predict_frame_queue. It is true that it does not check self.lock until the call self.predict_frame_queue_empty() returns True. But if you read the documentation on the queue_empty method you will see that this call is unreliable. Moreover, even if it were reliable, you would have a race condition arising the the period of time that exists between finding the queue empty and returning for in that small window of time it is possible that another item has been placed on the queue.

Now I cannot be sure that this is what is causing your hang. You may have another issue that I have not spotted. But at the very least, your multiprocessing queue readers must ensure that they do not return until they are certain they have read all the messages if you will be joining these threads and process in the main process.

Try This

Keep your threads and process as daemon, but do not do join calls on these in the main process. Instead, the main process also waits on the termination event:

if __name__ == "__main__":
    lock = multiprocessing.Event()
    sync_lock = multiprocessing.Barrier(3)
    display_queue = queue.Queue()
    predict_frame_queue = multiprocessing.Queue()
    predict_result_queue = multiprocessing.Queue()

    a = read_video("testingv.mp4", lock, sync_lock, display_queue, predict_frame_queue)
    b = display_video(lock, sync_lock, display_queue, predict_result_queue)
    c = Model("best200.pt", lock, sync_lock, predict_frame_queue, predict_result_queue)
    a.start()
    b.start()
    c.start()

    # Wait for this event:
    lock_event.wait()
    # Threads and process will still be running but will terminate
    # when we do.

Alternative

You can remove the daemon attribute from your process and threads and then make sure that no "readers" of a multiprocessing.Queue (the Model process doing get calls on predict_frame_result and the display_video thread doing get calls on predict_result_queue) are terminating before first getting all items from the queue. The safest way to do this is to have the writers of these queues (read_video and Model respectively) put a special sentinel item (None can be used if None cannot be a "normal" data item) indicating that this is the last item that will ever be put on the queue. Then the readers enter a loop getting items from the queue until it sees the sentinel item.

Model.predict should be:

 def predict(self):

        ...
        frame = '' # anything other than a sentinel value
        while not self.lock_is_set():
            frame = self.predict_frame_queue.get()
            # Sentinel?
            if frame is None:
                break
            predict = model(frame).xyxy[0].cpu().numpy()
            self.predict_result_queue.put(predict)

        # Put a sentinel item:
        predict_result_queue.put(None)
        # Empty the queue if we haven't seem the sentinel:
        while frame is not None:
            frame = self.predict_frame_queue.get()
        #self.close() # What is this?

        return

And read_video.read should be:

    def read(self):
        ...

        # Put sentinels:
        self.predict_frame_result.put(None)
        self.result_frame_result.put(None)

        return

And display_video.show should be:

    def show(self):
        ...

            if cv2.waitKey(1) & 0xff == ord('c'):
                self.lock.set()
                cv2.destroyAllWindows()
                # Empty queues:
                while display_frame_queue.get() is not None:
                    pass                    
                while predict_result_queue.get() is not None:
                    pass                    
                break

        return
Answered By: Booboo

I’m not sure about the first answer. for me I managed to terminate the queue thread by calling these lines of code to ensure the queue’s thread is terminated

    while self.images_queue.qsize() > 0:
        self.images_queue.get()

    del self.images_queue

This is the only thing that worked for me. emptying the queue alone didn’t work and also del the queue alone didn’t

Answered By: Abdelsalam Hamdi