How to use multithreading with and cv2.VideoCapture()?

Question

I want to display two or more separate videos with slight image processing applied using cv2.

If I don’t use threading/multiprocessing a lot of time is wasted just displaying frames using cv2.waitKey(timeThatCouldBeSpentFetchingAFrameForAnotherVideo).

I tried using threading, but it does not work for properly. I get this warning: WARNING: nextEventMatchingMask should only be called from the Main Thread! This will throw an exception in the future.. Which more often than not results in a plain crash.

So, I decided to try to implement multiprocessing, which in theory should be faster than threading with more complex image processing (right??). I used this as an example. However, in this case I ran into another issue this method ONLY gets me the first frame of the video. Also, I have tried putting the cv2.VideoCapture() outside of the Process, but that results in: TypeError: cannot pickle 'cv2.VideoCapture' object.

Here is my code:

class CamManager:
    def __init__(self):
        self.saving = Saving()
        self.cams = self.createCamObjList()
        self.buffers = [Buffer(cam.FPS*BUFFER_SIZE) for cam in self.cams]


    def createCamObjList(self):
        l = []
        for i, url in enumerate(STREAM_URLS):
            saved_data = self.saving.getDataFor(url)
            cam = BufferedCamera(url, i, saved_data)
            l.append(cam)
        return l


    def startStreamCapture(self):
        queue_for_cams = multiprocessing.Queue()
        processes = [multiprocessing.Process(
            target=cam.readloop, args=[queue_for_cams]).start() for cam in self.cams]
        while True:
            if not queue_for_cams.empty():
                from_queue = queue_for_cams.get()
                self.buffers[from_queue[0]].add(from_queue[1])



class BufferedCamera():
    def __init__(self, streamURL, cam_id, saved_data=None):
        self.streamURL = streamURL
        self.cam_id = cam_id

        # get current camera's framerate
        cam = cv2.VideoCapture(streamURL)
        self.FPS = cam.get(5)
        if self.FPS == 0:
            self.FPS = 24
        cam.release()
        print(f"Input res: {cam.get(3)}x{cam.get(4)} | FPS: {self.FPS}")

        # use that framerate to adjust the time between each iteration of mainloop
        self.period = int((1.0/self.FPS)*1000)


    def readloop(self, queue):
        while True:
            self.read(queue)

    
    def read(self, queue):
        cam = cv2.VideoCapture(self.streamURL)
        _, frame = cam.read()
        if frame is not None:
            queue.put((self.cam_id, frame))
        cv2.waitKey(self.period)

Asked By: Jonas

||

Source

Answer 1

I got multiprocessing to work by creating cv2.VideoCapture object inside my Process function and prior to the readloop:

def readloop(self, queue):
    cam = cv2.VideoCapture(self.streamURL)
    while True:
        self.read(queue, cam)

Answered By: Jonas

How to use multithreading with and cv2.VideoCapture()?

Question:

Answers: