Python video plays and hangs after 1 second with socket program

Question

I am trying to write a python socket program which will do the following:

A server program starts and plays a video
The timestamp of the video is sent continuously to a socket
On the client side, the client will get the value of the timestamp from the socket message

Server Code:

    import cv2
    import socket

    s = socket.socket()
    print ("Socket successfully created")
    port = 12345
    
    cap=cv2.VideoCapture("D:/video.mp4")
    if cap.isOpened()==False:
        print("Error opening")

    s.bind(('', port))
    print ("socket bound to %s" %(port))
    s.setblocking(1)
    s.listen(5)
    print ("socket is listening")
    try:
        while cap.isOpened():
            ret,frame=cap.read()

            if ret==True:
                cv2.imshow("Frame",frame)
                while True:
                    c, addr = s.accept()
                    print ('Got connection from', addr )
                    c.sendall(str(cap.get(cv2.CAP_PROP_POS_FRAMES)).encode("utf=8"))
            else:
                cap.set(cv2.CAP_PROP_POS_FRAMES,0)
                continue
            c.close()
    except Exception as e:
        print(e)
    cap.release()

Client Code:

# Import socket module
import socket

# Create a socket object
s = socket.socket()

# Define the port on which you want to connect
port = 12345

# connect to the server on local computer
s.connect(('localhost', port))

# receive data from the server and decoding to get the string.
print (s.recv(16).decode())
# close the connection
s.close()

When running this code, the video starts and then hangs immediately. Running the client program gives the output 1.0 . Running it multiple times always gives the same output.

I am new to socket programming and my guess is that this is due to the blocking nature of the sockets. However, setting s.setblocking(0) results in an exception

BlockingIOError: [WinError 10035] A non-blocking socket operation could not be completed immediately

Without any of the socket code, the video runs fine and I can print the timestamp to the console.

How do I fix the issue? Any help is greatly appreciated

Asked By: Monty Swanson

||

Source

Answer 1

accept() blocks code because it waits for client.
So you have to run opencv or socket in separated threads. And you will have to send data to client in separated thread.

You should run accept() before while-loop because current code needs new connection to send new data to the same client.

Your code will have another problem. socket is very primitive and it may send data splitted into many parts – and client will have to run loop to get all data – or it may send many packages connected in one package – and client will have to crop data to correct size. But socket doesn’t inform where is end of every data so client will not know which part display as first timestamp and which as next timestamp. You may have to send always string with the same size – and client will have to get always string with the same size. Or it would need to send size before data (and size would be data with constant size) and client would have to first get size and later use it to get correct data.

client.py

import socket

PORT = 12345

s = socket.socket()
s.connect(('', PORT))

buffer = ""
while True:
    buffer += s.recv(20).decode()
    if len(buffer) >= 20:   # server sends every timestamp as 20 chars
        text = buffer[:20]
        print(text, end='r')
        buffer = buffer[20:]

s.close()

server.py

import cv2
import socket
import threading

def player():
    global timestamp
    
    cap = cv2.VideoCapture("D:/video.mp4")
    #cap = cv2.VideoCapture(0)  # local webcam - but it has timestamp `-1.0`
    if not cap.isOpened():
        print("Error opening")

    while cap.isOpened():
        ret, frame = cap.read()

        if ret:
            cv2.imshow("Frame",frame)
            if cv2.waitKey(1) & 0xff == 27:  # key ESC
                break
            timestamp = cap.get(cv2.CAP_PROP_POS_FRAMES)
        else:
            cap.set(cv2.CAP_PROP_POS_FRAMES, 0)
            timestamp = 0

    cv2.destroyAllWindows()
    cap.release()

def process_client(c, addr):
   
    # run loop to send it continuously
    try:
        while True:
            text = f'{timestamp:20}'  # create string always with 20 chars
            c.sendall(text.encode("utf-8"))
    except Exception as ex:
        print('Exception:', ex)
        
    c.close()
    
def server():

    PORT = 12345
    
    threads = []
    
    s = socket.socket()
    print("Socket successfully created")
    
    s.bind(('', PORT))
    print(f"socket bound to {PORT}")
    
    #s.setblocking(True)
    s.listen(5)
    print("socket is listening")
    
    try:
        while True:
            print('Waiting for next client ...')
            c, addr = s.accept()
            print('Got connection from', addr)
            
            # process every client in separated thread
            t = threading.Thread(target=process_client, args=(c, addr))
            t.start()
            
            threads.append([c, addr, t])
    except Exception as ex:
        print('Exception:', ex)
        
    for c, addr, t in threads:
        c.close()
        
    s.close()
    
# --- main ---

timestamp = 0

# run server in separated thread
t = threading.Thread(target=server)
t.start()

player()

BTW:

HTTP uses other method to send headers with unknown size before data. It sends empty line at the end of headers – two new lines nn – and client checks if buffer has two nn to crop data in this places.

Answered By: furas

Python video plays and hangs after 1 second with socket program

Question:

Answers: