How to get fully sent data with socket.recv()?

Question

I have faced with some problems on python. I try to read sent data with socket.recv(1024), but sometime data is very long than 1024 bytes. I tried running this code:

data = b''
received = s.recv(1024)
    while len(received) > 0:
        data = data + received
        received = s.recv(1024)

But while statement causes code to run indefinitely. How can I read sent packets?

Asked By: TheMisir

||

Source

Answer 1

Here’s how you might handle this (untested):

MINIMUM_PACKET_LENGTH = 100
data = b''
while True:
    try:
        received = s.recv(1024)
    except:
        # you might put code here to handle the exception,
        # but otherwise:
        raise
    data = data + received
    while len(data) >= MINIMUM_PACKET_LENGTH:
        # consume a packet from the data buffer, but leave unused bytes in data
        packet = data[0:MINIMUM_PACKET_LENGTH]
        data = data[MINIMUM_PACKET_LENGTH:]
        # process packet (you could maybe use 'yield packet')
        # ....

Answered By: DisappointedByUnaccountableMod

Answer 2

After many years of writing various networking software, I decided to answer the question myself with more clarification hoping that it might be helpful for future readers.

In short the answer is – you don’t. You can not read the whole packet, because there’s no concept of packet in application protocol level.

A protocol is simply a way for both parties to communicate different set of messages. The "simplest" (I know of) protocol implementation would be something like this:

struct Packet {
  uint32_t packet_size;
  uint8_t  packet_body[];
}

The important part here is that we have a static sized header that defines dynamic properties of the payload – size in our case. Using this information the reader would first read 4 bytes of data and convert it to a uint32 n value[1], then try reading next n bytes from the socket.

[1] – the endianness of the value should be agreed upon beforehand on the protocol level by the client and the server.

Answered By: TheMisir

How to get fully sent data with socket.recv()?

Question:

Answers: