What does Python's socket.recv() return for non-blocking sockets if no data is received until a timeout occurs?
Question:
Basically, I’ve read in several places that socket.recv() will return whatever it can read, or an empty string signalling that the other side has shut down (the official docs don’t even mention what it returns when the connection is shut down… great!). This is all fine and dandy for blocking sockets, since we know that recv() only returns when there actually is something to receive, so when it returns an empty string, it MUST mean the other side has closed the connection, right?
Okay, fine, but what happens when my socket is non-blocking?? I have searched a bit (maybe not enough, who knows?) and can’t figure out how to tell when the other side has closed the connection using a non-blocking socket. There seems to be no method or attribute that tells us this, and comparing the return value of recv() to the empty string seems absolutely useless… is it just me having this problem?
As a simple example, let’s say my socket’s timeout is set to 1.2342342 (whatever non-negative number you like here) seconds and I call socket.recv(1024), but the other side doesn’t send anything during that 1.2342342 second period. The recv() call will return an empty string and I have no clue as to whether the connection is still standing or not…
Answers:
When you use recv in connection with select if the socket is ready to be read from but there is no data to read that means the client has closed the connection.
Here is some code that handles this, also note the exception that is thrown when recv is called a second time in the while loop. If there is nothing left to read this exception will be thrown it doesn’t mean the client has closed the connection :
def listenToSockets(self):
while True:
changed_sockets = self.currentSockets
ready_to_read, ready_to_write, in_error = select.select(changed_sockets, [], [], 0.1)
for s in ready_to_read:
if s == self.serverSocket:
self.acceptNewConnection(s)
else:
self.readDataFromSocket(s)
And the function that receives the data :
def readDataFromSocket(self, socket):
data = ''
buffer = ''
try:
while True:
data = socket.recv(4096)
if not data:
break
buffer += data
except error, (errorCode,message):
# error 10035 is no data available, it is non-fatal
if errorCode != 10035:
print 'socket.error - ('+str(errorCode)+') ' + message
if data:
print 'received '+ buffer
else:
print 'disconnected'
In the case of a non blocking socket that has no data available, recv will throw the socket.error exception and the value of the exception will have the errno of either EAGAIN or EWOULDBLOCK. Example:
import sys
import socket
import fcntl, os
import errno
from time import sleep
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('127.0.0.1',9999))
fcntl.fcntl(s, fcntl.F_SETFL, os.O_NONBLOCK)
while True:
try:
msg = s.recv(4096)
except socket.error, e:
err = e.args[0]
if err == errno.EAGAIN or err == errno.EWOULDBLOCK:
sleep(1)
print 'No data available'
continue
else:
# a "real" error occurred
print e
sys.exit(1)
else:
# got a message, do something :)
The situation is a little different in the case where you’ve enabled non-blocking behavior via a time out with socket.settimeout(n) or socket.setblocking(False). In this case a socket.error is stil raised, but in the case of a time out, the accompanying value of the exception is always a string set to ‘timed out’. So, to handle this case you can do:
import sys
import socket
from time import sleep
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('127.0.0.1',9999))
s.settimeout(2)
while True:
try:
msg = s.recv(4096)
except socket.timeout, e:
err = e.args[0]
# this next if/else is a bit redundant, but illustrates how the
# timeout exception is setup
if err == 'timed out':
sleep(1)
print 'recv timed out, retry later'
continue
else:
print e
sys.exit(1)
except socket.error, e:
# Something else happened, handle error, exit, etc.
print e
sys.exit(1)
else:
if len(msg) == 0:
print 'orderly shutdown on server end'
sys.exit(0)
else:
# got a message do something :)
As indicated in the comments, this is also a more portable solution since it doesn’t depend on OS specific functionality to put the socket into non-blockng mode.
See recv(2) and python socket for more details.
It is simple: if recv() returns 0 bytes; you will not receive any more data on this connection. Ever. You still might be able to send.
It means that your non-blocking socket have to raise an exception (it might be system-dependent) if no data is available but the connection is still alive (the other end may send).
Just to complete the existing answers, I’d suggest using select instead of nonblocking sockets. The point is that nonblocking sockets complicate stuff (except perhaps sending), so I’d say there is no reason to use them at all. If you regularly have the problem that your app is blocked waiting for IO, I would also consider doing the IO in a separate thread in the background.
Basically, I’ve read in several places that socket.recv() will return whatever it can read, or an empty string signalling that the other side has shut down (the official docs don’t even mention what it returns when the connection is shut down… great!). This is all fine and dandy for blocking sockets, since we know that recv() only returns when there actually is something to receive, so when it returns an empty string, it MUST mean the other side has closed the connection, right?
Okay, fine, but what happens when my socket is non-blocking?? I have searched a bit (maybe not enough, who knows?) and can’t figure out how to tell when the other side has closed the connection using a non-blocking socket. There seems to be no method or attribute that tells us this, and comparing the return value of recv() to the empty string seems absolutely useless… is it just me having this problem?
As a simple example, let’s say my socket’s timeout is set to 1.2342342 (whatever non-negative number you like here) seconds and I call socket.recv(1024), but the other side doesn’t send anything during that 1.2342342 second period. The recv() call will return an empty string and I have no clue as to whether the connection is still standing or not…
When you use recv in connection with select if the socket is ready to be read from but there is no data to read that means the client has closed the connection.
Here is some code that handles this, also note the exception that is thrown when recv is called a second time in the while loop. If there is nothing left to read this exception will be thrown it doesn’t mean the client has closed the connection :
def listenToSockets(self):
while True:
changed_sockets = self.currentSockets
ready_to_read, ready_to_write, in_error = select.select(changed_sockets, [], [], 0.1)
for s in ready_to_read:
if s == self.serverSocket:
self.acceptNewConnection(s)
else:
self.readDataFromSocket(s)
And the function that receives the data :
def readDataFromSocket(self, socket):
data = ''
buffer = ''
try:
while True:
data = socket.recv(4096)
if not data:
break
buffer += data
except error, (errorCode,message):
# error 10035 is no data available, it is non-fatal
if errorCode != 10035:
print 'socket.error - ('+str(errorCode)+') ' + message
if data:
print 'received '+ buffer
else:
print 'disconnected'
In the case of a non blocking socket that has no data available, recv will throw the socket.error exception and the value of the exception will have the errno of either EAGAIN or EWOULDBLOCK. Example:
import sys
import socket
import fcntl, os
import errno
from time import sleep
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('127.0.0.1',9999))
fcntl.fcntl(s, fcntl.F_SETFL, os.O_NONBLOCK)
while True:
try:
msg = s.recv(4096)
except socket.error, e:
err = e.args[0]
if err == errno.EAGAIN or err == errno.EWOULDBLOCK:
sleep(1)
print 'No data available'
continue
else:
# a "real" error occurred
print e
sys.exit(1)
else:
# got a message, do something :)
The situation is a little different in the case where you’ve enabled non-blocking behavior via a time out with socket.settimeout(n) or socket.setblocking(False). In this case a socket.error is stil raised, but in the case of a time out, the accompanying value of the exception is always a string set to ‘timed out’. So, to handle this case you can do:
import sys
import socket
from time import sleep
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('127.0.0.1',9999))
s.settimeout(2)
while True:
try:
msg = s.recv(4096)
except socket.timeout, e:
err = e.args[0]
# this next if/else is a bit redundant, but illustrates how the
# timeout exception is setup
if err == 'timed out':
sleep(1)
print 'recv timed out, retry later'
continue
else:
print e
sys.exit(1)
except socket.error, e:
# Something else happened, handle error, exit, etc.
print e
sys.exit(1)
else:
if len(msg) == 0:
print 'orderly shutdown on server end'
sys.exit(0)
else:
# got a message do something :)
As indicated in the comments, this is also a more portable solution since it doesn’t depend on OS specific functionality to put the socket into non-blockng mode.
See recv(2) and python socket for more details.
It is simple: if recv() returns 0 bytes; you will not receive any more data on this connection. Ever. You still might be able to send.
It means that your non-blocking socket have to raise an exception (it might be system-dependent) if no data is available but the connection is still alive (the other end may send).
Just to complete the existing answers, I’d suggest using select instead of nonblocking sockets. The point is that nonblocking sockets complicate stuff (except perhaps sending), so I’d say there is no reason to use them at all. If you regularly have the problem that your app is blocked waiting for IO, I would also consider doing the IO in a separate thread in the background.