python – How select.select() works?
Question:
Background:
I’m familiar with C’s select() function. I’ve been using this function for many purposes. Most of them, if not all, for reading and writing to pipes, files, etc. I must say that I’ve never used the error list, but this is not involved in the key question.
Question:
Does Python’s select() behave as follows?
It turns out to me that select() on Python behaves a different way despite the straightforward interface to C select(). It seems that select() returns the very first time a file is ready for reading. If you read the file while leaving some bytes to be read, calling select() will block. But, if you call select() again after a previous call to select() was returned without any read call between these two calls, select() will return as expected. For example:
import select
# Open the file (yes, playing around with joysticks)
file = open('/dev/input/js0', 'r')
# Hold on the select() function waiting
select.select([file], [], [])
# Say 16 bytes are sent to the file, select() will return.
([<open file '/dev/input/js0', mode 'r' at 0x7ff2949c96f0>], [], [])
# Call select() again, and select() will indeed return.
select.select([file], [], [])
([<open file '/dev/input/js0', mode 'r' at 0x7ff2949c96f0>], [], [])
# read 8 bytes. There are 8 bytes left for sure. Calling again file.read(8) will empty the queue and would be pointless for this example
file.read(8)
'<tx06x01x00x00x81x01'
# call select() again, and select() will block
select.select([file], [], [])
# Should it block? there are 8 bytes on the file to be read.
If this is the behaviour of select() in Python, I’m okay with that, I could handle it. Not what I expected, though, but it’s fine. I know what I can do with it.
But if this is not the behaviour of select() I would appreciate someone to tell me what I’m doing wrong. What I read about select() is what the Python docs say: "select() returns if any file in the read|write|error list is ready for read|write|error.". That’s OK no lies there. Maybe the questions should be:
- When a file is considered to be ready for reading in python?
- Does it means a file that has never been read?
- Does it means a file with bytes to be read?
Answers:
Python’s select() gets passed through as a select() system call as you are expecting, but the problem you have with it blocking is a different issue, probably relating to buffering. Just to satify yourself that select() is doing the right thing, try reading/writing a file on the file system rather than using a special device such as a joystick.
You probably want to change your open() call. Pythons open call will by default use buffered reads, so even if you do a read(8) it will likely read more data from the input file and buffer the results. You need to set the buffering option to open so that the joystick device is opened unbuffered.
Suggestions for you to try:
- Python defaults to opening files in text mode. You probably want the open
mode to be rb when dealing with special devices such as a joystick.
- Open file in unbuffered mode.
- Set the device into non-blocking mode if you’re going to be doing
select based calls. Try using os.open() with os.O_RDONLY|os.O_NONBLOCK flags.
Can I ask a stupid question – are you sure there are really 8 bytes left?
Device nodes don’t necessarily behave like normal files. It might be that you have to read the entire struct input_event in a single read() system call. (And if you don’t read enough, the rest gets thrown away). A bit like recvmsg() on datagram sockets.
Background:
I’m familiar with C’s select() function. I’ve been using this function for many purposes. Most of them, if not all, for reading and writing to pipes, files, etc. I must say that I’ve never used the error list, but this is not involved in the key question.
Question:
Does Python’s select() behave as follows?
It turns out to me that select() on Python behaves a different way despite the straightforward interface to C select(). It seems that select() returns the very first time a file is ready for reading. If you read the file while leaving some bytes to be read, calling select() will block. But, if you call select() again after a previous call to select() was returned without any read call between these two calls, select() will return as expected. For example:
import select
# Open the file (yes, playing around with joysticks)
file = open('/dev/input/js0', 'r')
# Hold on the select() function waiting
select.select([file], [], [])
# Say 16 bytes are sent to the file, select() will return.
([<open file '/dev/input/js0', mode 'r' at 0x7ff2949c96f0>], [], [])
# Call select() again, and select() will indeed return.
select.select([file], [], [])
([<open file '/dev/input/js0', mode 'r' at 0x7ff2949c96f0>], [], [])
# read 8 bytes. There are 8 bytes left for sure. Calling again file.read(8) will empty the queue and would be pointless for this example
file.read(8)
'<tx06x01x00x00x81x01'
# call select() again, and select() will block
select.select([file], [], [])
# Should it block? there are 8 bytes on the file to be read.
If this is the behaviour of select() in Python, I’m okay with that, I could handle it. Not what I expected, though, but it’s fine. I know what I can do with it.
But if this is not the behaviour of select() I would appreciate someone to tell me what I’m doing wrong. What I read about select() is what the Python docs say: "select() returns if any file in the read|write|error list is ready for read|write|error.". That’s OK no lies there. Maybe the questions should be:
- When a file is considered to be ready for reading in python?
- Does it means a file that has never been read?
- Does it means a file with bytes to be read?
Python’s select() gets passed through as a select() system call as you are expecting, but the problem you have with it blocking is a different issue, probably relating to buffering. Just to satify yourself that select() is doing the right thing, try reading/writing a file on the file system rather than using a special device such as a joystick.
You probably want to change your open() call. Pythons open call will by default use buffered reads, so even if you do a read(8) it will likely read more data from the input file and buffer the results. You need to set the buffering option to open so that the joystick device is opened unbuffered.
Suggestions for you to try:
- Python defaults to opening files in text mode. You probably want the open
modeto berbwhen dealing with special devices such as a joystick. - Open file in unbuffered mode.
- Set the device into non-blocking mode if you’re going to be doing
selectbased calls. Try usingos.open()withos.O_RDONLY|os.O_NONBLOCKflags.
Can I ask a stupid question – are you sure there are really 8 bytes left?
Device nodes don’t necessarily behave like normal files. It might be that you have to read the entire struct input_event in a single read() system call. (And if you don’t read enough, the rest gets thrown away). A bit like recvmsg() on datagram sockets.