fifo – reading in a loop

Question

I want to use os.mkfifo for simple communication between programs. I have a problem with reading from the fifo in a loop.

Consider this toy example, where I have a reader and a writer working with the fifo. I want to be able to run the reader in a loop to read everything that enters the fifo.

# reader.py
import os
import atexit

FIFO = 'json.fifo'

@atexit.register
def cleanup():
    try:
        os.unlink(FIFO)
    except:
        pass

def main():
    os.mkfifo(FIFO)
    with open(FIFO) as fifo:
#        for line in fifo:              # closes after single reading
#        for line in fifo.readlines():  # closes after single reading
        while True:
            line = fifo.read()          # will return empty lines (non-blocking)
            print repr(line)

main()

And the writer:

# writer.py
import sys

FIFO = 'json.fifo'


def main():
    with open(FIFO, 'a') as fifo:
        fifo.write(sys.argv[1])

main()

If I run python reader.py and later python writer.py foo, “foo” will be printed but the fifo will be closed and the reader will exit (or spin inside the while loop). I want reader to stay in the loop, so I can execute the writer many times.

Edit

I use this snippet to handle the issue:

def read_fifo(filename):
    while True:
        with open(filename) as fifo:
            yield fifo.read()

but maybe there is some neater way to handle it, instead of repetitively opening the file…

Related

Getting readline to block on a FIFO

Asked By: Jakub M.

||

Source

Answer 1

A FIFO works (on the reader side) exactly this way: it can be read from, until all writers are gone. Then it signals EOF to the reader.

If you want the reader to continue reading, you’ll have to open again and read from there. So your snippet is exactly the way to go.

If you have mutliple writers, you’ll have to ensure that each data portion written by them is smaller than PIPE_BUF on order not to mix up the messages.

Answered By: glglgl

Answer 2

You do not need to reopen the file repeatedly. You can use select to block until data is available.

with open(FIFO_PATH) as fifo:
    while True:
        select.select([fifo],[],[fifo])
        data = fifo.read()
        do_work(data)

In this example you won’t read EOF.

Answered By: steveayre

Answer 3

The following methods on the standard library’s pathlib.Path class are helpful here:

Here is a demo:

# reader.py
import os
from pathlib import Path

fifo_path = Path("fifo")
os.mkfifo(fifo_path)

while True:
    print(fifo_path.read_text())  # blocks until data becomes available

# writer.py
import sys
from pathlib import Path

fifo_path = Path("fifo")
assert fifo_path.is_fifo()

fifo_path.write_text(sys.argv[1])

Answered By: Jasha

fifo – reading in a loop

Question:

Answers: