linux tee is not working with python?
Question:
I made a python script which communicates with a web server using an infinite loop.
I want to log every communication data to a file and also monitor them from terminal at same time. so I used tee command like this.
python client.py | tee logfile
however, I got nothing from terminal nor logfile.
the python script is working fine.
what is happening here?
am I missing something?
some advice would be appreciated.
thank you in advance.
Answers:
From man python
:
-u Force stdin, stdout and stderr to be totally unbuffered. On systems
where it matters, also put stdin, stdout and stderr in binary mode. Note
that there is internal buffering in xreadlines(), readlines() and file-
object iterators ("for line in sys.stdin") which is not influenced by
this option. To work around this, you will want to use "sys.stdin.read‐
line()" inside a "while 1:" loop.
So what you can do is:
/usr/bin/python -u client.py >> logfile 2>&1
Or using tee
:
python -u client.py | tee logfile
Instead of making it fully unbuffered you can make it linebuffered as it is normally with sys.stdout.reconfigure(line_buffering=True)
(after import sys
of course).
This was added in 3.7, docs: https://docs.python.org/3/library/io.html#io.TextIOWrapper.reconfigure
I made a python script which communicates with a web server using an infinite loop.
I want to log every communication data to a file and also monitor them from terminal at same time. so I used tee command like this.
python client.py | tee logfile
however, I got nothing from terminal nor logfile.
the python script is working fine.
what is happening here?
am I missing something?
some advice would be appreciated.
thank you in advance.
From man python
:
-u Force stdin, stdout and stderr to be totally unbuffered. On systems
where it matters, also put stdin, stdout and stderr in binary mode. Note
that there is internal buffering in xreadlines(), readlines() and file-
object iterators ("for line in sys.stdin") which is not influenced by
this option. To work around this, you will want to use "sys.stdin.read‐
line()" inside a "while 1:" loop.
So what you can do is:
/usr/bin/python -u client.py >> logfile 2>&1
Or using tee
:
python -u client.py | tee logfile
Instead of making it fully unbuffered you can make it linebuffered as it is normally with sys.stdout.reconfigure(line_buffering=True)
(after import sys
of course).
This was added in 3.7, docs: https://docs.python.org/3/library/io.html#io.TextIOWrapper.reconfigure