Printing to screen and writing to a file at the same time
Question:
I found some code online that generally works, but I want to use it multiple times in the same program (write different things to different files, while still printing to the screen the whole time).
That is to say, when it closes, I think sys.stdout closes, so printing at all, and using this class again fails. I tried reimporting sys, and other dumb stuff, but I can’t get it to work.
Here’s the site, and the code
groups.google.com/group/comp.lang.python/browse_thread/thread/d25a9f5608e473af/
import sys
class MyWriter:
def __init__(self, stdout, filename):
self.stdout = stdout
self.logfile = file(filename, 'a')
def write(self, text):
self.stdout.write(text)
self.logfile.write(text)
def close(self):
self.stdout.close()
self.logfile.close()
writer = MyWriter(sys.stdout, 'log.txt')
sys.stdout = writer
print 'test'
Answers:
Remove the line that’s doing what you explicitly say you don’t want done: the first line of close(), which closes stdout.
You are trying to reproduce poorly something that is done very well by the Python Standard Library; please check the logging module.
With this module you can do exactly what you want, but in a much simpler, standard, and extensible manner. You can proceed as follows (this example is a copy/paste from the logging cookbook):
Let’s say you want to log to console and file with different message
formats and in differing circumstances. Say you want to log messages
with levels of DEBUG and higher to file, and those messages at level
INFO and higher to the console. Let’s also assume that the file should
contain timestamps, but the console messages should not. Here’s how
you can achieve this:
import logging
# set up logging to file - see previous section for more details
logging.basicConfig(level=logging.DEBUG,
format='%(asctime)s %(name)-12s %(levelname)-8s %(message)s',
datefmt='%m-%d %H:%M',
filename='/temp/myapp.log',
filemode='w')
# define a Handler which writes INFO messages or higher to the sys.stderr
console = logging.StreamHandler()
console.setLevel(logging.INFO)
# set a format which is simpler for console use
formatter = logging.Formatter('%(name)-12s: %(levelname)-8s %(message)s')
# tell the handler to use this format
console.setFormatter(formatter)
# add the handler to the root logger
logging.getLogger().addHandler(console)
# Now, we can log to the root logger, or any other logger. First the root...
logging.info('Jackdaws love my big sphinx of quartz.')
# Now, define a couple of other loggers which might represent areas in your
# application:
logger1 = logging.getLogger('myapp.area1')
logger2 = logging.getLogger('myapp.area2')
logger1.debug('Quick zephyrs blow, vexing daft Jim.')
logger1.info('How quickly daft jumping zebras vex.')
logger2.warning('Jail zesty vixen who grabbed pay from quack.')
logger2.error('The five boxing wizards jump quickly.')
When you run this, on the console you will see
root : INFO Jackdaws love my big sphinx of quartz.
myapp.area1 : INFO How quickly daft jumping zebras vex.
myapp.area2 : WARNING Jail zesty vixen who grabbed pay from quack.
myapp.area2 : ERROR The five boxing wizards jump quickly.
and in the file you will see something like
10-22 22:19 root INFO Jackdaws love my big sphinx of quartz.
10-22 22:19 myapp.area1 DEBUG Quick zephyrs blow, vexing daft Jim.
10-22 22:19 myapp.area1 INFO How quickly daft jumping zebras vex.
10-22 22:19 myapp.area2 WARNING Jail zesty vixen who grabbed pay from quack.
10-22 22:19 myapp.area2 ERROR The five boxing wizards jump quickly.
As you can see, the DEBUG message only shows up in the file. The other
messages are sent to both destinations.
This example uses console and file handlers, but you can use any
number and combination of handlers you choose.
That is to say, when it closes, I think sys.stdout closes, so printing
at all, and using this class again fails. I tried reimporting sys, and
other dumb stuff, but I can’t get it to work.
To answer your question, you should not be closing stdout. The python interpreter opens stdout, stdin and stderror at startup. In order for print to work, the interpreter requires stdout to be open. Reimporting sys does not do anything once a module has been loaded. You would need to reload the module. In this particular case, I am not sure a reload would fix the problem since sys.stdout allows stdout to be used as a file object.
Additionally, I think you have a bug in your code which may be causing print to
break. In line 2 you are assigning a MyWriter object to sys.stdout. This may by closing stdout when the garbage collector deletes the unused stdout file object.
writer = MyWriter(sys.stdout, 'log.txt')
sys.stdout = writer
Easy-peasy with Python 3.3 and above
Starting with Python 3.3, doing so has become significantly easier since logging.basicConfig now accepts the handlers = argument.
import logging
level = logging.INFO
format = ' %(message)s'
handlers = [logging.FileHandler('filename.log'), logging.StreamHandler()]
logging.basicConfig(level = level, format = format, handlers = handlers)
logging.info('Hey, this is working!')
Note however, that certain Python modules may also be posting logging messages to the INFO level.
This is where it comes handy to create a custom logging level, called for example OK, 5 levels above the default INFO level and 5 levels below the default WARNING level.
I know this is an old question, and the best answer is just to use logging for its intended purpose, but I just wanted to point out that if you’re concerned only with affecting calls specifically to print (and not other interaction with sys.stdout), and you just want to paste a few lines into some old one-off script, there’s nothing stopping you from simply reassigning the name to a different function which writes to two different files, since print is a function in Python 3+. You could even, god forbid, use a lambda with an or chain for the quickest, dirtiest solution out there:
old_print = print
log_file = open("logfile.log", "a")
print = lambda *args, **kw: old_print(*args, **kw) or old_print(*args, file=log_file, **kw)
print("Hello console and log file")
# ... more calls to print() ...
log_file.close()
Or for true fire-and-forget:
import atexit
old_print = print
log_file = open("logfile.log", "a")
atexit.register(log_file.close)
print = lambda *args, **kw: old_print(*args, **kw) or old_print(*args, file=log_file, **kw)
# ... do calls to print(), and you don't even have to close the file afterwards ...
It works fine assuming the program exits properly, but please no one use this in production code, just use logging 🙂
Edit: If you value some form of structure and want to write to the log file in real-time, consider something like:
from typing import Callable
def print_logger(
old_print: Callable,
file_name: str,
) -> Callable:
"""Returns a function which calls `old_print` twice, specifying a `file=` on the second call.
Arguments:
old_print: The `print` function to call twice.
file_name: The name to give the log file.
"""
def log_print(*args, **kwargs):
old_print(*args, **kwargs)
with open(file_name, "a") as log_file:
old_print(*args, file=log_file, **kwargs)
return log_print
And then invoke as follows:
print = print_logger(print, "logs/my_log.log")
I found some code online that generally works, but I want to use it multiple times in the same program (write different things to different files, while still printing to the screen the whole time).
That is to say, when it closes, I think sys.stdout closes, so printing at all, and using this class again fails. I tried reimporting sys, and other dumb stuff, but I can’t get it to work.
Here’s the site, and the code
groups.google.com/group/comp.lang.python/browse_thread/thread/d25a9f5608e473af/
import sys
class MyWriter:
def __init__(self, stdout, filename):
self.stdout = stdout
self.logfile = file(filename, 'a')
def write(self, text):
self.stdout.write(text)
self.logfile.write(text)
def close(self):
self.stdout.close()
self.logfile.close()
writer = MyWriter(sys.stdout, 'log.txt')
sys.stdout = writer
print 'test'
Remove the line that’s doing what you explicitly say you don’t want done: the first line of close(), which closes stdout.
You are trying to reproduce poorly something that is done very well by the Python Standard Library; please check the logging module.
With this module you can do exactly what you want, but in a much simpler, standard, and extensible manner. You can proceed as follows (this example is a copy/paste from the logging cookbook):
Let’s say you want to log to console and file with different message
formats and in differing circumstances. Say you want to log messages
with levels of DEBUG and higher to file, and those messages at level
INFO and higher to the console. Let’s also assume that the file should
contain timestamps, but the console messages should not. Here’s how
you can achieve this:
import logging
# set up logging to file - see previous section for more details
logging.basicConfig(level=logging.DEBUG,
format='%(asctime)s %(name)-12s %(levelname)-8s %(message)s',
datefmt='%m-%d %H:%M',
filename='/temp/myapp.log',
filemode='w')
# define a Handler which writes INFO messages or higher to the sys.stderr
console = logging.StreamHandler()
console.setLevel(logging.INFO)
# set a format which is simpler for console use
formatter = logging.Formatter('%(name)-12s: %(levelname)-8s %(message)s')
# tell the handler to use this format
console.setFormatter(formatter)
# add the handler to the root logger
logging.getLogger().addHandler(console)
# Now, we can log to the root logger, or any other logger. First the root...
logging.info('Jackdaws love my big sphinx of quartz.')
# Now, define a couple of other loggers which might represent areas in your
# application:
logger1 = logging.getLogger('myapp.area1')
logger2 = logging.getLogger('myapp.area2')
logger1.debug('Quick zephyrs blow, vexing daft Jim.')
logger1.info('How quickly daft jumping zebras vex.')
logger2.warning('Jail zesty vixen who grabbed pay from quack.')
logger2.error('The five boxing wizards jump quickly.')
When you run this, on the console you will see
root : INFO Jackdaws love my big sphinx of quartz.
myapp.area1 : INFO How quickly daft jumping zebras vex.
myapp.area2 : WARNING Jail zesty vixen who grabbed pay from quack.
myapp.area2 : ERROR The five boxing wizards jump quickly.
and in the file you will see something like
10-22 22:19 root INFO Jackdaws love my big sphinx of quartz.
10-22 22:19 myapp.area1 DEBUG Quick zephyrs blow, vexing daft Jim.
10-22 22:19 myapp.area1 INFO How quickly daft jumping zebras vex.
10-22 22:19 myapp.area2 WARNING Jail zesty vixen who grabbed pay from quack.
10-22 22:19 myapp.area2 ERROR The five boxing wizards jump quickly.
As you can see, the DEBUG message only shows up in the file. The other
messages are sent to both destinations.This example uses console and file handlers, but you can use any
number and combination of handlers you choose.
That is to say, when it closes, I think sys.stdout closes, so printing
at all, and using this class again fails. I tried reimporting sys, and
other dumb stuff, but I can’t get it to work.
To answer your question, you should not be closing stdout. The python interpreter opens stdout, stdin and stderror at startup. In order for print to work, the interpreter requires stdout to be open. Reimporting sys does not do anything once a module has been loaded. You would need to reload the module. In this particular case, I am not sure a reload would fix the problem since sys.stdout allows stdout to be used as a file object.
Additionally, I think you have a bug in your code which may be causing print to
break. In line 2 you are assigning a MyWriter object to sys.stdout. This may by closing stdout when the garbage collector deletes the unused stdout file object.
writer = MyWriter(sys.stdout, 'log.txt')
sys.stdout = writer
Easy-peasy with Python 3.3 and above
Starting with Python 3.3, doing so has become significantly easier since logging.basicConfig now accepts the handlers = argument.
import logging
level = logging.INFO
format = ' %(message)s'
handlers = [logging.FileHandler('filename.log'), logging.StreamHandler()]
logging.basicConfig(level = level, format = format, handlers = handlers)
logging.info('Hey, this is working!')
Note however, that certain Python modules may also be posting logging messages to the INFO level.
This is where it comes handy to create a custom logging level, called for example OK, 5 levels above the default INFO level and 5 levels below the default WARNING level.
I know this is an old question, and the best answer is just to use logging for its intended purpose, but I just wanted to point out that if you’re concerned only with affecting calls specifically to print (and not other interaction with sys.stdout), and you just want to paste a few lines into some old one-off script, there’s nothing stopping you from simply reassigning the name to a different function which writes to two different files, since print is a function in Python 3+. You could even, god forbid, use a lambda with an or chain for the quickest, dirtiest solution out there:
old_print = print
log_file = open("logfile.log", "a")
print = lambda *args, **kw: old_print(*args, **kw) or old_print(*args, file=log_file, **kw)
print("Hello console and log file")
# ... more calls to print() ...
log_file.close()
Or for true fire-and-forget:
import atexit
old_print = print
log_file = open("logfile.log", "a")
atexit.register(log_file.close)
print = lambda *args, **kw: old_print(*args, **kw) or old_print(*args, file=log_file, **kw)
# ... do calls to print(), and you don't even have to close the file afterwards ...
It works fine assuming the program exits properly, but please no one use this in production code, just use logging 🙂
Edit: If you value some form of structure and want to write to the log file in real-time, consider something like:
from typing import Callable
def print_logger(
old_print: Callable,
file_name: str,
) -> Callable:
"""Returns a function which calls `old_print` twice, specifying a `file=` on the second call.
Arguments:
old_print: The `print` function to call twice.
file_name: The name to give the log file.
"""
def log_print(*args, **kwargs):
old_print(*args, **kwargs)
with open(file_name, "a") as log_file:
old_print(*args, file=log_file, **kwargs)
return log_print
And then invoke as follows:
print = print_logger(print, "logs/my_log.log")