Redirect Python 'print' output to Logger
Question:
I have a Python script that makes use of ‘Print’ for printing to stdout. I’ve recently added logging via Python Logger and would like to make it so these print statements go to logger if logging is enabled. I do not want to modify or remove these print statements.
I can log by doing ‘log.info(“some info msg”)’. I want to be able to do something like this:
if logging_enabled:
sys.stdout=log.info
print("test")
If logging is enabled, “test” should be logged as if I did log.info(“test”). If logging isn’t enabled, “test” should just be printed to the screen.
Is this possible? I know I can direct stdout to a file in a similar manner (see: redirect prints to log file)
Answers:
You have two options:
-
Open a logfile and replace sys.stdout with it, not a function:
log = open("myprog.log", "a")
sys.stdout = log
>>> print("Hello")
>>> # nothing is printed because it goes to the log file instead.
-
Replace print with your log function:
# If you're using python 2.x, uncomment the next line
#from __future__ import print_function
print = log.info
>>> print("Hello!")
>>> # nothing is printed because log.info is called instead of print
You really should do that the other way: by adjusting your logging configuration to use print
statements or something else, depending on the settings. Do not overwrite print
behaviour, as some of the settings that may be introduced in the future (eg. by you or by someone else using your module) may actually output it to the stdout
and you will have problems.
There is a handler that is supposed to redirect your log messages to proper stream (file, stdout
or anything else file-like). It is called StreamHandler
and it is bundled with logging
module.
So basically in my opinion you should do, what you stated you don’t want to do: replace print
statements with actual logging.
Of course, you can both print to the standard output and append to a log file, like this:
# Uncomment the line below for python 2.x
#from __future__ import print_function
import logging
logging.basicConfig(level=logging.INFO, format='%(message)s')
logger = logging.getLogger()
logger.addHandler(logging.FileHandler('test.log', 'a'))
print = logger.info
print('yo!')
One more method is to wrap the logger in an object that translates calls to write
to the logger’s log
method.
Ferry Boender does just this, provided under the GPL license in a post on his website. The code below is based on this but solves two issues with the original:
- The class doesn’t implement the flush method which is called when the program exits.
- The class doesn’t buffer the writes on newline as io.TextIOWrapper objects are supposed to which results in newlines at odd points.
import logging
import sys
class StreamToLogger(object):
"""
Fake file-like stream object that redirects writes to a logger instance.
"""
def __init__(self, logger, log_level=logging.INFO):
self.logger = logger
self.log_level = log_level
self.linebuf = ''
def write(self, buf):
temp_linebuf = self.linebuf + buf
self.linebuf = ''
for line in temp_linebuf.splitlines(True):
# From the io.TextIOWrapper docs:
# On output, if newline is None, any 'n' characters written
# are translated to the system default line separator.
# By default sys.stdout.write() expects 'n' newlines and then
# translates them so this is still cross platform.
if line[-1] == 'n':
self.logger.log(self.log_level, line.rstrip())
else:
self.linebuf += line
def flush(self):
if self.linebuf != '':
self.logger.log(self.log_level, self.linebuf.rstrip())
self.linebuf = ''
logging.basicConfig(
level=logging.DEBUG,
format='%(asctime)s:%(levelname)s:%(name)s:%(message)s',
filename="out.log",
filemode='a'
)
stdout_logger = logging.getLogger('STDOUT')
sl = StreamToLogger(stdout_logger, logging.INFO)
sys.stdout = sl
stderr_logger = logging.getLogger('STDERR')
sl = StreamToLogger(stderr_logger, logging.ERROR)
sys.stderr = sl
This allows you to easily route all output to a logger of your choice. If needed, you can save sys.stdout
and/or sys.stderr
as mentioned by others in this thread before replacing it if you need to restore it later.
A much simpler option,
import logging, sys
logging.basicConfig(filename='path/to/logfile', level=logging.DEBUG)
logger = logging.getLogger()
sys.stderr.write = logger.error
sys.stdout.write = logger.info
Once your defined your logger, use this to make print redirect to logger even with mutiple parameters of print.
print = lambda *tup : logger.info(str(" ".join([str(x) for x in tup])))
Below snipped works perfectly inside my PySpark code. If someone need in case for understanding –>
import os
import sys
import logging
import logging.handlers
log = logging.getLogger(__name_)
handler = logging.FileHandler("spam.log")
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
handler.setFormatter(formatter)
log.addHandler(handler)
sys.stderr.write = log.error
sys.stdout.write = log.info
(will log every error in “spam.log” in the same directory, nothing will be on console/stdout)
(will log every info in “spam.log” in the same directory,nothing will be on console/stdout)
to print output error/info in both file as well as in console remove above two line.
Happy Coding
Cheers!!!
I have a Python script that makes use of ‘Print’ for printing to stdout. I’ve recently added logging via Python Logger and would like to make it so these print statements go to logger if logging is enabled. I do not want to modify or remove these print statements.
I can log by doing ‘log.info(“some info msg”)’. I want to be able to do something like this:
if logging_enabled:
sys.stdout=log.info
print("test")
If logging is enabled, “test” should be logged as if I did log.info(“test”). If logging isn’t enabled, “test” should just be printed to the screen.
Is this possible? I know I can direct stdout to a file in a similar manner (see: redirect prints to log file)
You have two options:
-
Open a logfile and replace sys.stdout with it, not a function:
log = open("myprog.log", "a") sys.stdout = log >>> print("Hello") >>> # nothing is printed because it goes to the log file instead.
-
Replace print with your log function:
# If you're using python 2.x, uncomment the next line #from __future__ import print_function print = log.info >>> print("Hello!") >>> # nothing is printed because log.info is called instead of print
You really should do that the other way: by adjusting your logging configuration to use print
statements or something else, depending on the settings. Do not overwrite print
behaviour, as some of the settings that may be introduced in the future (eg. by you or by someone else using your module) may actually output it to the stdout
and you will have problems.
There is a handler that is supposed to redirect your log messages to proper stream (file, stdout
or anything else file-like). It is called StreamHandler
and it is bundled with logging
module.
So basically in my opinion you should do, what you stated you don’t want to do: replace print
statements with actual logging.
Of course, you can both print to the standard output and append to a log file, like this:
# Uncomment the line below for python 2.x
#from __future__ import print_function
import logging
logging.basicConfig(level=logging.INFO, format='%(message)s')
logger = logging.getLogger()
logger.addHandler(logging.FileHandler('test.log', 'a'))
print = logger.info
print('yo!')
One more method is to wrap the logger in an object that translates calls to write
to the logger’s log
method.
Ferry Boender does just this, provided under the GPL license in a post on his website. The code below is based on this but solves two issues with the original:
- The class doesn’t implement the flush method which is called when the program exits.
- The class doesn’t buffer the writes on newline as io.TextIOWrapper objects are supposed to which results in newlines at odd points.
import logging
import sys
class StreamToLogger(object):
"""
Fake file-like stream object that redirects writes to a logger instance.
"""
def __init__(self, logger, log_level=logging.INFO):
self.logger = logger
self.log_level = log_level
self.linebuf = ''
def write(self, buf):
temp_linebuf = self.linebuf + buf
self.linebuf = ''
for line in temp_linebuf.splitlines(True):
# From the io.TextIOWrapper docs:
# On output, if newline is None, any 'n' characters written
# are translated to the system default line separator.
# By default sys.stdout.write() expects 'n' newlines and then
# translates them so this is still cross platform.
if line[-1] == 'n':
self.logger.log(self.log_level, line.rstrip())
else:
self.linebuf += line
def flush(self):
if self.linebuf != '':
self.logger.log(self.log_level, self.linebuf.rstrip())
self.linebuf = ''
logging.basicConfig(
level=logging.DEBUG,
format='%(asctime)s:%(levelname)s:%(name)s:%(message)s',
filename="out.log",
filemode='a'
)
stdout_logger = logging.getLogger('STDOUT')
sl = StreamToLogger(stdout_logger, logging.INFO)
sys.stdout = sl
stderr_logger = logging.getLogger('STDERR')
sl = StreamToLogger(stderr_logger, logging.ERROR)
sys.stderr = sl
This allows you to easily route all output to a logger of your choice. If needed, you can save sys.stdout
and/or sys.stderr
as mentioned by others in this thread before replacing it if you need to restore it later.
A much simpler option,
import logging, sys
logging.basicConfig(filename='path/to/logfile', level=logging.DEBUG)
logger = logging.getLogger()
sys.stderr.write = logger.error
sys.stdout.write = logger.info
Once your defined your logger, use this to make print redirect to logger even with mutiple parameters of print.
print = lambda *tup : logger.info(str(" ".join([str(x) for x in tup])))
Below snipped works perfectly inside my PySpark code. If someone need in case for understanding –>
import os
import sys
import logging
import logging.handlers
log = logging.getLogger(__name_)
handler = logging.FileHandler("spam.log")
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
handler.setFormatter(formatter)
log.addHandler(handler)
sys.stderr.write = log.error
sys.stdout.write = log.info
(will log every error in “spam.log” in the same directory, nothing will be on console/stdout)
(will log every info in “spam.log” in the same directory,nothing will be on console/stdout)
to print output error/info in both file as well as in console remove above two line.
Happy Coding
Cheers!!!