How to use a custom console logger for the entire application in Python?

Question

After reading the logging documentation of Python and several examples of how to customise the log format and output, I came to the conclusion that formatting can only be set for a single logger instance. But since all libraries in other modules use their own logger instance (so that their source can be identified), they completely ignore how my main application wants to output the log messages.

How can I create a custom logger that formats the records in a specific way (for example with abbreviated levels and corresponding console colours) that is automatically used for all modules when set up in the main function?

What I saw is this:

# main.py:
logger = logging.getLogger('test')
logger.addHandler(ConsoleHandler())

# module.py:
logger = logging.getLogger(__name__)
...
logger.info("a message from the library")

This prints my library log message as before, not with the custom format. It seems rather pointless to apply an output format to a single logger instance if each module has their own one. The formatter must be applied at the application level, not for each library individually. At least that’s how I understand and successfully use things around ILogger in .NET. Can Python do that? I guess what I need is my own ILogger implementation that is made accessible throughout the application through dependency injection. That’s not how Python logging looks like to me.

Asked By: ygoe

||

Source

Answer 1

This is what I found about how it works:

There is a "root" logger that seems to be the parent of all other loggers, and all other loggers also inherit their properties from the root logger. The root logger can be retrieved with

rootLogger = logging.getLogger()

After that, the handlers can be modified on that instance:

for handler in rootLogger.handlers:
    rootLogger.removeHandler(handler)
newHandler = ConsoleHandler()
rootLogger.addHandler(newHandler)

Then all other loggers will also use this handler and the output goes in the desired format to the desired destination(s):

logging.info("Like this...")

logger = logging.getLogger(__name__)
logger.warning("...or that")

Answered By: ygoe

Answer 2

Output is done by Handlers, but formatting is controlled by Formatters. You don’t show any code which sets a Formatter on the handler, and I can’t tell if your ConsoleHandler sets one internally. But it’s absolutely possible to control output as the application developer wants, as well-designed libraries only do the code in your module.py snippet. Here are a couple of example files:

# mylib.py
import logging

logger = logging.getLogger(__name__)

def foo():
    logger.debug('A DEBUG message')
    logger.info('An INFO message')
    logger.warning('A WARNING message')
    logger.error('An ERROR message')
    logger.critical('A CRITICAL message')

and

# main.py
import logging

import mylib

logger = logging.getLogger(__name__)

def bar():
    logger.debug('A DEBUG message')
    logger.info('An INFO message')
    logger.warning('A WARNING message')
    logger.error('An ERROR message')
    logger.critical('A CRITICAL message')

h = logging.StreamHandler()
f = logging.Formatter('%(levelname).1s %(name)-10s %(filename)10s %(lineno)2d %(message)s')
h.setFormatter(f)
root = logging.getLogger()
root.addHandler(h)
root.setLevel(logging.DEBUG)
bar()
mylib.foo()

When you run python main.py, you get

D __main__      main.py  8 A DEBUG message
I __main__      main.py  9 An INFO message
W __main__      main.py 10 A WARNING message
E __main__      main.py 11 An ERROR message
C __main__      main.py 12 A CRITICAL message
D mylib        mylib.py  6 A DEBUG message
I mylib        mylib.py  7 An INFO message
W mylib        mylib.py  8 A WARNING message
E mylib        mylib.py  9 An ERROR message
C mylib        mylib.py 10 A CRITICAL message

which shows both modules using a common, custom format (of course, this example doesn’t colorize).

Answered By: Vinay Sajip

How to use a custom console logger for the entire application in Python?

Question:

Answers: