Python logging output doesn't show unless I call another logger first

Question

I have some behaviour I can’t explain wrt the Python logging library, and none of the docs nor tutorials I’ve read match with what I’m seeing (well they probably do, I guess I’m just missing some crucial detail somewhere). Consider first what does work as expected:

import logging

logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)

logging.debug("Test 1")
logging.info("Test 2")
logging.warning("Test 3")

logger.debug("Test 4")
logger.info("Test 5")

Output is

$ e:Python311python.exe testlogger.py
ERROR:root:Test 3
DEBUG:__main__:Test 4
INFO:__main__:Test 5

The default minimum log level for the ‘module level’ logger (so to call it) is logging.WARNING, so "Test 1" & "Test 2" are not shown, but "Test 3" is; and the instantiated logger is configured to log from level logging.DEBUG onwards so "Test 4" and "Test 5" are shown, too. So far so good.

But now I take out the calls to the ‘module level’ logger like so:

import logging

logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)

#logging.debug("Test 1")
#logging.info("Test 2")
#logging.warning("Test 3")

logger.debug("Test 4")
logger.info("Test 5")

and now the output from this program is empty, i.e. nothing is output at all. Why aren’t "Test 4" and "Test 5" shown? How and why are calls to the logging functions of my instantiated logger influenced by earlier calls to the ‘module level’ logger? Thanks.

(The above is Python 3.11 on Windows, but I tried on online-python.com too and got the same result, it uses 3.8.5 on I presume Linux)

Asked By: Roel

||

Source

Answer 1

There are 2 missing pieces of information:

Loggers in python require "handlers" to actually print stuff
Loggers propagate all messages to parent loggers, with the root logger being the parent of all loggers

Some step by step code:

In [1]: import logging
   ...:
   ...: logger = logging.getLogger("mylogger")
   ...: logger.setLevel(logging.DEBUG)

In [2]: logger
Out[2]: <Logger mylogger (DEBUG)>

In [3]: logger.handlers # your logger has no handlers
Out[3]: []

In [4]: logging.root
Out[4]: <RootLogger root (WARNING)>

In [5]: logger.root.handlers # the root logger has no handlers
Out[5]: []

# first call to the module level logging sets a handler on the root logger
In [6]: logging.info('hello') 

In [7]: logger.root.handlers
Out[7]: [<StreamHandler <stderr> (NOTSET)>]

In [8]: logger.info('hello')
INFO:mylogger:hello

# your logger still has no handlers, but it propagates messages to the root logger, which does
In [9]: logger.handlers
Out[9]: []

In your second example, you didn’t call the "module logging" logger at all, so it never created an handler for the root logger.

So whats the way to do that correctly?
There are many ways, the fastest or most basic way is using basicConfig

In [1]: import logging

In [2]: logging.basicConfig(level='INFO')

In [3]: logger = logging.getLogger('mylogger')

In [4]: logger.handlers
Out[4]: []

In [5]: logger.root.handlers
Out[5]: [<StreamHandler <stderr> (NOTSET)>]

In [6]: logger.info('hello')
INFO:mylogger:hello

basicConfig just configures the root logger, and since all loggers propagate to it, ALL LOGGERS will now be visible, this includes third-party libraries (although you can selectively suppress them)

I prefer to selectively setup my logger instead of the root logger. Many ways to that to, this is what I do

In [1]: import logging

In [2]: logger = logging.getLogger('mylogger')

In [3]: handler = logging.StreamHandler()

In [4]: handler.setFormatter(logging.Formatter('%(asctime)s | %(name)s | %(levelname)s | %(message)s'))

In [5]: logger.addHandler(handler)

In [6]: logger.setLevel('INFO')

In [7]: logger.info('hello world')
2023-02-21 11:39:07,425 | mylogger | INFO | hello world

there are many other ways specified in the docs to interact with the loggers, like adding filters, additional handlers, and more

Answered By: Ron Serruya

Python logging output doesn't show unless I call another logger first

Question:

Answers: