Perplexed by Python logging

Question

I started using Python logging to tidy up the messages generated by some code. I have the statement

logging.basicConfig(filename='my_log.log', filemode='w', encoding='utf-8', level=logging.INFO)

in the main Python module and was expecting the log file to be created from scratch every time the program runs. However, Python appends the new messages to the existing file. I wrote the program below to try to test what is going on but now I don’t get a log file created and the messages appear in the console (Spyder 5). I am running WinPython 3.9 on a Windows 10 machine.

import datetime
import logging 

logging.basicConfig(filename='my_log.log', filemode='w', level=logging.INFO)

t = datetime.datetime.now()
logging.info("Something happened at %s" % t)

All suggestions welcome – thanks!

Update: following @bzu’s suggestion, I changed my code to this:

import datetime
import logging 

logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
handler = logging.FileHandler('my_log.log', 'w')
logger.addHandler(handler)

t = datetime.datetime.now()
logging.info("Something happened at %s" % t)

This created the log file but nothing was written to it. If I change the final line to
logger.info("Something happened at %s" % t)
the the message appears in the log file. However, I want to log the messages from different modules which import logging and so know about logging in general but I don’t know how they can access the specific logger object

Asked By: PetGriffin

||

Source

Answer 1

If you run the program with plain python (python log_test_file.py), it should work as expected.

I suspect that the problem with Spyder is, that it overrides the logging module config.

To avoid this kind of problems, the recommended way is to use custom logger per file:

import logging 

logger = logging.getLogger(__name__)

# boilerplate: can be converted into a method and reused
logger.setLevel(logging.INFO)
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
handler = logging.FileHandler(filename='my_log.log', mode='w', encoding='utf-8')
handler.setFormatter(formatter)
logger.addHandler(handler)

logger.info("Something happened")

Answered By: bzu

Answer 2

After some searching on the web, adding force=True to logging.basicConfig seemed to solve the problem ie:
logging.basicConfig(filename='my_log.log', filemode='w', encoding='utf-8', level=logging.INFO, force=True)

Answered By: PetGriffin

Perplexed by Python logging

Question:

Answers: