Logging messages appear twice in console Python

Question:

I have found this answer to a seemingly similar issue, however (since I’m novice into Python) I am not sure how to implement this solution in my code (if it’s the same issue after all).

In my code I have the following section:

logging.basicConfig(level=logging.DEBUG,
                    format='%(asctime)s %(name)-12s %(levelname)-8s %(message)s',
                    filename='C:\Tests\TRACE.log',
                    filemode='a')
console = logging.StreamHandler()
console.setLevel(logging.INFO)
consoleFormatter = logging.Formatter('%(name)-12s: %(levelname)-8s %(message)s')
console.setFormatter(consoleFormatter)
logging.getLogger('').addHandler(console)
localLog = logging.getLogger('text')

The funny thing is that it used to work fine but at some moment it started writing these duplicate messages to console.

Could someone give me a direction here please?

Asked By: Eugene S

||

Answers:

Typically duplicate log statements occur because there are two separate handlers attached that are directing your log statements to the same place. There are a couple of things worth trying to get to the root of the problem:

  1. Comment out the call to logging.basicConfig – if this eliminates the duplicate log statements then this means it is likely that you don’t need to manually configure the second log handler.
  2. If you are using an IDE, it may be worth putting a breakpoint onto a log statement, and stepping in using the debugger so you can introspect the state of pythons log setup to get a clearer picture of what different handlers are attached.

To make your logging easier to manage, it may be worth looking to move the configuration out of the code and into a configuration file – the Python document on logging configuration is a great place to start.

Answered By: robjohncox

It seems that I have figured out the source of this problem.

The thing is that I used to get logger at module level. It looked pretty logical but there is a pitfall – Python logging module respects all created logger before you load the configuration from a file. So basically, when I was importing a module(which uses gets logger internally) to a main code(where I was calling for a logger as well) it resulted in streaming the logger data twice.

The possible solutions to this problem are:

  1. Do not get logger at the module level
  2. Set disable_existing_loggers to False. Added since Python 2.7
Answered By: Eugene S

I had the same situation with my logger configuration, the fix that i used was:

class Logger:

@staticmethod
def setup(name, file_name):

    log_file = <path to save the file>
    log_file_max_size = 1024 * 1024 * 20  # megabytes
    log_num_backups = 3
    log_format = "%(asctime)s [%(levelname)s]: %(filename)s(%(funcName)s:%(lineno)s) >> %(message)s"
    log_filemode = "w"  # w: overwrite; a: append
    logging.basicConfig(filename=log_file, format=log_format, filemode=log_filemode, level=logging.DEBUG)
    rotate_file = logging.handlers.RotatingFileHandler(
        log_file, maxBytes=log_file_max_size, backupCount=log_num_backups
    )
    logger = logging.getLogger(name)
    # Console output line.
    console_handler = logging.StreamHandler()
    console_handler.setLevel(logging.INFO)
    log_formatter = logging.Formatter(log_format)
    console_handler.setFormatter(log_formatter)
    logger.handlers = rotate_file, console_handler

    return logger
Answered By: Joshua Briceño Z

Here is some solution if you want to create logger in separate file. I used information from @(Joshua Briceño Z) answer, however I had some issues with saving logs to a file.

Basically I dropped the basicConfig, and to console_handler I added propagate=0 to skip printing twice.

Here I have two files:

  1. config.py – where is a Logger Class that setups the logger;
  2. program.py – where I call a Logger Class from config and tell to which file save the logs.

config.py:

import logging

class Logger:
    """
    Sets the logger.

    Attributes:
    :file_name: set a filename in which logs will be saved.
    :name: set the name of specific logger.

    :return: function
    """

    @staticmethod
    def setup(name: str, file_name: str):
        log_file = file_name
        log_format = "%(asctime)s - %(name)-12s: %(levelname)-8s %(message)s"
        log_level = logging.INFO
  
        logger = logging.getLogger(name)

        # File output line
        logger.setLevel(logging.INFO)
        formatter = logging.Formatter(log_format)
        file_handler = logging.FileHandler(log_file)
        file_handler.setFormatter(formatter)
        logger.addHandler(file_handler)

        # Console output line
        console_handler = logging.StreamHandler()
        console_handler.setLevel(log_level)
        log_formatter = logging.Formatter(log_format)
        console_handler.setFormatter(log_formatter)
        logger.addHandler(console_handler)
        logger.propagate = 0

        return logger

program.py

import config
logger_1 = config.Logger.setup(name='program', file_name='program.log')

NOTE: Tested on python 3.8.10.

Answered By: Bart Bart
Categories: questions Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.