Guarantee all logs go to file

Question

I need to set up logging so it will always log to a file, no matter what a user or other programmer might setLevel too. For instance I have this logger set up:

initialize_logging.py

import logging

filename="./files/logs/attribution.log", level=logging.INFO)
logger = logging.getLogger('attribution')

formatter = logging.Formatter('%(asctime)s - %(levelname)s: %(message)s')
fh = logging.FileHandler('./files/logs/attribution.log')
fh.setLevel(logging.DEBUG)
fh.setFormatter(formatter)

ch = logging.StreamHandler()
ch.setLevel(logging.WARNING)
ch.setFormatter(formatter)

logger.setLevel(logging.DEBUG)
logger.addHandler(fh)
logger.addHandler(ch)

then I have this file:

main.py

%load_ext autotime
import initialize_logging
import logging

logger = logging.getLogger('attribution')
logger.setLevel(logging.WARNING)
logger.info('TEST')

The above results in nothing being logged to my file and nothing being output.

However in main, if I setLevel(logging.INFO) then everything gets written to file and to the sterrout (I’m using a jupyter notebook so it prints on screen immediatly.)

The behavior I would like is that the user of my notebook can setLevel to determine what they want to see printed on screen but no matter what all logs get sent to the log file.

How do I do this?

Asked By: Jamie Marshall

||

Source

Answer 1

Create a Logger subclass that is enabled for all levels and set it as the LoggerClass. Override the isEnabledFor function to return True regardless of the Logger’s level. Override the setLevel function to find all of the Logger’s StreamHandlers and set their level to be the same as the Logger. The behavior you want will rely on the level set for the handlers.

logging_init.py


class AllLevels(logging.Logger):
    def isEnabledFor(self,level):
        """
        Is this logger enabled for level 'level'?
        """
        return True
    def setLevel(self,level):
        super().setLevel(level)
        for h in self.handlers:
            if isinstance(h,logging.StreamHandler):
                h.setLevel(self.level)

filename="attribution.log"

logging.setLoggerClass(AllLevels)

logger = logging.getLogger('attribution')

formatter = logging.Formatter('%(asctime)s - %(levelname)s: %(message)s')
fh = logging.FileHandler(filename)
fh.setLevel(logging.DEBUG)
fh.setFormatter(formatter)

ch = logging.StreamHandler()
ch.setLevel(logging.WARNING)
ch.setFormatter(formatter)

logger.setLevel(logging.DEBUG)
logger.addHandler(fh)
logger.addHandler(ch)

caveat: I have not tried to think of all or any ways that this could go wrong. I only attempted to create the behavior.

Maybe instead of setting all StreamHandlers to the same level as the logger, only set a StreamHandler with a specific name.

Nothing will stop someone from inspecting the logger; finding the handlers; arbitrarily changing their levels. Maybe use a FileHandler subclass with its setLevel function overridden.

class AlwaysFileHandler(logging.FileHandler):
    def setLevel(self,level):
        """
        Set the logging level of this handler to DEBUG.
        """
        self.level = logging.DEBUG

Answered By: wwii

Guarantee all logs go to file

Question:

Answers: