Can Python's logging format be modified depending on the message log level?

Question:

I’m using Python’s logging mechanism to print output to the screen. I could do this with print statements, but I want to allow a finer-tuned granularity for the user to disable certain types of output. I like the format printed for errors, but would prefer a simpler format when the output level is “info.”

For example:

  logger.error("Running cmd failed")
  logger.info("Running cmd passed")

In this example, I would like the format of the error to be printed differently:

# error
Aug 27, 2009 - ERROR: Running cmd failed
# info
Running cmd passed

Is it possible to have different formats for different log levels without having multiple logging objects? I’d prefer to do this without modifying the logger once it’s created since there are a high number of if/else statements to determine how the output should be logged.

Asked By: bedwyr

||

Answers:

Yes, you can do this by having a custom Formatter class:

class MyFormatter(logging.Formatter):
    def format(self, record):
        #compute s according to record.levelno
        #for example, by setting self._fmt
        #according to the levelno, then calling
        #the superclass to do the actual formatting
        return s

Then attach a MyFormatter instance to your handlers.

Answered By: Vinay Sajip

I just ran into this issue and had trouble filling in the “holes” left in the above example. Here’s a more complete, working version that I used. Hopefully this helps someone:

# Custom formatter
class MyFormatter(logging.Formatter):

    err_fmt  = "ERROR: %(msg)s"
    dbg_fmt  = "DBG: %(module)s: %(lineno)d: %(msg)s"
    info_fmt = "%(msg)s"


    def __init__(self, fmt="%(levelno)s: %(msg)s"):
        logging.Formatter.__init__(self, fmt)


    def format(self, record):

        # Save the original format configured by the user
        # when the logger formatter was instantiated
        format_orig = self._fmt

        # Replace the original format with one customized by logging level
        if record.levelno == logging.DEBUG:
            self._fmt = MyFormatter.dbg_fmt

        elif record.levelno == logging.INFO:
            self._fmt = MyFormatter.info_fmt

        elif record.levelno == logging.ERROR:
            self._fmt = MyFormatter.err_fmt

        # Call the original formatter class to do the grunt work
        result = logging.Formatter.format(self, record)

        # Restore the original format configured by the user
        self._fmt = format_orig

        return result

Edit:

Compliments of Halloleo, here’s an example of how to use the above in your script:

fmt = MyFormatter()
hdlr = logging.StreamHandler(sys.stdout)

hdlr.setFormatter(fmt)
logging.root.addHandler(hdlr)
logging.root.setLevel(DEBUG)

Edit 2:

Python3 logging has changed a bit. See here for a Python3 approach.

Answered By: JS.

And again like JS answer but more compact.

class SpecialFormatter(logging.Formatter):
    FORMATS = {logging.DEBUG :"DBG: %(module)s: %(lineno)d: %(message)s",
               logging.ERROR : "ERROR: %(message)s",
               logging.INFO : "%(message)s",
               'DEFAULT' : "%(levelname)s: %(message)s"}

    def format(self, record):
        self._fmt = self.FORMATS.get(record.levelno, self.FORMATS['DEFAULT'])
        return logging.Formatter.format(self, record)

hdlr = logging.StreamHandler(sys.stderr)
hdlr.setFormatter(SpecialFormatter())
logging.root.addHandler(hdlr)
logging.root.setLevel(logging.INFO)
Answered By: estani

This is an adaptation of estani’s answer to the new implementation of logging.Formatter which now relies on formatting styles. Mine relies on '{' style format, but it can be adapted. Could be refined to be more general and allow selection of formatting style and custom messages as arguments to __init__, too.

class SpecialFormatter(logging.Formatter):
    FORMATS = {logging.DEBUG : logging._STYLES['{']("{module} DEBUG: {lineno}: {message}"),
           logging.ERROR : logging._STYLES['{']("{module} ERROR: {message}"),
           logging.INFO : logging._STYLES['{']("{module}: {message}"),
           'DEFAULT' : logging._STYLES['{']("{module}: {message}")}

    def format(self, record):
        # Ugly. Should be better
        self._style = self.FORMATS.get(record.levelno, self.FORMATS['DEFAULT'])
        return logging.Formatter.format(self, record)

hdlr = logging.StreamHandler(sys.stderr)
hdlr.setFormatter(SpecialFormatter())
logging.root.addHandler(hdlr)
logging.root.setLevel(logging.INFO)
Answered By: Evpok

The above solution works with 3.3.3 release.
However with 3.3.4 you get the following error.

FORMATS = { logging.DEBUG : logging._STYLES['{']("{module} DEBUG: {lineno}: {message}"),

TypeError: ‘tuple’ object is not callable

After some searching around in the logging class
Liblogging__init__.py
I found that a data structure has changed from 3.3.3 to 3.3.4 that causes the issue

3.3.3

_STYLES = {
    '%': PercentStyle,
    '{': StrFormatStyle,
    '$': StringTemplateStyle
}

3.3.4

_STYLES = {
   '%': (PercentStyle, BASIC_FORMAT),
   '{': (StrFormatStyle, '{levelname}:{name}:{message} AA'),
    '$': (StringTemplateStyle, '${levelname}:${name}:${message} BB'),
}

The updated solution is therefore

class SpecialFormatter(logging.Formatter):
     FORMATS = {logging.DEBUG : logging._STYLES['{'][0]("{module} DEBUG: {lineno}: {message}"),
       logging.ERROR : logging._STYLES['{'][0]("{module} ERROR: {message}"),
       logging.INFO : logging._STYLES['{'][0]("{module}: {message}"),
       'DEFAULT' : logging._STYLES['{'][0]("{module}: {message}")}

 def format(self, record):
    # Ugly. Should be better
    self._style = self.FORMATS.get(record.levelno, self.FORMATS['DEFAULT'])
    return logging.Formatter.format(self, record)
Answered By: user1837990

If you are just looking to skip formatting certain levels, you can do something simpler than the other answers like the following:

class FormatterNotFormattingInfo(logging.Formatter):
    def __init__(self, fmt = '%(levelname)s:%(message)s'):
        logging.Formatter.__init__(self, fmt)

    def format(self, record):
        if record.levelno == logging.INFO:
            return record.getMessage()
        return logging.Formatter.format(self, record)

This also has the advantage of working before and after the 3.2 release by not using internal variables like self._fmt nor self._style.

Answered By: JDiMatteo

Instead of relying on styles or internal fields, you could also create a Formatter that delegates to other formatters depending on record.levelno (or other criteria). This is a slightly cleaner solution in my humble opinion. Code below should work for any python version >= 2.7:

The simple way would look something like this:

class MyFormatter(logging.Formatter):

    default_fmt = logging.Formatter('%(levelname)s in %(name)s: %(message)s')
    info_fmt = logging.Formatter('%(message)s')

    def format(self, record):
        if record.levelno == logging.INFO:
            return self.info_fmt.format(record)
        else:
            return self.default_fmt.format(record)

But you could make it more generic:

class VarFormatter(logging.Formatter):

    default_formatter = logging.Formatter('%(levelname)s in %(name)s: %(message)s')

    def __init__(self, formats):
        """ formats is a dict { loglevel : logformat } """
        self.formatters = {}
        for loglevel in formats:
            self.formatters[loglevel] = logging.Formatter(formats[loglevel])

    def format(self, record):
        formatter = self.formatters.get(record.levelno, self.default_formatter)
        return formatter.format(record)

I used a dict as input here, but obviously you could also use tuples, **kwargs, whatever floats your boat. This would then be used like:

formatter = VarFormatter({logging.INFO: '[%(message)s]', 
                          logging.WARNING: 'warning: %(message)s'})
<... attach formatter to logger ...>
Answered By: Joris

ONE WAY OF DOING THIS

Define a class

import logging

class CustomFormatter(logging.Formatter):
    """Logging Formatter to add colors and count warning / errors"""

    grey = "x1b[38;21m"
    yellow = "x1b[33;21m"
    red = "x1b[31;21m"
    bold_red = "x1b[31;1m"
    reset = "x1b[0m"
    format = "%(asctime)s - %(name)s - %(levelname)s - %(message)s (%(filename)s:%(lineno)d)"

    FORMATS = {
        logging.DEBUG: grey + format + reset,
        logging.INFO: grey + format + reset,
        logging.WARNING: yellow + format + reset,
        logging.ERROR: red + format + reset,
        logging.CRITICAL: bold_red + format + reset
    }

    def format(self, record):
        log_fmt = self.FORMATS.get(record.levelno)
        formatter = logging.Formatter(log_fmt)
        return formatter.format(record)

Instantiate logger

# create logger with 'spam_application'
logger = logging.getLogger("My_app")
logger.setLevel(logging.DEBUG)

# create console handler with a higher log level
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)

ch.setFormatter(CustomFormatter())

logger.addHandler(ch)

And use!

logger.debug("debug message")
logger.info("info message")
logger.warning("warning message")
logger.error("error message")
logger.critical("critical message")

Result
enter image description here

Answered By: Sergey Pleshakov