Logging, remove, inspect, and modify handlers configured by fileConfig()

Question:

How can I remove, inspect, and modify handlers configured for my loggers using the fileConfig() function?

For removing, there is Logger.removeHandler(hdlr) method, but how do I get the handler in first place if it was configured from file?

Asked By: Pawel Furmaniak

||

Answers:

logger.handlers contains a list with all handlers of a logger.

Answered By: leoluk

Another approach might be to use a JSON or YAML config file which gets loaded into a dictionary which you can then view/manipulate before it’s passed to the logger.config.

import yaml
import logging.config

with open (LOG_CONFIG, 'rt') as f:
   config=yaml.safe_load(f)
   config['handlers']['error_file_handler']['filename']='foo'
logging.config.dictConfig(config)
Answered By: perlyking

This code will print all the loggers and for each logger its handlers

for k,v in  logging.Logger.manager.loggerDict.items()  :
        print('+ [%s] {%s} ' % (str.ljust( k, 20)  , str(v.__class__)[8:-2]) )
        if not isinstance(v, logging.PlaceHolder):
            for h in v.handlers:
                print('     +++',str(h.__class__)[8:-2] )

This will print out the loggers and handlers in your system including their states and levels.

This will help you debug your logging issues

output:
+ [root                ] {logging.RootLogger} {DEBUG}
-------------------------
   -name=root
   -handlers=[<logging.FileHandler object at 0x7fc599585390>, <logging.StreamHandler object at 0x7fc599585550>]
   -filters=[]
   -propagate=True
   -level=10
   -disabled=False
   -parent=None
     +++logging.FileHandler {NOTSET}
   -stream=<_io.TextIOWrapper name='/dev/logs/myapp.log' mode='w' encoding='UTF-8'>
   -mode=w
   -filters=[]
   -encoding=None
   -baseFilename=/home/dev/logs/myapp.log
   -level=0
   -lock=<unlocked _thread.RLock object owner=0 count=0 at 0x7fc5a85a4240>
   -delay=False
   -_name=None
   -formatter=<logging.Formatter object at 0x7fc599585358>
     +++logging.StreamHandler {DEBUG}
   -lock=<unlocked _thread.RLock object owner=0 count=0 at 0x7fc5a85a4210>
   -filters=[]
   -stream=<ipykernel.iostream.OutStream object at 0x7fc5aa6abb00>
   -level=10
   -_name=None
   -formatter=<logging.Formatter object at 0x7fc5995853c8>
+ [PathFinder          ] {logging.Logger} {NOTSET}
-------------------------
   -name=PathFinder
   -handlers=[]
   -filters=[]
   -manager=<logging.Manager object at 0x7fc5b09757f0>
   -propagate=True
   -level=0
   -disabled=False
   -parent=<logging.RootLogger object at 0x7fc5b09757b8>
Answered By: Mickey Perlstein

The answer by Mickey-Perlstein was just what I was looking for, but the listing appears to come from a much more complete version than the code provided.

This code is kind of even cruder, but crucially for my use and understanding, includes the root logger.

import logging

def listloggers():
    rootlogger = logging.getLogger()
    print(rootlogger)
    for h in rootlogger.handlers:
        print('     %s' % h)

    for nm, lgr in logging.Logger.manager.loggerDict.items():
        print('+ [%-20s] %s ' % (nm, lgr))
        if not isinstance(lgr, logging.PlaceHolder):
            for h in lgr.handlers:
                print('     %s' % h)

Output:

<RootLogger root (DEBUG)>
     <TimedRotatingFileHandler /path/to/myapp.log (DEBUG)>
     <StreamHandler <stdout> (DEBUG)>
+ [concurrent.futures  ] <Logger concurrent.futures (DEBUG)>
+ [concurrent          ] <logging.PlaceHolder object at 0x7f72f624eba8>
+ [asyncio             ] <Logger asyncio (DEBUG)>
+ [myapp               ] <Logger myapp (DEBUG)>
+ [flask.app           ] <Logger flask.app (DEBUG)>
+ [flask               ] <Logger flask (DEBUG)>
+ [werkzeug            ] <Logger werkzeug (ERROR)>
Answered By: Alias_Knagg

Re removing handlers – if it helps anyone, I wanted to remove all handlers for all the logs I’d created (I was learning about Python logging) and came up with this:

def log_close():
    # Get all loggers
    loggers = [logging.getLogger(name) if 'your_app_name' in name else None for name in logging.root.manager.loggerDict]

    # For each valid logger remove all handlers
    for log in loggers:
        if log != None:
            while bool(len(log.handlers)):
                for handler in log.handlers:
                    log.removeHandler(handler)
# Set up log
log = logging.getLogger('your_app_name')
log.setLevel('DEBUG')
log_format = '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
date_format = '%Y-%m-%d %H:%M:%S'
formatter = logging.Formatter(log_format, date_format)
log_filename = 'your_app_name.log'
log_write_mode = 'w+'
file_handler = logging.FileHandler(log_filename, log_write_mode)
file_handler.setFormatter(formatter)
log.addHandler(file_handler)
console_handler = logging.StreamHandler()
console_handler.setFormatter(formatter)
log.addHandler(console_handler)
Answered By: Sicro
Categories: questions Tags: ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.