Removing handlers from python's logging loggers

Question

I am playing with Python’s logging system. I have noticed a strange behavior while removing handlers from a Logger object in a loop. Namely, my for loop removes all but one handler. Additional call to .removeHandler removes the last handler smoothly. No error messages are issued during the calls.

This is the test code:

import logging
import sys
logging.basicConfig()
dbg = logging.getLogger('dbg')
dbg.setLevel(logging.DEBUG)

testLogger = logging.getLogger('mylogger')
sh = logging.StreamHandler(sys.stdout)
fh = logging.FileHandler('mylogfile.log')
dbg.debug('before adding handlers: %d handlers'%len(testLogger.handlers))
testLogger.addHandler(fh)
testLogger.addHandler(sh)

dbg.debug('before removing. %d handlers: %s'%(len(testLogger.handlers), 
                                              str(testLogger.handlers)))
for h in testLogger.handlers:
    dbg.debug('removing handler %s'%str(h))
    testLogger.removeHandler(h)
    dbg.debug('%d more to go'%len(testLogger.handlers))

#HERE I EXPECT THAT NO HANDLER WILL REMAIN    
dbg.debug('after removing: %d handlers: %s'%(len(testLogger.handlers), 
                                              str(testLogger.handlers)))
if len(testLogger.handlers) > 0:
    #Why is this happening?
    testLogger.removeHandler(testLogger.handlers[0])
dbg.debug('after manually removing the last handler: %d handlers'%len(testLogger.handlers))

I expect that at the end of the loop no handlers will remain in the testLogger object, however
the last call to .removeHandler apparently fails, as can be seen from the output below. Nevertheless
additional call to this function removes the handler as expected. Here is the output:

DEBUG:dbg:before adding handlers: 0 handlers
DEBUG:dbg:before removing. 2 handlers: [<logging.FileHandler instance at 0x021263F0>, <logging.StreamHandler instance at 0x021262B0>]
DEBUG:dbg:removing handler <logging.FileHandler instance at 0x021263F0>
DEBUG:dbg:1 more to go
DEBUG:dbg:after removing: 1 handlers: [<logging.StreamHandler instance at 0x021262B0>]
DEBUG:dbg:after manually removing the last handler: 0 handlers

More interestingly, if I replace the original loop with the following one, the loop
works as expected and no handlers remain in the testLogger object at the end of the loop.
Here is the modified loop:

while len(testLogger.handlers) > 0:
    h = testLogger.handlers[0]
    dbg.debug('removing handler %s'%str(h))
    testLogger.removeHandler(h)
    dbg.debug('%d more to go'%len(testLogger.handlers))

What explains this behaviour? Is this a bug or am I missing something?

Asked By: Boris Gorelik

||

Source

Answer 1

This isn’t logger-specific behaviour. Never mutate (insert/remove elements) the list you’re currently iterating on. If you need, make a copy. In this case testLogger.handlers.clear() should do the trick.

Answered By: Cat Plus Plus

Answer 2

If you don’t want to delete them all (thanks for the tip @CatPlusPlus):

testLogger.handlers = [
    h for h in testLogger.handlers if not isinstance(h, logging.StreamHandler)]

Answered By: hobs

Answer 3

I’ve just found out that you can also do that within a logging .ini file, with the following block:

[logger_stpipe]
handlers=
propagate=1
qualname=stpipe

It basically deactivates all handlers for a given logger. But it’s somewhat limited because you have to know the Logger’s name in advance.

Answered By: Autiwa

Answer 4

instead of mutating undocumented .handler:

Option 1

logging.getLogger().removeHandler(logging.getLogger().handlers[0])

this way you remove exactly the preexisting handler object via offical api. Or to remove all handlers:

logger = logging.getLogger()
while logger.hasHandlers():
    logger.removeHandler(logger.handlers[0])

Option 2

logging.config.dictConfig(config={'level': logging.DEBUG, 'handlers': []}

Not only removes but prevents its creation. List root will have [] handlers

Answered By: droid192

Answer 5

I would say if the intent for removing the logging handlers is to prevent sending logs to additional handlers (this is the case most of the times) then an alternative approach I follow is to set propagate to False for your current logger.

logger = logging.getLogger(__name__)
logger.propagate = False

That way your log messages will only go to the handlers you explicitly add and will not propagate to parent loggers.

Answered By: Rohit

Removing handlers from python's logging loggers

Question:

Answers:

Option 1

Option 2