Python Logging – Disable logging from imported modules

@Bakuriu quite elegantly explains the function. Conversely, you can use the getLogger() method to retrieve and reconfigure/disable the unwanted loggers.

I also wanted to add the logging.fileConfig() method accepts a parameter called disable_existing_loggers which will disable any loggers previously defined (i.e., in imported modules).

If you’re going to use the python logging package, it’s a common convention to define a logger in every module that uses it.

logger = logging.getLogger(__name__)

Many popular python packages do this, including requests. If a package uses this convention, it’s easy to enable/disable logging for it, because the logger name will be the same name as the package (or will be a child of that logger). You can even log it to the same file as your other loggers.

logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)

requests_logger = logging.getLogger('requests')
requests_logger.setLevel(logging.DEBUG)

handler = logging.StreamHandler()
handler.setLevel(logging.DEBUG)
logger.addHandler(handler)
requests_logger.addHandler(handler)

This disables all existing loggers, such as those created by imported modules, while still using the root logger (and without having to load an external file).

import logging.config
logging.config.dictConfig({
    'version': 1,
    'disable_existing_loggers': True,
})

Note that you need to import all modules you don’t want logged first! Otherwise those won’t be considered as "existing loggers". It will then disable all loggers from those modules. This might lead you to also miss out on important errors!

For more detailed examples using related options for configuration, see https://gist.github.com/st4lk/6287746, and here is a (partially working) example using YAML for config with the coloredlog library.

I was getting debug logs from matplotlib despite following the pretty straightforward documentation at the logging advanced tutorial
and the troubleshooting. I was initiating my logger in main() of one file and importing a function to create a plot from another file (where I had imported matplotlib).

What worked for me was setting the level of matplotlib before importing it, rather than after as I had for other modules in my main file. This seemed counterintuitive to me so if anyone has insight into how you can set the config for a logger that hasn’t been imported yet I’d be curious to find out how this works.

In my main file:

import logging
import requests
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
logging.getLogger('requests').setLevel(logging.DEBUG)

def main():
  ...

In my plot.py file:

import logging
logging.getLogger('matplotlib').setLevel(logging.WARNING)
import matplotlib.pyplot as plt

def generatePlot():
  ...

You could use something like:

logging.getLogger("imported_module").setLevel(logging.WARNING)
logging.getLogger("my_own_logger_name").setLevel(logging.DEBUG)

This will set my own module’s log level to DEBUG, while preventing the imported module from using the same level.

Note:
"imported_module" can be replaced with imported_module.__name__ (without quotes), and "my_own_logger_name" can be replaced by __name__ if that’s the way you prefer to do it.

I had the same problem.
I have a logging_config.py file which I import in all other py files.
In logging_config.py file I set root logger logging level to ERROR (by default its warning):

logging.basicConfig(
    handlers=[
        RotatingFileHandler('logs.log',maxBytes=1000, backupCount=2),
        logging.StreamHandler(), #print to console
    ],
    level=logging.ERROR
)

In other modules I import logging_config.py and declare a new logger and set its level to debug:

log = logging.getLogger(__name__)
log.setLevel(logging.DEBUG)

This way everything I log in my py files is logged, but stuff logged at debug and info level by imported modules like urllib, request,boto3 etc is not logged. If there is some error in those import module then its logged, since I set root loggers level to ERROR.

Another thing to consider is the propagate property of the Logger class.

For example, py-suds library for handling soap calls, even put to ERROR

logging.getLogger('suds.client').setLevel(logging.ERROR)
logging.getLogger('suds.transport').setLevel(logging.ERROR)
logging.getLogger('suds.xsdschema').setLevel(logging.ERROR)
logging.getLogger('suds.wsdl').setLevel(logging.ERROR)

logs logs about a module called sxbasics.py creationg a huge amount of logs

that because the propagation of the logs is True by default, setting to False, instead, i recovered 514MB of logs.

import logging
logging.getLogger("suds").propagate = False
logging.getLogger('suds.client').setLevel(logging.ERROR)
logging.getLogger('suds.transport').setLevel(logging.ERROR)
logging.getLogger('suds.xsdschema').setLevel(logging.ERROR)
logging.getLogger('suds.wsdl').setLevel(logging.ERROR)

Simply doing something like this solves the problem:

logging.config.dictConfig({'disable_existing_loggers': True,})

NOTE: Wherever you’re placing this line, make sure all the imports everywhere in your project are done within that file itself. 🙂

After trying various answers in this thread and other forums, I found this method efficient at silencing other modules’ loggers. This was inspired by the following link:

https://kmasif.com/2019-11-12-ignore-logging-from-imported-module/

import logging
# Initialize your own logger
logger = logging.getLogger('<module name>')
logger.setLevel(logging.DEBUG)

# Silence other loggers
for log_name, log_obj in logging.Logger.manager.loggerDict.items():
     if log_name != '<module name>':
          log_obj.disabled = True

Note that you would want to do this after importing other modules. However, I found this to be a convenient and fast way to disabled other modules’ loggers.

Inspired by @brendan’s answer, I created a simple logger_blocklist to which I can just all loggers I want to increase the level of. Thus, I was able to silence them all at once:

import logging

logging.basicConfig(level=logging.DEBUG)

logger_blocklist = [
    "fiona",
    "rasterio",
    "matplotlib",
    "PIL",
]

for module in logger_blocklist:
    logging.getLogger(module).setLevel(logging.WARNING)

You can find the logger’s name from the format="%(name)s" parameter, also included in logging.basicConfig(), for example: DEBUG:matplotlib.font_manager:findfont: score(FontEntry(fname='/usr/share/fonts/truetype/ubuntu/Ubuntu-R.ttf', name='Ubuntu', style='normal', variant='normal', weight=400, stretch='normal', size='scalable')) = 10.05.

In this case you want to add the logger matplotlib to your block list.

As opposed to @Finn’s answer it was not necessary to have it outside main(), etc.

In the case that your imported module uses the root logger, then you can add a filter (https://docs.python.org/3/library/logging.html#filter-objects) on the root logger to remove unwanted logs:

class FilterUnwantedRecords():
    def filter(self, record):
        if '3rdpartymodule' in record.pathname:
            return False
        return True

logging.getLogger().addFilter(FilterUnwantedRecords())

# mymodule.py

# Import 'logging' module from standard library
import logging

# Set all loggers to show only WARNING and above
logging.basicConfig(level=logging.WARNING)

# Create your module-specific logger (__name__ will be 'mymodule' in this example)
logger = logging.getLogger(__name__)

# Set your desired logging level for your logger
logger.setLevel(level=logging.DEBUG)

# Add handlers if you want
fh = logging.StreamHandler()
fh_formatter = logging.Formatter('%(asctime)s %(levelname)s %(lineno)d:% filename)s(%(process)d) - %(message)s')
fh.setFormatter(fh_formatter)
logger.addHandler(fh)

Another example, this time using a helper function and the RichHandler from the rich 3rd-party module.

# loghelper.py
import logging
from rich.logging import RichHandler

def new_logger(logger_name: str) -> logging.Logger:
    FORMAT = "%(message)s"
    logging.basicConfig(
        level=logging.WARNING, format=FORMAT, datefmt="[%X]", handlers=[RichHandler()]
    )
    log = logging.getLogger(logger_name)
    # Add handlers ...
    # ...
    return log

Then import the helper function into your module

# mymodule.py

from loghelper import new_logger

# Nice and tidy!
logger = new_logger(__name__)