Dynamically changing log level without restarting the application

This might be what you are looking for:

import logging
logging.getLogger().setLevel(logging.INFO)

Note that getLogger() called without any arguments returns the root logger.

It is certainly possible to use fileConfig() to change logging configuration on the fly, though for simple changes a programmatic approach as suggested in Martijn Pieters’ answer might be appropriate. Logging even provides a socket server to listen for config changes using the listen() / stopListening() APIs, as documented here. To get logging to listen on a particular port, you use

t = logging.config.listen(PORT_NUMBER)
t.start()

and to stop listening, call

logging.config.stopListening()

To send data to the server, you can use e.g.

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('localhost', PORT_NUMBER))
with open(CONFIG_FILE) as f:
    data_to_send = f.read()
s.send(struct.pack('>L', len(data_to_send)))
s.send(data_to_send)
s.close()

Update: Due to backwards-compatibility constraints, the internal implementation of the fileConfig() call means that you can’t specify disable_existing_loggers=False in the call, which makes this feature less useful in certain scenarios. You can use the same API to send a JSON file using the dictConfig schema, which will allow better control over the reconfiguration. This requires Python 2.7/3.2 or above (where dictConfig() was added). Or, you can use the stdlib code to implement your own listener which works in the same way but which is tailored to your specific needs.

Depending on your app, you first need to find a way for reloading that file or resetting the log level based on your own config file during execution.

Easiest way would be to use a timer. Either use threading to do that, or make your async framework to do that (if you use any; they usually implement it).

Using threading.Timer:

import threading
import time


def reset_level():
    # you can reload your own config file or use logging.config.fileConfig here
    print 'Something else'
    pass


t = threading.Timer(10, reset_level)
t.start()

while True:
    # your app code
    print 'Test'
    time.sleep(2)

Output:

Test
Test
Test
Test
Test
Something else
Test
Test

Update:
Please check the solution proposed by Martijn Pieters.

I finally settled with using inotify and gevent to check for the file write operation, and once I know the file has been changed then I go and set the level for each logger I have based on the config.

import gevent
import gevent_inotifyx as inotify
from gevent.queue import Queue

class FileChangeEventProducer(gevent.Greenlet):
    def __init__(self, fd, queue):
        gevent.Greenlet.__init__(self)
        self.fd = fd
        self.queue = queue

    def _run(self):
        while True:
            events = inotify.get_events(self.fd)
            for event in events:
                self.queue.put(event)
                gevent.sleep(0)


class FileChangeEventConsumer(gevent.Greenlet):
    def __init__(self, queue, callBack):
        gevent.Greenlet.__init__(self)
        self.queue = queue
        self.callback = callBack

    def _run(self):
        while True:
            _ = self.queue.get()
            self.callback()
            gevent.sleep(0)


class GeventManagedFileChangeNotifier:
    def __init__(self, fileLocation, callBack):
        self.fileLocation = fileLocation
        self.callBack = callBack
        self.queue = Queue()
        self.fd = inotify.init()
        self.wd = inotify.add_watch(self.fd, self.fileLocation, inotify.IN_CLOSE_WRITE)


    def start(self):
        producer = FileChangeEventProducer(self.fd, self.queue)
        producer.start()
        consumer = FileChangeEventConsumer(self.queue, self.callBack)
        consumer.start()
        return (producer, consumer)

The above code gets used like below,

    def _setUpLoggingConfigFileChangeNotifier(self):
        loggingFileNameWithFullPath = self._getFullPathForLoggingConfig()
        self.gFsNotifier = GeventManagedFileChangeNotifier(loggingFileNameWithFullPath, self._onLogConfigChanged)
        self.fsEventProducer, self.fsEventConsumer = self.gFsNotifier.start()


    def _onLogConfigChanged(self):
        self.rootLogger.info('Log file config has changed - examining the changes')
        newLoggingConfig = Config(self.resourcesDirectory, [self.loggingConfigFileName]).config.get('LOG')
        self.logHandler.onLoggingConfigChanged(newLoggingConfig)

Once I have the new log file config I can wire in the right logging level for each logger from config. I just wanted to share the answer and it might help someone if they are trying to use it with gevent.

In addition to the accepted answer: Depending on how you initialized the logger, you might also have to update the logger’s handlers:

import logging

level = logging.DEBUG
logger = logging.getLogger()
logger.setLevel(level)
for handler in logger.handlers:
    handler.setLevel(level)

Expanding on sfinken’s answer, and Starman’s subsequent comment, you can also check the type of the handler to target a specific outputter – for instance:

import logging
logger = logging.getLogger()
for handler in logger.handlers:
    if isinstance(handler, type(logging.StreamHandler())):
        handler.setLevel(logging.DEBUG)
        logger.debug('Debug logging enabled')

import logging
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
logger.debug('This message should appear on the console')
logger.warning('So should this')
logger.setLevel(logging.WARNING)
logger.debug('This message should NOT appear on the console')
logger.warning('This one should')

You can even set up different loggers