How to implement a Global Python Logger?

Question

How can I implement a global logger for all of my python files? Some relevant SE questions are one and two, but neither do exactly what I want, simply. I want the log file output to be seen in the console as well.

Asked By: xinthose

||

Source

Answer 1

Here is the solution that has been working for me for years:

main.py

import log
import test

logger = log.setup_custom_logger('root')

def main():
    logger.info("informational message")
    logger.debug("debugging message")
    logger.critical("critical message")
    test.test_message()
    return 0

if __name__ == '__main__':
    main()

log.py

import logging
import logging.handlers

def setup_custom_logger(name):
    # logger settings
    log_file = "log/testing.log"
    log_file_max_size = 1024 * 1024 * 20 # megabytes
    log_num_backups = 3
    log_format = "%(asctime)s [%(levelname)s] %(filename)s/%(funcName)s:%(lineno)s >> %(message)s"
    log_filemode = "w" # w: overwrite; a: append
    
    # setup logger
    logging.basicConfig(filename=log_file, format=log_format,
                        filemode=log_filemode, level=logging.DEBUG)
    rotate_file = logging.handlers.RotatingFileHandler(
        log_file, maxBytes=log_file_max_size, backupCount=log_num_backups
    )
    logger = logging.getLogger(name)
    logger.addHandler(rotate_file)

    # print log messages to console
    consoleHandler = logging.StreamHandler()
    logFormatter = logging.Formatter(log_format)
    consoleHandler.setFormatter(logFormatter)
    logger.addHandler(consoleHandler)
    
    return logger
        
# source: https://docs.python.org/2/howto/logging.html
# logger.debug("")      // Detailed information, typically of interest only when diagnosing problems.
# logger.info("")       // Confirmation that things are working as expected.
# logger.warning("")    // An indication that something unexpected happened, or indicative of some problem in the near future
# logger.error("")      // Due to a more serious problem, the software has not been able to perform some function.
# logger.critical("")   // A serious error, indicating that the program itself may be unable to continue running.

test.py

import logging

logger = logging.getLogger('root')

def test_message():
    logger.warning("warning message")
    logger.error("error message")
    return 0

Make sure you have a directory called log for the file testing.log from where you call python3 main.py from. For each file that uses logging, you have to call these two lines like in test.py: import logging and logger = logging.getLogger('root').

Sample console and log file output:

2016-06-21 23:24:43,945 [DEBUG] main.py/main:11 >> debugging message
2016-06-21 23:24:43,945 [INFO] main.py/main:10 >> informational message    
2016-06-21 23:24:43,945 [CRITICAL] main.py/main:12 >> critical message
2016-06-21 23:24:43,946 [WARNING] test.py/test_message:9 >> warning message
2016-06-21 23:24:43,946 [ERROR] test.py/test_message:10 >> error message

Answered By: xinthose

How to implement a Global Python Logger?

Question:

Answers:

Sample console and log file output: