Django Setup Default Logging
Question:
I can’t seem to figure out how to setup a “default” logger for my Django installation. I would like to use Django 1.3’s new LOGGING setting in settings.py.
I’ve looked at the Django Logging Doc’s example, but it looks to me like they only setup handlers which will do logging for particular loggers. In the case of their example they setup handler for the loggers named ‘django’,’django.request’, and ‘myproject.custom’.
All I want to do is setup a default logging.handlers.RotatingFileHandler which will handle all loggers by default. i.e., if I make a new module somewhere in my project and it is denoted by something like: my_app_name.my_new_module, I should be able to do this and have all logging goto the rotating file logs.
# In file './my_app_name/my_new_module.py'
import logging
logger = logging.getLogger('my_app_name.my_new_module')
logger.debug('Hello logs!') # <-- This should get logged to my RotatingFileHandler that I setup in `settings.py`!
Answers:
Figured it out…
You set the ‘catch all’ logger by referencing it with the empty string: ''.
As an example, in the following setup I have the all log events getting saved to logs/mylog.log, with the exception of django.request log events which will be saved to logs/django_request.log. Because 'propagate' is set to False for my django.request logger, the log event will never reach the the ‘catch all’ logger.
LOGGING = {
'version': 1,
'disable_existing_loggers': True,
'formatters': {
'standard': {
'format': '%(asctime)s [%(levelname)s] %(name)s: %(message)s'
},
},
'handlers': {
'default': {
'level':'DEBUG',
'class':'logging.handlers.RotatingFileHandler',
'filename': 'logs/mylog.log',
'maxBytes': 1024*1024*5, # 5 MB
'backupCount': 5,
'formatter':'standard',
},
'request_handler': {
'level':'DEBUG',
'class':'logging.handlers.RotatingFileHandler',
'filename': 'logs/django_request.log',
'maxBytes': 1024*1024*5, # 5 MB
'backupCount': 5,
'formatter':'standard',
},
},
'loggers': {
'': {
'handlers': ['default'],
'level': 'DEBUG',
'propagate': True
},
'django.request': {
'handlers': ['request_handler'],
'level': 'DEBUG',
'propagate': False
},
}
}
As you said in your answer, Chris, one option to define a default logger is to use the empty string as its key.
However, I think the intended way is to define a special logger under the root key of the logging configuration dictionary. I found this in the Python documentation:
root – this will be the configuration for the root logger. Processing of the configuration will be as for any logger, except that the propagate setting will not be applicable.
Here’s the configuration from your answer changed to use the root key:
LOGGING = {
'version': 1,
'disable_existing_loggers': True,
'formatters': {
'standard': {
'format': '%(asctime)s [%(levelname)s] %(name)s: %(message)s'
},
},
'handlers': {
'default': {
'level':'DEBUG',
'class':'logging.handlers.RotatingFileHandler',
'filename': 'logs/mylog.log',
'maxBytes': 1024*1024*5, # 5 MB
'backupCount': 5,
'formatter':'standard',
},
'request_handler': {
'level':'DEBUG',
'class':'logging.handlers.RotatingFileHandler',
'filename': 'logs/django_request.log',
'maxBytes': 1024*1024*5, # 5 MB
'backupCount': 5,
'formatter':'standard',
},
},
'root': {
'handlers': ['default'],
'level': 'DEBUG'
},
'loggers': {
'django.request': {
'handlers': ['request_handler'],
'level': 'DEBUG',
'propagate': False
},
}
}
To be fair, I can’t see any difference in behaviour between the two configurations. It appears that defining a logger with an empty string key will modify the root logger, because logging.getLogger('') will return the root logger.
The only reason I prefer 'root' over '' is that it is explicit about modifying the root logger. In case you were curious, 'root' overrides '' if you define both, just because the root entry is processed last.
import logging
logger = logging.getLogger(__name__)
after add:
logging.basicConfig(
level = logging.DEBUG,
format = '%(name)s %(levelname)s %(message)s',
)
we may change format to:
format = '"%(levelname)s:%(name)s:%(message)s" ',
or
format = '%(name)s %(asctime)s %(levelname)s %(message)s',
I made a quick sample to check what configuration is used when both root key and the empty '' logger are referenced in config dict.
import logging.config
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'fmt1': {
'format': '[FMT1] %(asctime)-15s %(message)s',
},
'fmt2': {
'format': '[FMT2] %(asctime)-15s %(message)s',
}
},
'handlers': {
'console1': {
'level': 'INFO',
'class': 'logging.StreamHandler',
'formatter': 'fmt1',
},
'console2': {
'level': 'INFO',
'class': 'logging.StreamHandler',
'formatter': 'fmt2',
},
},
# First config for root logger: console1 -> fmt1
'root': {
'handlers': ['console1'],
'level': 'DEBUG',
'propagate': True,
},
'loggers': {
# Second config for root logger: console2 -> fmt2
'': {
'handlers': ['console2'],
'level': 'DEBUG',
'propagate': True,
},
},
}
logging.config.dictConfig(LOGGING)
l1 = logging.getLogger()
l2 = logging.getLogger('')
root = logging.root
l1.info("l1")
l2.info("l2")
root.info("root logger")
Prints the following result:
[FMT1] 2018-12-18 17:24:47,691 l1
[FMT1] 2018-12-18 17:24:47,691 l2
[FMT1] 2018-12-18 17:24:47,691 root logger
indicating that configuration under root key has the highest priority. If the block is removed, the result is:
[FMT2] 2018-12-18 17:25:43,757 l1
[FMT2] 2018-12-18 17:25:43,757 l2
[FMT2] 2018-12-18 17:25:43,757 root logger
In both case, I was able to debug and determine that all three loggers (l1, l2 and root) referenced the same logger instance, the root logger.
Hope that will help others who, like me, were confused by the 2 different ways to configure the root logger.
I can’t seem to figure out how to setup a “default” logger for my Django installation. I would like to use Django 1.3’s new LOGGING setting in settings.py.
I’ve looked at the Django Logging Doc’s example, but it looks to me like they only setup handlers which will do logging for particular loggers. In the case of their example they setup handler for the loggers named ‘django’,’django.request’, and ‘myproject.custom’.
All I want to do is setup a default logging.handlers.RotatingFileHandler which will handle all loggers by default. i.e., if I make a new module somewhere in my project and it is denoted by something like: my_app_name.my_new_module, I should be able to do this and have all logging goto the rotating file logs.
# In file './my_app_name/my_new_module.py'
import logging
logger = logging.getLogger('my_app_name.my_new_module')
logger.debug('Hello logs!') # <-- This should get logged to my RotatingFileHandler that I setup in `settings.py`!
Figured it out…
You set the ‘catch all’ logger by referencing it with the empty string: ''.
As an example, in the following setup I have the all log events getting saved to logs/mylog.log, with the exception of django.request log events which will be saved to logs/django_request.log. Because 'propagate' is set to False for my django.request logger, the log event will never reach the the ‘catch all’ logger.
LOGGING = {
'version': 1,
'disable_existing_loggers': True,
'formatters': {
'standard': {
'format': '%(asctime)s [%(levelname)s] %(name)s: %(message)s'
},
},
'handlers': {
'default': {
'level':'DEBUG',
'class':'logging.handlers.RotatingFileHandler',
'filename': 'logs/mylog.log',
'maxBytes': 1024*1024*5, # 5 MB
'backupCount': 5,
'formatter':'standard',
},
'request_handler': {
'level':'DEBUG',
'class':'logging.handlers.RotatingFileHandler',
'filename': 'logs/django_request.log',
'maxBytes': 1024*1024*5, # 5 MB
'backupCount': 5,
'formatter':'standard',
},
},
'loggers': {
'': {
'handlers': ['default'],
'level': 'DEBUG',
'propagate': True
},
'django.request': {
'handlers': ['request_handler'],
'level': 'DEBUG',
'propagate': False
},
}
}
As you said in your answer, Chris, one option to define a default logger is to use the empty string as its key.
However, I think the intended way is to define a special logger under the root key of the logging configuration dictionary. I found this in the Python documentation:
root – this will be the configuration for the root logger. Processing of the configuration will be as for any logger, except that the
propagatesetting will not be applicable.
Here’s the configuration from your answer changed to use the root key:
LOGGING = {
'version': 1,
'disable_existing_loggers': True,
'formatters': {
'standard': {
'format': '%(asctime)s [%(levelname)s] %(name)s: %(message)s'
},
},
'handlers': {
'default': {
'level':'DEBUG',
'class':'logging.handlers.RotatingFileHandler',
'filename': 'logs/mylog.log',
'maxBytes': 1024*1024*5, # 5 MB
'backupCount': 5,
'formatter':'standard',
},
'request_handler': {
'level':'DEBUG',
'class':'logging.handlers.RotatingFileHandler',
'filename': 'logs/django_request.log',
'maxBytes': 1024*1024*5, # 5 MB
'backupCount': 5,
'formatter':'standard',
},
},
'root': {
'handlers': ['default'],
'level': 'DEBUG'
},
'loggers': {
'django.request': {
'handlers': ['request_handler'],
'level': 'DEBUG',
'propagate': False
},
}
}
To be fair, I can’t see any difference in behaviour between the two configurations. It appears that defining a logger with an empty string key will modify the root logger, because logging.getLogger('') will return the root logger.
The only reason I prefer 'root' over '' is that it is explicit about modifying the root logger. In case you were curious, 'root' overrides '' if you define both, just because the root entry is processed last.
import logging
logger = logging.getLogger(__name__)
after add:
logging.basicConfig(
level = logging.DEBUG,
format = '%(name)s %(levelname)s %(message)s',
)
we may change format to:
format = '"%(levelname)s:%(name)s:%(message)s" ',
or
format = '%(name)s %(asctime)s %(levelname)s %(message)s',
I made a quick sample to check what configuration is used when both root key and the empty '' logger are referenced in config dict.
import logging.config
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'fmt1': {
'format': '[FMT1] %(asctime)-15s %(message)s',
},
'fmt2': {
'format': '[FMT2] %(asctime)-15s %(message)s',
}
},
'handlers': {
'console1': {
'level': 'INFO',
'class': 'logging.StreamHandler',
'formatter': 'fmt1',
},
'console2': {
'level': 'INFO',
'class': 'logging.StreamHandler',
'formatter': 'fmt2',
},
},
# First config for root logger: console1 -> fmt1
'root': {
'handlers': ['console1'],
'level': 'DEBUG',
'propagate': True,
},
'loggers': {
# Second config for root logger: console2 -> fmt2
'': {
'handlers': ['console2'],
'level': 'DEBUG',
'propagate': True,
},
},
}
logging.config.dictConfig(LOGGING)
l1 = logging.getLogger()
l2 = logging.getLogger('')
root = logging.root
l1.info("l1")
l2.info("l2")
root.info("root logger")
Prints the following result:
[FMT1] 2018-12-18 17:24:47,691 l1
[FMT1] 2018-12-18 17:24:47,691 l2
[FMT1] 2018-12-18 17:24:47,691 root logger
indicating that configuration under root key has the highest priority. If the block is removed, the result is:
[FMT2] 2018-12-18 17:25:43,757 l1
[FMT2] 2018-12-18 17:25:43,757 l2
[FMT2] 2018-12-18 17:25:43,757 root logger
In both case, I was able to debug and determine that all three loggers (l1, l2 and root) referenced the same logger instance, the root logger.
Hope that will help others who, like me, were confused by the 2 different ways to configure the root logger.