How do you log server errors on django sites
Question:
So, when playing with the development I can just set settings.DEBUG to True and if an error occures I can see it nicely formatted, with good stack trace and request information.
But on kind of production site I’d rather use DEBUG=False and show visitors some standard error 500 page with information that I’m working on fixing this bug at this moment 😉
At the same time I’d like to have some way of logging all those information (stack trace and request info) to a file on my server – so I can just output it to my console and watch errors scroll, email the log to me every hour or something like this.
What logging solutions would you recomend for a django-site, that would meet those simple requirements? I have the application running as fcgi server and I’m using apache web server as frontend (although thinking of going to lighttpd).
Answers:
Well, when DEBUG = False, Django will automatically mail a full traceback of any error to each person listed in the ADMINS setting, which gets you notifications pretty much for free. If you’d like more fine-grained control, you can write and add to your settings a middleware class which defines a method named process_exception(), which will have access to the exception that was raised:
http://docs.djangoproject.com/en/dev/topics/http/middleware/#process-exception
Your process_exception() method can then perform whatever type of logging you’d like: writing to console, writing to a file, etc., etc.
Edit: though it’s a bit less useful, you can also listen for the got_request_exception signal, which will be sent whenever an exception is encountered during request processing:
http://docs.djangoproject.com/en/dev/ref/signals/#got-request-exception
This does not give you access to the exception object, however, so the middleware method is much easier to work with.
Obviously James is correct, but if you wanted to log exceptions in a datastore, there are a few open source solutions already available:
1) CrashLog is a good choice: http://code.google.com/p/django-crashlog/
2) Db-Log is a good choice as well: http://code.google.com/p/django-db-log/
What is the difference between the two? Almost nothing that I can see, so either one will suffice.
I’ve used both and they work well.
django-db-log, mentioned in another answer, has been replaced with:
Django Sentry is a good way to go, as already mentioned, but there is a bit of work involved in setting it up properly (as a separate website). If you just want to log everything to a simple text file here’s the logging configuration to put in your settings.py
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
# Include the default Django email handler for errors
# This is what you'd get without configuring logging at all.
'mail_admins': {
'class': 'django.utils.log.AdminEmailHandler',
'level': 'ERROR',
# But the emails are plain text by default - HTML is nicer
'include_html': True,
},
# Log to a text file that can be rotated by logrotate
'logfile': {
'class': 'logging.handlers.WatchedFileHandler',
'filename': '/var/log/django/myapp.log'
},
},
'loggers': {
# Again, default Django configuration to email unhandled exceptions
'django.request': {
'handlers': ['mail_admins'],
'level': 'ERROR',
'propagate': True,
},
# Might as well log any errors anywhere else in Django
'django': {
'handlers': ['logfile'],
'level': 'ERROR',
'propagate': False,
},
# Your own app - this assumes all your logger names start with "myapp."
'myapp': {
'handlers': ['logfile'],
'level': 'WARNING', # Or maybe INFO or DEBUG
'propagate': False
},
},
}
Some time has passed since EMP’s most helpful code submission. I just now implemented it, and while thrashing around with some manage.py option, to try to chase down a bug, I got a deprecation warning to the effect that with my current version of Django (1.5.?) a require_debug_false filter is now needed for the mail_admins handler.
Here is the revised code:
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'filters': {
'require_debug_false': {
'()': 'django.utils.log.RequireDebugFalse'
}
},
'handlers': {
# Include the default Django email handler for errors
# This is what you'd get without configuring logging at all.
'mail_admins': {
'class': 'django.utils.log.AdminEmailHandler',
'level': 'ERROR',
'filters': ['require_debug_false'],
# But the emails are plain text by default - HTML is nicer
'include_html': True,
},
# Log to a text file that can be rotated by logrotate
'logfile': {
'class': 'logging.handlers.WatchedFileHandler',
'filename': '/home/username/public_html/djangoprojectname/logfilename.log'
},
},
'loggers': {
# Again, default Django configuration to email unhandled exceptions
'django.request': {
'handlers': ['mail_admins'],
'level': 'ERROR',
'propagate': True,
},
# Might as well log any errors anywhere else in Django
'django': {
'handlers': ['logfile'],
'level': 'ERROR',
'propagate': False,
},
# Your own app - this assumes all your logger names start with "myapp."
'myapp': {
'handlers': ['logfile'],
'level': 'DEBUG', # Or maybe INFO or WARNING
'propagate': False
},
},
}
I just had an annoying problem with my fcgi script. It occurred before django even started. The lack of logging is sooo painful. Anyway, redirecting stderr to a file as the very first thing helped a lot:
#!/home/user/env/bin/python
sys.stderr = open('/home/user/fcgi_errors', 'a')
You can use the logging library in Python, no need to pip install anything.
Replace any print() with logging.debug()
but,
Django Sentry is a good way to go
as EMP said.
So, when playing with the development I can just set settings.DEBUG to True and if an error occures I can see it nicely formatted, with good stack trace and request information.
But on kind of production site I’d rather use DEBUG=False and show visitors some standard error 500 page with information that I’m working on fixing this bug at this moment 😉
At the same time I’d like to have some way of logging all those information (stack trace and request info) to a file on my server – so I can just output it to my console and watch errors scroll, email the log to me every hour or something like this.
What logging solutions would you recomend for a django-site, that would meet those simple requirements? I have the application running as fcgi server and I’m using apache web server as frontend (although thinking of going to lighttpd).
Well, when DEBUG = False, Django will automatically mail a full traceback of any error to each person listed in the ADMINS setting, which gets you notifications pretty much for free. If you’d like more fine-grained control, you can write and add to your settings a middleware class which defines a method named process_exception(), which will have access to the exception that was raised:
http://docs.djangoproject.com/en/dev/topics/http/middleware/#process-exception
Your process_exception() method can then perform whatever type of logging you’d like: writing to console, writing to a file, etc., etc.
Edit: though it’s a bit less useful, you can also listen for the got_request_exception signal, which will be sent whenever an exception is encountered during request processing:
http://docs.djangoproject.com/en/dev/ref/signals/#got-request-exception
This does not give you access to the exception object, however, so the middleware method is much easier to work with.
Obviously James is correct, but if you wanted to log exceptions in a datastore, there are a few open source solutions already available:
1) CrashLog is a good choice: http://code.google.com/p/django-crashlog/
2) Db-Log is a good choice as well: http://code.google.com/p/django-db-log/
What is the difference between the two? Almost nothing that I can see, so either one will suffice.
I’ve used both and they work well.
django-db-log, mentioned in another answer, has been replaced with:
Django Sentry is a good way to go, as already mentioned, but there is a bit of work involved in setting it up properly (as a separate website). If you just want to log everything to a simple text file here’s the logging configuration to put in your settings.py
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
# Include the default Django email handler for errors
# This is what you'd get without configuring logging at all.
'mail_admins': {
'class': 'django.utils.log.AdminEmailHandler',
'level': 'ERROR',
# But the emails are plain text by default - HTML is nicer
'include_html': True,
},
# Log to a text file that can be rotated by logrotate
'logfile': {
'class': 'logging.handlers.WatchedFileHandler',
'filename': '/var/log/django/myapp.log'
},
},
'loggers': {
# Again, default Django configuration to email unhandled exceptions
'django.request': {
'handlers': ['mail_admins'],
'level': 'ERROR',
'propagate': True,
},
# Might as well log any errors anywhere else in Django
'django': {
'handlers': ['logfile'],
'level': 'ERROR',
'propagate': False,
},
# Your own app - this assumes all your logger names start with "myapp."
'myapp': {
'handlers': ['logfile'],
'level': 'WARNING', # Or maybe INFO or DEBUG
'propagate': False
},
},
}
Some time has passed since EMP’s most helpful code submission. I just now implemented it, and while thrashing around with some manage.py option, to try to chase down a bug, I got a deprecation warning to the effect that with my current version of Django (1.5.?) a require_debug_false filter is now needed for the mail_admins handler.
Here is the revised code:
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'filters': {
'require_debug_false': {
'()': 'django.utils.log.RequireDebugFalse'
}
},
'handlers': {
# Include the default Django email handler for errors
# This is what you'd get without configuring logging at all.
'mail_admins': {
'class': 'django.utils.log.AdminEmailHandler',
'level': 'ERROR',
'filters': ['require_debug_false'],
# But the emails are plain text by default - HTML is nicer
'include_html': True,
},
# Log to a text file that can be rotated by logrotate
'logfile': {
'class': 'logging.handlers.WatchedFileHandler',
'filename': '/home/username/public_html/djangoprojectname/logfilename.log'
},
},
'loggers': {
# Again, default Django configuration to email unhandled exceptions
'django.request': {
'handlers': ['mail_admins'],
'level': 'ERROR',
'propagate': True,
},
# Might as well log any errors anywhere else in Django
'django': {
'handlers': ['logfile'],
'level': 'ERROR',
'propagate': False,
},
# Your own app - this assumes all your logger names start with "myapp."
'myapp': {
'handlers': ['logfile'],
'level': 'DEBUG', # Or maybe INFO or WARNING
'propagate': False
},
},
}
I just had an annoying problem with my fcgi script. It occurred before django even started. The lack of logging is sooo painful. Anyway, redirecting stderr to a file as the very first thing helped a lot:
#!/home/user/env/bin/python
sys.stderr = open('/home/user/fcgi_errors', 'a')
You can use the logging library in Python, no need to pip install anything.
Replace any print() with logging.debug()
but,
Django Sentry is a good way to go
as EMP said.