Django: display time it took to load a page on every page
Question:
In Django, how can I return the time it took to load a page (not the date) in every page of the site, without having to write in every views.py a code similar to the following one?
start = time.time()
#model operations
loadingpagetime = time.time() - start
If using a TEMPLATE_CONTEXT_PROCESSOR is the best option.
How would I get the whole page loading time from there, instead of just getting the template loading time?
UPDATE:
As the initial question doesn’t seem to be clear enough, here is an approach of what would be the Python version of what I want to do.
#!/usr/bin/env python
import cgitb; cgitb.enable()
import time
print 'Content-type: text/htmlnn'
start = time.time()
print '<html>'
print '<head>'
print '</head>'
print '<body>'
print '<div>HEADER</div>'
print '<div>'
print '<p>Welcome to my Django Webpage!</p>'
print '<p>Welcome to my Django Webpage!</p>'
print '<p>Welcome to my Django Webpage!</p>'
print '</div>'
time.sleep(3)
loadingtime = time.time() - start
print '<div>It took ',loadingtime,' seconds to load the page</div>'
print '</body>'
print '</html>'
Answers:
You can create a custom middleware to log this. Here is how I create a middleware to achieve this purpose base on http://djangosnippets.org/snippets/358/ (I modified the code a bit).
Firstly, assuming your project has a name: test_project, create a file name middlewares.py, I place it in the same folder as settings.py:
from django.db import connection
from time import time
from operator import add
import re
class StatsMiddleware(object):
def process_view(self, request, view_func, view_args, view_kwargs):
'''
In your base template, put this:
<div id="stats">
<!-- STATS: Total: %(total_time).2fs Python: %(python_time).2fs DB: %(db_time).2fs Queries: %(db_queries)d ENDSTATS -->
</div>
'''
# Uncomment the following if you want to get stats on DEBUG=True only
#if not settings.DEBUG:
# return None
# get number of db queries before we do anything
n = len(connection.queries)
# time the view
start = time()
response = view_func(request, *view_args, **view_kwargs)
total_time = time() - start
# compute the db time for the queries just run
db_queries = len(connection.queries) - n
if db_queries:
db_time = reduce(add, [float(q['time'])
for q in connection.queries[n:]])
else:
db_time = 0.0
# and backout python time
python_time = total_time - db_time
stats = {
'total_time': total_time,
'python_time': python_time,
'db_time': db_time,
'db_queries': db_queries,
}
# replace the comment if found
if response and response.content:
s = response.content
regexp = re.compile(r'(?P<cmt><!--s*STATS:(?P<fmt>.*?)ENDSTATSs*-->)')
match = regexp.search(s)
if match:
s = (s[:match.start('cmt')] +
match.group('fmt') % stats +
s[match.end('cmt'):])
response.content = s
return response
Secondly, modify settings.py to add your middleware:
MIDDLEWARE_CLASSES = (
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
# ... your existing middlewares ...
# your custom middleware here
'test_project.middlewares.StatsMiddleware',
)
Note: you have to add the full path to your middleware class like above, the format is:
<project_name>.<middleware_file_name>.<middleware_class_name>
A second note is I added this middleware to the end of the list because I just want to log the template load time alone. If you want to log the load time of templates + all middlewares, please put it in the beginning of MIDDLEWARE_CLASSES list (credits to @Symmitchry).
Back to the main topic, the next step is to modify your base.html or whatever pages you want to log load time, add this:
<div id="stats">
<!-- STATS: Total: %(total_time).2fs Python: %(python_time).2fs DB: %(db_time).2fs Queries: %(db_queries)d ENDSTATS -->
</div>
Note: you can name the <div id="stats"> and use CSS for that div however you want, but DON’T change the comment <!-- STATS: .... -->. If you want to change it, be sure that you test it against the regex pattern in the created middlewares.py.
Voila, enjoy the statistics.
EDIT:
For those who use CBVs (Class Based Views) a lot, you might have encountered the error ContentNotRenderedError with above solution. Have no fear, here is the fix in middlewares.py:
# replace the comment if found
if response:
try:
# detects TemplateResponse which are not yet rendered
if response.is_rendered:
rendered_content = response.content
else:
rendered_content = response.rendered_content
except AttributeError: # django < 1.5
rendered_content = response.content
if rendered_content:
s = rendered_content
regexp = re.compile(
r'(?P<cmt><!--s*STATS:(?P<fmt>.*?)ENDSTATSs*-->)'
)
match = regexp.search(s)
if match:
s = (s[:match.start('cmt')] +
match.group('fmt') % stats +
s[match.end('cmt'):])
response.content = s
return response
I got it working with Django 1.6.x, if you have problem with other version of Django, please ping me in comment section.
If you want to show page loading info to your visitors, the answer from Hieu Nguyen will be a great option. But let me also recommend you some good tools.
So if you need this data just for development purposes, take a look at django-debug-toolbar. It will attach a nice toolbar with useful statistics for every page, including time it took for DB queries, template rendering and so on.
Finally, if you need professional profiling tools for Djagno, New Relic might be a very good choice.
Geordi gives you an awesome breakdown of everything that happens in the request cycle. It’s a middleware that generates a full call-tree to show you exactly what’s going on and how long is spent in each function.
It looks like this:
I highly recommend it :)
Image credit: http://evzijst.bitbucket.org/pycon.in
I can be wrong but the first thing I remembered was document.ready and ajax…
I don’t think it is the best solution, but it should be quite simple to implement not only in django but anywhere else.
When you process the request in a view you start timer, and when document.ready is triggered you make an ajax call to stop the timer.
But I should admit that @Hieu Nguyen is quite brilliant)
I’ve upgraded middleware.py. In few words, we add some unique text in base template, then replace it in Middleware process_response. This method looks little tricky, but we have generation time closer to true, and we have not problem with javascript. Javascript on client cannot use page headers on firs loaded page, only if create XHR object, then get page.
In Django, how can I return the time it took to load a page (not the date) in every page of the site, without having to write in every views.py a code similar to the following one?
start = time.time()
#model operations
loadingpagetime = time.time() - start
If using a TEMPLATE_CONTEXT_PROCESSOR is the best option.
How would I get the whole page loading time from there, instead of just getting the template loading time?
UPDATE:
As the initial question doesn’t seem to be clear enough, here is an approach of what would be the Python version of what I want to do.
#!/usr/bin/env python
import cgitb; cgitb.enable()
import time
print 'Content-type: text/htmlnn'
start = time.time()
print '<html>'
print '<head>'
print '</head>'
print '<body>'
print '<div>HEADER</div>'
print '<div>'
print '<p>Welcome to my Django Webpage!</p>'
print '<p>Welcome to my Django Webpage!</p>'
print '<p>Welcome to my Django Webpage!</p>'
print '</div>'
time.sleep(3)
loadingtime = time.time() - start
print '<div>It took ',loadingtime,' seconds to load the page</div>'
print '</body>'
print '</html>'
You can create a custom middleware to log this. Here is how I create a middleware to achieve this purpose base on http://djangosnippets.org/snippets/358/ (I modified the code a bit).
Firstly, assuming your project has a name: test_project, create a file name middlewares.py, I place it in the same folder as settings.py:
from django.db import connection
from time import time
from operator import add
import re
class StatsMiddleware(object):
def process_view(self, request, view_func, view_args, view_kwargs):
'''
In your base template, put this:
<div id="stats">
<!-- STATS: Total: %(total_time).2fs Python: %(python_time).2fs DB: %(db_time).2fs Queries: %(db_queries)d ENDSTATS -->
</div>
'''
# Uncomment the following if you want to get stats on DEBUG=True only
#if not settings.DEBUG:
# return None
# get number of db queries before we do anything
n = len(connection.queries)
# time the view
start = time()
response = view_func(request, *view_args, **view_kwargs)
total_time = time() - start
# compute the db time for the queries just run
db_queries = len(connection.queries) - n
if db_queries:
db_time = reduce(add, [float(q['time'])
for q in connection.queries[n:]])
else:
db_time = 0.0
# and backout python time
python_time = total_time - db_time
stats = {
'total_time': total_time,
'python_time': python_time,
'db_time': db_time,
'db_queries': db_queries,
}
# replace the comment if found
if response and response.content:
s = response.content
regexp = re.compile(r'(?P<cmt><!--s*STATS:(?P<fmt>.*?)ENDSTATSs*-->)')
match = regexp.search(s)
if match:
s = (s[:match.start('cmt')] +
match.group('fmt') % stats +
s[match.end('cmt'):])
response.content = s
return response
Secondly, modify settings.py to add your middleware:
MIDDLEWARE_CLASSES = (
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
# ... your existing middlewares ...
# your custom middleware here
'test_project.middlewares.StatsMiddleware',
)
Note: you have to add the full path to your middleware class like above, the format is:
<project_name>.<middleware_file_name>.<middleware_class_name>
A second note is I added this middleware to the end of the list because I just want to log the template load time alone. If you want to log the load time of templates + all middlewares, please put it in the beginning of MIDDLEWARE_CLASSES list (credits to @Symmitchry).
Back to the main topic, the next step is to modify your base.html or whatever pages you want to log load time, add this:
<div id="stats">
<!-- STATS: Total: %(total_time).2fs Python: %(python_time).2fs DB: %(db_time).2fs Queries: %(db_queries)d ENDSTATS -->
</div>
Note: you can name the <div id="stats"> and use CSS for that div however you want, but DON’T change the comment <!-- STATS: .... -->. If you want to change it, be sure that you test it against the regex pattern in the created middlewares.py.
Voila, enjoy the statistics.
EDIT:
For those who use CBVs (Class Based Views) a lot, you might have encountered the error ContentNotRenderedError with above solution. Have no fear, here is the fix in middlewares.py:
# replace the comment if found
if response:
try:
# detects TemplateResponse which are not yet rendered
if response.is_rendered:
rendered_content = response.content
else:
rendered_content = response.rendered_content
except AttributeError: # django < 1.5
rendered_content = response.content
if rendered_content:
s = rendered_content
regexp = re.compile(
r'(?P<cmt><!--s*STATS:(?P<fmt>.*?)ENDSTATSs*-->)'
)
match = regexp.search(s)
if match:
s = (s[:match.start('cmt')] +
match.group('fmt') % stats +
s[match.end('cmt'):])
response.content = s
return response
I got it working with Django 1.6.x, if you have problem with other version of Django, please ping me in comment section.
If you want to show page loading info to your visitors, the answer from Hieu Nguyen will be a great option. But let me also recommend you some good tools.
So if you need this data just for development purposes, take a look at django-debug-toolbar. It will attach a nice toolbar with useful statistics for every page, including time it took for DB queries, template rendering and so on.
Finally, if you need professional profiling tools for Djagno, New Relic might be a very good choice.
Geordi gives you an awesome breakdown of everything that happens in the request cycle. It’s a middleware that generates a full call-tree to show you exactly what’s going on and how long is spent in each function.
It looks like this:
I highly recommend it :)
Image credit: http://evzijst.bitbucket.org/pycon.in
I can be wrong but the first thing I remembered was document.ready and ajax…
I don’t think it is the best solution, but it should be quite simple to implement not only in django but anywhere else.
When you process the request in a view you start timer, and when document.ready is triggered you make an ajax call to stop the timer.
But I should admit that @Hieu Nguyen is quite brilliant)
I’ve upgraded middleware.py. In few words, we add some unique text in base template, then replace it in Middleware process_response. This method looks little tricky, but we have generation time closer to true, and we have not problem with javascript. Javascript on client cannot use page headers on firs loaded page, only if create XHR object, then get page.