How to clear the whole cache when using django's page_cache decorator
Question:
I’ve got a pretty simple site where I’m using the page_cache decorator.
I have a cronjob that checks for new data and processes it if it’s available.
(This is run using management commands executed with crontab)
I want to then clear all the page caches when new data is processed.
I’m looking at the documentation here:
https://docs.djangoproject.com/en/stable/topics/cache/
and found cache.clear(), which seems to be what I want.
I’ve added a flag to the data processing portion and execute cache.clear() when new data is found.
However, after the command runs the cache isn’t cleared.
(I’ve cleared the browser cached and checked to be sure it’s not the browser)
Does cache.clear() not work in to clear all cached pages?
I’m using the DatabaseCache, so I guess I could go in and clear the cache table manually, but is there a better way?
Answers:
Put something into cache, then try to call cache.clear() from the manage.py shell console and then manually check database cache contents. If that works then maybe your cache.clear() is just not called when new data is found.
The easiest way to understand what is going under the hood is just to put import pdb; pdb.set_trace() to the beginning of the cache.clear() function, then run debug server and wait, then some code call this func you’ll be able to execute step-by-step its code or you’ll just see that this func is not called as your expected.
I’ve had this problem with an SQLite database cache – the clear() method doesn’t clear the cache although it works fine with a MySQL database cache. It seems that a SQLite cache needs a call to django.db.transation.commit_unless_managed() after the DELETE from [table] statement is run.
I have been using multiple caches since before official support was added into core as part of 1.3 and so have a wrapper round several of the cache calls – including clear() – so I was able to override this method and include the commit_unless_managed(). I think I should probably log it as a bug.
Here’s the outline of the code I’m using to flush a memcache cache (the default cache in django.core.cache) and a database cache stored in the cache_table of the settings.DATABASES['cache_database'] database.
from django.db import connections, transaction
from django.core.cache import cache # This is the memcache cache.
def flush():
# This works as advertised on the memcached cache:
cache.clear()
# This manually purges the SQLite cache:
cursor = connections['cache_database'].cursor()
cursor.execute('DELETE FROM cache_table')
transaction.commit_unless_managed(using='cache_database')
Rather than being lazy and hard coding it the way I have it should be pretty easy to get the values from settings.CACHES and django.db.router.
It’s a bug #19896 that looks to be fixed in 1.6.
If you are using an older version doing something like the following should make the clear work as expected.
from django.db import router, transaction
def clear_cache(the_cache):
the_cache.clear()
# commit the transaction
db = router.db_for_write(the_cache.cache_model_class)
transaction.commit_unless_managed(using=db)
This just makes sure that the transaction gets committed.
I’ve got a pretty simple site where I’m using the page_cache decorator.
I have a cronjob that checks for new data and processes it if it’s available.
(This is run using management commands executed with crontab)
I want to then clear all the page caches when new data is processed.
I’m looking at the documentation here:
https://docs.djangoproject.com/en/stable/topics/cache/
and found cache.clear(), which seems to be what I want.
I’ve added a flag to the data processing portion and execute cache.clear() when new data is found.
However, after the command runs the cache isn’t cleared.
(I’ve cleared the browser cached and checked to be sure it’s not the browser)
Does cache.clear() not work in to clear all cached pages?
I’m using the DatabaseCache, so I guess I could go in and clear the cache table manually, but is there a better way?
Put something into cache, then try to call cache.clear() from the manage.py shell console and then manually check database cache contents. If that works then maybe your cache.clear() is just not called when new data is found.
The easiest way to understand what is going under the hood is just to put import pdb; pdb.set_trace() to the beginning of the cache.clear() function, then run debug server and wait, then some code call this func you’ll be able to execute step-by-step its code or you’ll just see that this func is not called as your expected.
I’ve had this problem with an SQLite database cache – the clear() method doesn’t clear the cache although it works fine with a MySQL database cache. It seems that a SQLite cache needs a call to django.db.transation.commit_unless_managed() after the DELETE from [table] statement is run.
I have been using multiple caches since before official support was added into core as part of 1.3 and so have a wrapper round several of the cache calls – including clear() – so I was able to override this method and include the commit_unless_managed(). I think I should probably log it as a bug.
Here’s the outline of the code I’m using to flush a memcache cache (the default cache in django.core.cache) and a database cache stored in the cache_table of the settings.DATABASES['cache_database'] database.
from django.db import connections, transaction
from django.core.cache import cache # This is the memcache cache.
def flush():
# This works as advertised on the memcached cache:
cache.clear()
# This manually purges the SQLite cache:
cursor = connections['cache_database'].cursor()
cursor.execute('DELETE FROM cache_table')
transaction.commit_unless_managed(using='cache_database')
Rather than being lazy and hard coding it the way I have it should be pretty easy to get the values from settings.CACHES and django.db.router.
It’s a bug #19896 that looks to be fixed in 1.6.
If you are using an older version doing something like the following should make the clear work as expected.
from django.db import router, transaction
def clear_cache(the_cache):
the_cache.clear()
# commit the transaction
db = router.db_for_write(the_cache.cache_model_class)
transaction.commit_unless_managed(using=db)
This just makes sure that the transaction gets committed.