Caching at QuerySet level in Django

Question

I’m trying to get a queryset from the cache, but am unsure if this even has a point.

I have the following method (simplified) inside a custom queryset:

    def queryset_from_cache(self, key: str=None, timeout: int=60):
        # Generate a key based on the query.
        if key is None:
            key = self.__generate_key # ()

        # If the cache has the key, return the cached object.
        cached_object = cache.get(key, None)

        # If the cache doesn't have the key, set the cache, 
        # and then return self (from DB) as cached_object
        if cached_object is None:
            cached_object = self
            cache.set(key, cached_object , timeout=timeout)

        return cached_object

The usage is basically to append it to a django QuerySet method, for example:

queryset = MyModel.objects.filter(id__range=[0,99]).queryset_from_cache()

My question:

Would usage like this work?

Or would it call MyModel.objects.filter(id__range=[0,99]) from the database no matter what?

Since normally caching would be done like this:

cached_object = cache.get(key, None)

if cached_object is None:
    cached_object = MyModel.objects.filter(id__range=[0,99])
    #Only now call the query
    cache.set(key, cached_object , timeout=timeout)

And thus the queryset filter() method only gets called when the key is not present in the cache, as opposed to always calling it, and then trying to get it from the cache with the queryset_from_cache method.

Asked By: nigel239

||

Source

Answer 1

This is a really cool idea, but ~~I’m not sure if you can Cache full-on Objects.. I think it’s only attributes~~

Now this having a point. Grom what I’m seeing from the limited code I’ve seen idk if it does have a point, unless filtering for Jane and John (and only them) is very common. Very narrow.
Maybe just try caching ALL the users or just individual Users, and only the attributes you need

Update

Yes! you are completetly correct, you can cache full on objects- how cool!

I don’t think your example method of queryset = MyModel.objects.filter(id__range=[0,99]).queryset_from_cache() would work.

but you can do something similar by using Model Managers and do something like: queryset = MyModel.objects.queryset_from_cache(filterdict)

Models

Natually you can return just the qs, this is just for the example to show it actually is from the cache

from django.db import models

class MyModelManager(models.Manager):
  def queryset_from_cache(self, filterdict):
    from django.core.cache import cache

    cachekey = 'MyModelCache'
    qs = cache.get(cachekey)

    if qs:
      d = {
        'in_cache': True,
        'qs': qs
        }
    else:
      qs = MyModel.objects.filter(**filterdict)
      cache.set(cachekey, qs, 300) # 5 min cache
      d = {
        'in_cache': False,
        'qs': qs
        }
    return d


class MyModel(models.Model):
  name    =   models.CharField(max_length=200)
  #
  # other attributes
  #

  objects = MyModelManager()

Example Use

from app.models import MyModel
filterdict = {'pk__range':[0,99]}
r = MyModel.objects.queryset_from_cache(filterdict)
print(r['qs'])

While it’s not exactly what you wanted, it might be close enough

Answered By: Nealium