Atomic increment of a counter in django

Question

I’m trying to atomically increment a simple counter in Django. My code looks like this:

from models import Counter
from django.db import transaction

@transaction.commit_on_success
def increment_counter(name):
    counter = Counter.objects.get_or_create(name = name)[0]
    counter.count += 1
    counter.save()

If I understand Django correctly, this should wrap the function in a transaction and make the increment atomic. But it doesn’t work and there is a race condition in the counter update. How can this code be made thread-safe?

Asked By: Björn Lindqvist

||

Source

Answer 1

Use an F expression:

from django.db.models import F

either in update():

Counter.objects.get_or_create(name=name)
Counter.objects.filter(name=name).update(count=F("count") + 1)

or on the object instance:

counter, _ = Counter.objects.get_or_create(name=name)
counter.count = F("count") + 1
counter.save(update_fields=["count"])

Remember to specify update_fields, or you might encounter race conditions on other fields of the model.

A note on the race condition avoided by using F expressions has been added to the official documentation.

Answered By: Oduvan

Answer 2

Or if you just want a counter and not a persistent object you can use itertools counter which is implemented in C. The GIL will provide the safety needed.

–Sai

Answered By: Sai Venkat

Answer 3

In Django 1.4 there is support for SELECT … FOR UPDATE clauses, using database locks to make sure no data is accesses concurrently by mistake.

Answered By: Emil Stenström

Answer 4

Keeping it simple and building on @Oduvan’s answer:

counter, created = Counter.objects.get_or_create(name = name, 
                                                 defaults={'count':1})
if not created:
    counter.count = F('count') +1
    counter.save()

The advantage here is that if the object was created in the first statement, you don’t have to do any further updates.

Answered By: mlissner

Answer 5

Django 1.7

from django.db.models import F

counter, created = Counter.objects.get_or_create(name = name)
counter.count = F('count') +1
counter.save()

Answered By: derevo

Answer 6

If you don’t need to know the value of the counter when you set it, the top answer is definitely your best bet:

counter, _ = Counter.objects.get_or_create(name = name)
counter.count = F('count') + 1
counter.save()

This tells your database to add 1 to the value of count, which it can do perfectly well without blocking other operations. The drawback is that you have no way of knowing what count you just set. If two threads simultaneously hit this function, they would both see the same value, and would both tell the db to add 1. The db would end up adding 2 as expected, but you won’t know which one went first.

If you do care about the count right now, you can use the select_for_update option referenced by Emil Stenstrom. Here’s what that looks like:

from models import Counter
from django.db import transaction

@transaction.atomic
def increment_counter(name):
    counter = (Counter.objects
               .select_for_update()
               .get_or_create(name=name)[0]
    counter.count += 1
    counter.save()

This reads the current value and locks matching rows until the end of the transaction. Now only one worker can read at a time. See the docs for more on select_for_update.

Answered By: Xephryous

Atomic increment of a counter in django

Question:

Answers: