How to query as GROUP BY in django?


I query a model:


And it returns:

Eric, Salesman, X-Shop
Freddie, Manager, X2-Shop
Teddy, Salesman, X2-Shop
Sean, Manager, X2-Shop

What I want is to know the best Django way to fire
a group_by query to my database, like:


Which doesn’t work, of course.
I know we can do some tricks on django/db/models/, but I am just curious to know how to do it without patching.

Asked By: simplyharsh



You need to do custom SQL as exemplified in this snippet:

Custom SQL via subquery

Or in a custom manager as shown in the online Django docs:

Adding extra Manager methods

Answered By: Van Gale

An easy solution, but not the proper way is to use raw SQL:

results = Members.objects.raw('SELECT * FROM myapp_members GROUP BY designation')

Another solution is to use the group_by property:

query = Members.objects.all().query
query.group_by = ['designation']
results = QuerySet(query=query, model=Members)

You can now iterate over the results variable to retrieve your results. Note that group_by is not documented and may be changed in future version of Django.

And… why do you want to use group_by? If you don’t use aggregation, you can use order_by to achieve an alike result.

Answered By: Michael

If you mean to do aggregation you can use the aggregation features of the ORM:

from django.db.models import Count
result = (Members.objects

This results in a query similar to

SELECT designation, COUNT(designation) AS dcount
FROM members GROUP BY designation

and the output would be of the form

[{'designation': 'Salesman', 'dcount': 2}, 
 {'designation': 'Manager', 'dcount': 2}]

If you don’t include the order_by(), you may get incorrect results if the default sorting is not what you expect.

If you want to include multiple fields in the results, just add them as arguments to values, for example:

    .values('designation', 'first_name', 'last_name')


Answered By: Guðmundur H

Django does not support free group by queries. I learned it in the very bad way. ORM is not designed to support stuff like what you want to do, without using custom SQL. You are limited to:

  • RAW sql (i.e. MyModel.objects.raw())
  • cr.execute sentences (and a hand-made parsing of the result).
  • .annotate() (the group by sentences are performed in the child model for .annotate(), in examples like aggregating lines_count=Count(‘lines’))).

Over a queryset qs you can call qs.query.group_by = ['field1', 'field2', ...] but it is risky if you don’t know what query are you editing and have no guarantee that it will work and not break internals of the QuerySet object. Besides, it is an internal (undocumented) API you should not access directly without risking the code not being anymore compatible with future Django versions.

Answered By: Luis Masuelli

The following module allows you to group Django models and still work with a QuerySet in the result:

For example:

from django_group_by import GroupByMixin

class BookQuerySet(QuerySet, GroupByMixin):

class Book(Model):
    title = TextField(...)
    author = ForeignKey(User, ...)
    shop = ForeignKey(Shop, ...)
    price = DecimalField(...)

class GroupedBookListView(PaginationMixin, ListView):
    template_name = 'book/books.html'
    model = Book
    paginate_by = 100

    def get_queryset(self):
        return Book.objects.group_by('title', 'author').annotate(
            shop_count=Count('shop'), price_avg=Avg('price')).order_by(
            'name', 'author').distinct()

    def get_context_data(self, **kwargs):
        return super().get_context_data(total_count=self.get_queryset().count(), **kwargs)


{% for book in object_list %}
        <h2>{{ book.title }}</td>
        <p>{{ }}, {{ }}</p>
        <p>{{ book.shop_count }}</p>
        <p>{{ book.price_avg }}</p>
{% endfor %}

The difference to the annotate/aggregate basic Django queries is the use of the attributes of a related field, e.g.

If you need the PKs of the instances that have been grouped together, add the following annotation:


NOTE: ArrayAgg is a Postgres specific function, available from Django 1.9 onwards:

Answered By: Risadinha

The documentation says that you can use values to group the queryset .

class Travel(models.Model):
    interest = models.ForeignKey(Interest)
    user = models.ForeignKey(User)
    time = models.DateTimeField(auto_now_add=True)

# Find the travel and group by the interest:

>>> Travel.objects.values('interest').annotate(Count('user'))
<QuerySet [{'interest': 5, 'user__count': 2}, {'interest': 6, 'user__count': 1}]>
# the interest(id=5) had been visited for 2 times, 
# and the interest(id=6) had only been visited for 1 time.

>>> Travel.objects.values('interest').annotate(Count('user', distinct=True)) 
<QuerySet [{'interest': 5, 'user__count': 1}, {'interest': 6, 'user__count': 1}]>
# the interest(id=5) had been visited by only one person (but this person had 
#  visited the interest for 2 times

You can find all the books and group them by name using this code:

Book.objects.values('name').annotate(Count('id')).order_by() # ensure you add the order_by()

You can watch some cheat sheet here.

Answered By: ramwin

You can also use the regroup template tag to group by attributes. From the docs:

cities = [
    {'name': 'Mumbai', 'population': '19,000,000', 'country': 'India'},
    {'name': 'Calcutta', 'population': '15,000,000', 'country': 'India'},
    {'name': 'New York', 'population': '20,000,000', 'country': 'USA'},
    {'name': 'Chicago', 'population': '7,000,000', 'country': 'USA'},
    {'name': 'Tokyo', 'population': '33,000,000', 'country': 'Japan'},


{% regroup cities by country as countries_list %}

    {% for country in countries_list %}
        <li>{{ country.grouper }}
            {% for city in country.list %}
                <li>{{ }}: {{ city.population }}</li>
            {% endfor %}
    {% endfor %}

Looks like this:

  • India
    • Mumbai: 19,000,000
    • Calcutta: 15,000,000
  • USA
    • New York: 20,000,000
    • Chicago: 7,000,000
  • Japan
    • Tokyo: 33,000,000

It also works on QuerySets I believe.


edit: note the regroup tag does not work as you would expect it to if your list of dictionaries is not key-sorted. It works iteratively. So sort your list (or query set) by the key of the grouper before passing it to the regroup tag.

Answered By: inostia
from django.db.models import Sum

first you need to import Sum
then ..

If, in other words, you need to just "remove duplicates" based on some field, and otherwise just to query the ORM objects as they are, I came up with the following workaround:

from django.db.models import OuterRef, Exists

qs = Members.objects.all()
qs = qs.annotate(is_duplicate=Exists(
qs = qs.filter(is_duplicate=False)

So, basically we’re just annotating the is_duplicate value by using some convenient filtering (which might vary based on your model and requirements), and then simply using that field to filter out the duplicates.

Answered By: Raekkeri

You could also use pythons built-in itertools.groupby directly:

from itertools import groupby

designation_key_func = lambda member: member.designation
queryset = Members.objects.all().select_related("designation")

for designation, member_group in groupby(queryset, designation_key_func):
    print(f"{designation} : {list(member_group)}")

No raw sql, subqueries, third-party-libs or templatetags needed and pythonic and explicit in my eyes.

Answered By: ralfzen

If you want the model objects, and not just plain values or dictionaries, you can do something like this:

members = Member.objects.filter(foobar=True)
designations = Designation.objects.filter(member__in=members).order_by('pk').distinct()

Replace member__in with the lowercase version of your model name, followed by __in. For example, if your model name is Car, use car__in.

Answered By: Flimm

This is little complex, but get questioner what he/she expected with only one DB hit.

from django.db.models import Subquery, OuterRef

member_qs = Members.objects.filter(
    pk__in = Members.objects.values('designation').distinct().annotate(
        pk = Subquery(
            designation= OuterRef("designation")
        .order_by("pk") # you can set other column, e.g. -pk, create_date...
   .values_list("pk", flat=True)
Answered By: rumbarum

For some reason, the above mentioned solutions did not work for me. This is what worked:

dupes_query = MyModel.objects.all().values('my_field').annotate(

I hope it helps.

Answered By: Özer
Categories: questions Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.