How to implement sorting in Django Admin for calculated model properties without writing the logic twice?
Question:
In my Django model, I defined a @property which worked nicely and the property can be shown in the admin list_display without any problems.
I need this property not only in admin but in my code logic in other places as well, so it makes sense to have it as property for my model.
Now I wanted to make the column of this property sortable, and with help of the Django documentation of the When object, this StackOverflow question for the F()-calculation and this link for the sorting I managed to build the working solution shown below.
The reason for posing a question here is: In fact, I implemented my logic twice, once in python and once in form of an expression, which is against the design rule of implementing the same logic only once. So I wanted to ask whether I missed a better solution to my problem. Any ideas are appreciated.
This is the model (identifyers modified):
class De(models.Model):
fr = models.BooleanField("[...]")
de = models.SmallIntegerField("[...]")
gd = models.SmallIntegerField("[...]")
na = models.SmallIntegerField("[...]")
# [several_attributes, Meta, __str__() removed for readability]
@property
def s_d(self):
if self.fr:
return self.de
else:
return self.gd + self.na
This is the Model Admin:
class DeAdmin(admin.ModelAdmin):
list_display = ("[...]", "s_d", "gd", "na", "de", "fr" )
def get_queryset(self, request):
queryset = super().get_queryset(request)
queryset = queryset.annotate(
_s_d=Case(
When(fr=True, then='s_d'),
When(fr=False, then=F('gd') + F('na')),
default=Value(0),
output_field=IntegerField(),
)
)
return queryset
def s_d(self, obj):
return obj._s_d
s_d.admin_order_field = '_s_d'
If there is no other way, I would also appreciate confirmation of the fact as an answer.
Answers:
Although I think your solution is very good (or even better), the another approach can be to extract admin query to the model manager:
class DeManager(models.Manager):
def get_queryset(self):
return super().get_queryset().annotate(
s_d=Case(
When(fr=True, then='s_d'),
When(fr=False, then=F('gd') + F('na')),
default=Value(0),
output_field=IntegerField(),
)
)
class De(models.Model):
fr = models.BooleanField("[...]")
de = models.SmallIntegerField("[...]")
gd = models.SmallIntegerField("[...]")
na = models.SmallIntegerField("[...]")
objects = DeManager()
class DeAdmin(admin.ModelAdmin):
list_display = ("[...]", "s_d", "gd", "na", "de", "fr" )
In this case you don’t need the property because each object will have s_d attribute, although this is true only for existing objects (from the database). If you create a new object in Python and try to access obj.s_d you will get an error. Another disadvantage is that each query will be annotated with this attribute even if you don’t use it, but this can be solved by customizing the manager’s queryset.
TL/DR: Yes your solution seems to follow the only way that makes sense.
Well, what you have composed here seems to be the recommended way from the sources you list in your question and for good reason.
What is the good reason though?
I haven’t found a definitive, in the codebase, answer for that but I imagine that it has to do with the way @property decorator works in Python.
When we set a property with the decorator then we cannot add attributes to it and since the admin_order_field is an attribute then we can’t have that in there. That statement seems to be reinforced from the Django Admin’s list_display documentation where the following passage exists:
Elements of list_display can also be properties. Please note however, that due to the way properties work in Python, setting short_description on a property is only possible when using the property() function and not with the @property decorator.
That quote in combination with this QA: AttributeError: 'property' object has no attribute 'admin_order_field' seems to explain why it is not possible to have an orderable from a model property directly into the admin panel.
That explained (probably?) it is time for some mental gymnastics!!
In the previously mentioned part of the documentation we can also see that the admin_order_field can accept query expressions since version 2.1:
Query expressions may be used in admin_order_field. For example:
from django.db.models import Value
from django.db.models.functions import Concat
class Person(models.Model):
first_name = models.CharField(max_length=50)
last_name = models.CharField(max_length=50)
def full_name(self):
return self.first_name + ' ' + self.last_name
full_name.admin_order_field = Concat('first_name', Value(' '), 'last_name')
That in conjunction with the previous part about the property() method, allows us to refactor your code and essentially move the annotation part to the model:
class De(models.Model):
...
def calculate_s_d(self):
if self.fr:
return self.de
else:
return self.gd + self.na
calculate_s_d.admin_order_field = Case(
When(fr=True, then='s_d'),
When(fr=False, then=F('gd') + F('na')),
default=Value(0),
output_field=IntegerField(),
)
s_d = property(calculate_s_d)
Finally, on the admin.py we only need:
class DeAdmin(admin.ModelAdmin):
list_display = ("[...]", "s_d")
Unfortunately, this is impossible in current stable Django version (up to 2.2) due to Django admin not fetching admin_order_field from object properties.
Fortunately, it will be possible in upcoming Django version (3.0 and up) which should be released on 2nd of December.
The way to achieve it:
class De(models.Model):
fr = models.BooleanField("[...]")
de = models.SmallIntegerField("[...]")
gd = models.SmallIntegerField("[...]")
na = models.SmallIntegerField("[...]")
# [several_attributes, Meta, __str__() removed for readability]
def s_d(self):
if self.fr:
return self.de
else:
return self.gd + self.na
s_d.admin_order_field = '_s_d'
s_d = property(s_d)
Alternatively, you can create some decorator that will add any attribute to function, before converting it to property:
def decorate(**kwargs):
def wrap(function):
for name, value in kwargs.iteritems():
setattr(function, name, value)
return function
return wrap
class De(models.Model):
fr = models.BooleanField("[...]")
de = models.SmallIntegerField("[...]")
gd = models.SmallIntegerField("[...]")
na = models.SmallIntegerField("[...]")
# [several_attributes, Meta, __str__() removed for readability]
@property
@decorate(admin_order_field='_s_d')
def s_d(self):
if self.fr:
return self.de
else:
return self.gd + self.na
Another possible solution might be to convert the s_d property to a model field and override the model save method to keep it up to date.
# models.py
class De(models.Model):
fr = models.BooleanField("[...]")
de = models.SmallIntegerField("[...]")
gd = models.SmallIntegerField("[...]")
na = models.SmallIntegerField("[...]")
s_d = models.SmallIntegerField("[...]", blank=True)
# [several_attributes, Meta, __str__() removed for readability]
def save(self, *args, **kwargs):
if self.fr:
self.s_d = self.de
else:
self.s_d = self.gd + self.na
super().save(*args, **kwargs)
# admin.py
class DeAdmin(admin.ModelAdmin):
list_display = ("[...]", "s_d", "gd", "na", "de", "fr" )
The default sorting in admin.py will be applied and the value of s_d will be updated every time the model is saved.
There is a caveat to this method if you plan to do a lot of bulk operations, such as bulk_create, update, or delete.
Overridden model methods are not called on bulk operations
Note that the delete() method for an object is not necessarily called
when deleting objects in bulk using a QuerySet or as a result of a
cascading delete. To ensure customized delete logic gets executed, you
can use pre_delete and/or post_delete signals.
Unfortunately, there isn’t a workaround when creating or updating
objects in bulk, since none of save(), pre_save, and post_save are
called.
The solutions presented in answers to this question work only if the property inherits ordering from one or more db columns. In this answer I share a solution that works with any property and operates on queryset results with some limitations on multiple ordering.
I am using Django 4.1.
Given the model in the question I put a random value generation in the property just as an example of values retrieved independent from db columns so that we cannot retrieve ordering from db.
import random
class De(models.Model):
fr = models.BooleanField("[...]")
de = models.SmallIntegerField("[...]")
gd = models.SmallIntegerField("[...]")
na = models.SmallIntegerField("[...]")
# [several_attributes, Meta, __str__() removed for readability]
def s_d(self):
"""
Just as example: adds a perturbation so it is not possible
to inherit ordering by db columns
"""
X = random.randrange(128)
if self.fr:
return self.de*X
else:
return self.gd*X + self.na
s_d.admin_order_field = 'cielcio_to_be_removed'
s_d = property(s_d)
The solution I have used sets ChangeList.result_list overloading the ModelAdmin.paginator. This choice has the benefit to operate on computed results while the cons of not using the database ordering.
I use also a class DeChangeList inherited from ChangeList to remove the fake field set as value of admin_order_field so that I got the admin table header clickable.
from django.contrib import admin
from django.contrib.admin.views.main import ChangeList
class DeChangeList(ChangeList):
def get_ordering(self, request, queryset):
"""
Removes the fake field used to show upper
and lower arrow in changelist table header
"""
ordering = super().get_ordering(request, queryset)
if 'cielcio_to_be_removed' in ordering:
ordering.remove('cielcio_to_be_removed')
if '-cielcio_to_be_removed' in ordering:
ordering.remove('-cielcio_to_be_removed')
return ordering
class DeAdmin(admin.ModelAdmin):
list_display = ("[...]", "s_d", "gd", "na", "de", "fr" )
def get_changelist(self, request, **kwargs):
return DeChangeList
def get_paginator(self, request, queryset, per_page, orphans=0, allow_empty_first_page=True):
"""
Intercepts queryset and order by values with 'sorted'
"""
index_s_p = self.list_display.index('s_p')
ordering = request.GET.get('o', "99999")
instances = []
for nf in ordering.split("."):
reverse = int(nf) < 0
if abs(int(nf)) == index_s_p+1:
instances = sorted(
queryset,
key=lambda a: (a.s_p is not None if reverse else a.s_p is None, a.s_p),
reverse=reverse
)
return super().get_paginator(
request, instances or queryset,
per_page, orphans, allow_empty_first_page)
In my Django model, I defined a @property which worked nicely and the property can be shown in the admin list_display without any problems.
I need this property not only in admin but in my code logic in other places as well, so it makes sense to have it as property for my model.
Now I wanted to make the column of this property sortable, and with help of the Django documentation of the When object, this StackOverflow question for the F()-calculation and this link for the sorting I managed to build the working solution shown below.
The reason for posing a question here is: In fact, I implemented my logic twice, once in python and once in form of an expression, which is against the design rule of implementing the same logic only once. So I wanted to ask whether I missed a better solution to my problem. Any ideas are appreciated.
This is the model (identifyers modified):
class De(models.Model):
fr = models.BooleanField("[...]")
de = models.SmallIntegerField("[...]")
gd = models.SmallIntegerField("[...]")
na = models.SmallIntegerField("[...]")
# [several_attributes, Meta, __str__() removed for readability]
@property
def s_d(self):
if self.fr:
return self.de
else:
return self.gd + self.na
This is the Model Admin:
class DeAdmin(admin.ModelAdmin):
list_display = ("[...]", "s_d", "gd", "na", "de", "fr" )
def get_queryset(self, request):
queryset = super().get_queryset(request)
queryset = queryset.annotate(
_s_d=Case(
When(fr=True, then='s_d'),
When(fr=False, then=F('gd') + F('na')),
default=Value(0),
output_field=IntegerField(),
)
)
return queryset
def s_d(self, obj):
return obj._s_d
s_d.admin_order_field = '_s_d'
If there is no other way, I would also appreciate confirmation of the fact as an answer.
Although I think your solution is very good (or even better), the another approach can be to extract admin query to the model manager:
class DeManager(models.Manager):
def get_queryset(self):
return super().get_queryset().annotate(
s_d=Case(
When(fr=True, then='s_d'),
When(fr=False, then=F('gd') + F('na')),
default=Value(0),
output_field=IntegerField(),
)
)
class De(models.Model):
fr = models.BooleanField("[...]")
de = models.SmallIntegerField("[...]")
gd = models.SmallIntegerField("[...]")
na = models.SmallIntegerField("[...]")
objects = DeManager()
class DeAdmin(admin.ModelAdmin):
list_display = ("[...]", "s_d", "gd", "na", "de", "fr" )
In this case you don’t need the property because each object will have s_d attribute, although this is true only for existing objects (from the database). If you create a new object in Python and try to access obj.s_d you will get an error. Another disadvantage is that each query will be annotated with this attribute even if you don’t use it, but this can be solved by customizing the manager’s queryset.
TL/DR: Yes your solution seems to follow the only way that makes sense.
Well, what you have composed here seems to be the recommended way from the sources you list in your question and for good reason.
What is the good reason though?
I haven’t found a definitive, in the codebase, answer for that but I imagine that it has to do with the way @property decorator works in Python.
When we set a property with the decorator then we cannot add attributes to it and since the admin_order_field is an attribute then we can’t have that in there. That statement seems to be reinforced from the Django Admin’s list_display documentation where the following passage exists:
Elements of
list_displaycan also be properties. Please note however, that due to the way properties work in Python, settingshort_descriptionon a property is only possible when using theproperty()function and not with the@propertydecorator.
That quote in combination with this QA: AttributeError: 'property' object has no attribute 'admin_order_field' seems to explain why it is not possible to have an orderable from a model property directly into the admin panel.
That explained (probably?) it is time for some mental gymnastics!!
In the previously mentioned part of the documentation we can also see that the admin_order_field can accept query expressions since version 2.1:
Query expressions may be used in admin_order_field. For example:
from django.db.models import Value from django.db.models.functions import Concat class Person(models.Model): first_name = models.CharField(max_length=50) last_name = models.CharField(max_length=50) def full_name(self): return self.first_name + ' ' + self.last_name full_name.admin_order_field = Concat('first_name', Value(' '), 'last_name')
That in conjunction with the previous part about the property() method, allows us to refactor your code and essentially move the annotation part to the model:
class De(models.Model):
...
def calculate_s_d(self):
if self.fr:
return self.de
else:
return self.gd + self.na
calculate_s_d.admin_order_field = Case(
When(fr=True, then='s_d'),
When(fr=False, then=F('gd') + F('na')),
default=Value(0),
output_field=IntegerField(),
)
s_d = property(calculate_s_d)
Finally, on the admin.py we only need:
class DeAdmin(admin.ModelAdmin):
list_display = ("[...]", "s_d")
Unfortunately, this is impossible in current stable Django version (up to 2.2) due to Django admin not fetching admin_order_field from object properties.
Fortunately, it will be possible in upcoming Django version (3.0 and up) which should be released on 2nd of December.
The way to achieve it:
class De(models.Model):
fr = models.BooleanField("[...]")
de = models.SmallIntegerField("[...]")
gd = models.SmallIntegerField("[...]")
na = models.SmallIntegerField("[...]")
# [several_attributes, Meta, __str__() removed for readability]
def s_d(self):
if self.fr:
return self.de
else:
return self.gd + self.na
s_d.admin_order_field = '_s_d'
s_d = property(s_d)
Alternatively, you can create some decorator that will add any attribute to function, before converting it to property:
def decorate(**kwargs):
def wrap(function):
for name, value in kwargs.iteritems():
setattr(function, name, value)
return function
return wrap
class De(models.Model):
fr = models.BooleanField("[...]")
de = models.SmallIntegerField("[...]")
gd = models.SmallIntegerField("[...]")
na = models.SmallIntegerField("[...]")
# [several_attributes, Meta, __str__() removed for readability]
@property
@decorate(admin_order_field='_s_d')
def s_d(self):
if self.fr:
return self.de
else:
return self.gd + self.na
Another possible solution might be to convert the s_d property to a model field and override the model save method to keep it up to date.
# models.py
class De(models.Model):
fr = models.BooleanField("[...]")
de = models.SmallIntegerField("[...]")
gd = models.SmallIntegerField("[...]")
na = models.SmallIntegerField("[...]")
s_d = models.SmallIntegerField("[...]", blank=True)
# [several_attributes, Meta, __str__() removed for readability]
def save(self, *args, **kwargs):
if self.fr:
self.s_d = self.de
else:
self.s_d = self.gd + self.na
super().save(*args, **kwargs)
# admin.py
class DeAdmin(admin.ModelAdmin):
list_display = ("[...]", "s_d", "gd", "na", "de", "fr" )
The default sorting in admin.py will be applied and the value of s_d will be updated every time the model is saved.
There is a caveat to this method if you plan to do a lot of bulk operations, such as bulk_create, update, or delete.
Overridden model methods are not called on bulk operations
Note that the delete() method for an object is not necessarily called
when deleting objects in bulk using a QuerySet or as a result of a
cascading delete. To ensure customized delete logic gets executed, you
can use pre_delete and/or post_delete signals.Unfortunately, there isn’t a workaround when creating or updating
objects in bulk, since none of save(), pre_save, and post_save are
called.
The solutions presented in answers to this question work only if the property inherits ordering from one or more db columns. In this answer I share a solution that works with any property and operates on queryset results with some limitations on multiple ordering.
I am using Django 4.1.
Given the model in the question I put a random value generation in the property just as an example of values retrieved independent from db columns so that we cannot retrieve ordering from db.
import random
class De(models.Model):
fr = models.BooleanField("[...]")
de = models.SmallIntegerField("[...]")
gd = models.SmallIntegerField("[...]")
na = models.SmallIntegerField("[...]")
# [several_attributes, Meta, __str__() removed for readability]
def s_d(self):
"""
Just as example: adds a perturbation so it is not possible
to inherit ordering by db columns
"""
X = random.randrange(128)
if self.fr:
return self.de*X
else:
return self.gd*X + self.na
s_d.admin_order_field = 'cielcio_to_be_removed'
s_d = property(s_d)
The solution I have used sets ChangeList.result_list overloading the ModelAdmin.paginator. This choice has the benefit to operate on computed results while the cons of not using the database ordering.
I use also a class DeChangeList inherited from ChangeList to remove the fake field set as value of admin_order_field so that I got the admin table header clickable.
from django.contrib import admin
from django.contrib.admin.views.main import ChangeList
class DeChangeList(ChangeList):
def get_ordering(self, request, queryset):
"""
Removes the fake field used to show upper
and lower arrow in changelist table header
"""
ordering = super().get_ordering(request, queryset)
if 'cielcio_to_be_removed' in ordering:
ordering.remove('cielcio_to_be_removed')
if '-cielcio_to_be_removed' in ordering:
ordering.remove('-cielcio_to_be_removed')
return ordering
class DeAdmin(admin.ModelAdmin):
list_display = ("[...]", "s_d", "gd", "na", "de", "fr" )
def get_changelist(self, request, **kwargs):
return DeChangeList
def get_paginator(self, request, queryset, per_page, orphans=0, allow_empty_first_page=True):
"""
Intercepts queryset and order by values with 'sorted'
"""
index_s_p = self.list_display.index('s_p')
ordering = request.GET.get('o', "99999")
instances = []
for nf in ordering.split("."):
reverse = int(nf) < 0
if abs(int(nf)) == index_s_p+1:
instances = sorted(
queryset,
key=lambda a: (a.s_p is not None if reverse else a.s_p is None, a.s_p),
reverse=reverse
)
return super().get_paginator(
request, instances or queryset,
per_page, orphans, allow_empty_first_page)