Django models and orm and foreign key problems

Question

i want got all of related model’s data, but i got only some of related data. to avoid N+1 problem, i used select_related() and prefetch_related() methods.

At first, i have these models:

class OrderList(models.Model):
    order_id=models.CharField(max_length=100)
    order_status=models.CharField(max_length=100)

class ProductInOrder(models.Model):
    order_key=models.ForeignKey(OrderList, on_delete=models.CASCADE, related_name="order_key")
    
    product_id=models.CharField(max_length=100)
    product_price=models.CharField(max_length=100)

class MemosInProduct(models.Model):
    product_key=models.ForeignKey(ProductInOrder, on_delete=models.CASCADE, related_name="product_key")

    memo=models.CharField(max_length=100)
    blahblah some codes...

a short explan of this models, one OrderList has got many of ProductInOrder ( one to many )
and one ProductInOrder has got many of MemosInProduct( one to many )

then, i run this codes in django shell:

order_list=OrderList.object.select_related("order_key", "product_key").all()

i excepted all of OrderList datas with related all of datas combine with it(product, memos):
EXCEPTED

order_list[0].order_key[0].product_key[0].memo
order_list[0].order_key[0].product_key[1].memo
order_list[0].order_key[1].product_key[0].memo
…

but i got:
OUTPUT

Invalid field name(s) given in select_related: ‘order_key’, ‘product_key’. Choices are: (none)

i also tried this:

order_list=MemosInProduct.object.select_related("order_key", "product_key").all()

but outputs are not matched i except.

Asked By: Loire

||

Source

Answer 1

From Django Documentation select_related fetch the related object(foreign key relationship) if the foreign key is present in the model. That means it fetches the forward-related objects.

For example:

def OrderList(models.Model):
    order_id=models.CharField(max_length=100)
    order_status=models.CharField(max_length=100))

You can’t use select_related as there are no forward relations in this model. So, Django raises errors.

Invalid field name(s) given in select_related: ‘order_key’, ‘product_key’. Choices are: (none)

But, you can use select_related for ProductInOrder and MemosInProduct as there are forward relationships which are order_key and product_key respectively for ProductInOrder and MemosInProduct.

For backward relationships (the foreign key is not present in the query model) you should use prefetch_related.

So, In your case, the query should be

OrderList.objects.prefetch_related("order_key", "order_key__product_key").all()

Here, you can see I also changed the product_key to order_key__product_key because the product_key is not the related object of the OrderList it is a related object of ProductInOrder

This is the theory though. Maybe you need to use Prefetch in your query. An example

OrderList.objects.prefetch_related(Prefetch('order_key', queryset=ProductInOrder.objects.all(), to_attr='order_keys')).all()

However, read the Django docs for details.

Answered By: Almabud

Answer 2

I’d tackle this by working in the opposite direction, and retrieve MemosInProduct objects each with their related objects. For example, to get everything related to OrderList objects with order_status = 'overdue' (I’m guessing):

queryset = MemosInProduct.objects.order_by(
               'product_key__order__key__order_id',
               'product_key__product_id'
            ).select_related(
               'product_key', 'product_key__order_key'
            ).filter(
               product_key__order__key__order_status='overdue'
            )

In the loop over the retrieved objects you can tell when you have progressed to a different ProductInOrder or OrderList if you keep track of the previous item within the loop

last_object = None
for memo in queryset:
    product = memo.product_key
    order = product.order_key
    if ( last_object is None or 
         last_object.product_key.order_id != order.id ):
       # do stuff for a changed order compared to last (or the first loop)
    if ( last_object is None or 
         last_object.product_key_id != product.pk) :
       # do stuff for a changed product compared to last

    # do stuff for every memo
    last_object = memo

Alternatively you can construct a list of lists of lists, which is an easy thing to walk through in the template language when passed in the context to be rendered.

Remember that Python assignment is name-binding, not object-copying, so this sort of thing is efficient. In the loop, product and order were prefetched by the select_related.

Answered By: nigel222

Django models and orm and foreign key problems

Question:

Answers: