Serializing Foreign Key objects in Django

Question:

I have been working on developing some RESTful Services in Django to be used with both Flash and Android apps.

Developing the services interface has been quite simple, but I have been running into an issue with serializing objects that have foreign key and many to many relationships.

I have a model like this:

class Artifact( models.Model ):
    name                = models.CharField( max_length = 255 )
    year_of_origin      = models.IntegerField( max_length = 4, blank = True, null = True )
    object_type         = models.ForeignKey( ObjectType, blank = True, null = True )
    individual          = models.ForeignKey( Individual, blank = True, null = True )
    notes               = models.TextField( blank = True, null = True )

Then I would perform a query on this model like this, using select_related(), to be sure that foreign key relationships are followed:

artifact = Artifact.objects.select_related().get(pk=pk)

Once I have the object, I serialize it, and pass that back to my view:

serializers.serialize( "json", [ artifact ] )

This is what I get back, note that the foreign keys (object_type and individual) are just the id’s to their related objects.

[
      {
            pk: 1
            model: "artifacts.artifact"
            fields: {
                year_of_origin: 2010
                name: "Dummy Title"
                notes: ""
                object_type: 1
                individual: 1
            }
      }
]

This is great, but what I was hoping for when using select_related() was that it would automatically populate the foreign key fields with the related object, not just the object’s id.

I am recent convert to Django, but put in a fair amount of time developing with CakePHP.

What I really like about the Cake ORM was that it would follow the relationships and create nested objects by default, with the ability to unbind the relationships when you were calling your query.

This made it very easy to abstract the services in a way that did not require any intervention on a case by case basis.

I see that Django does not do this by default, but is there a way to automatically serialize an object and all of it’s related objects? Any tips or reading would be much appreciated.

Asked By: Kieran Lynn

||

Answers:

I had a similar requirement although not for RESTful purposes. I was able to achieve what I needed by using a “full” serializing module, in my case Django Full Serializers. This is part of wadofstuff and is distributed under the new BSD license.

Wadofstuff makes this quite easy. For e.g. in your case you’d need to do the following:

First, install wadofstuff.

Second, add the following setting to your settings.py file:

SERIALIZATION_MODULES = {
    'json': 'wadofstuff.django.serializers.json'
}

Third, make a slight change to the code used for serialization:

artifact = Artifact.objects.select_related().get(pk=pk)
serializers.serialize( "json", [ artifact ], indent = 4, 
    relations = ('object_type', 'individual',))

The key change is the relations keyword parameter. The only (minor) gotcha is to use the name of the fields forming the relation not the names of the related models.

Caveat

From the documentation:

The Wad of Stuff serializers are 100% compatible with the Django serializers when serializing a model. When deserializing a data stream the the Deserializer class currently only works with serialized data returned by the standard Django serializers.

(Emphasis added)

Hope this helps.

Answered By: Manoj Govindan

You can find more information on this ticket:

Allow In-depth serialization by specifying depth to follow relationship
https://code.djangoproject.com/ticket/4656

Answered By: guettli

UPDATE:
Actually Manoj’s solution is a bit outdated, Wad of Stuff’s serializer has been left un-updated for some time and when I tried that, it seems that it does not support Django 1.6 anymore.

However, take a look at Django’s official doc here. It does provide some way around using the built-in natural key. It seems that django’s built-in serializer has a a little problem supporting using ImageField as part of the natural key. But that can be easily fixed by your self.

Answered By: Shen Haocheng

Adding a newer answer to this older question: I created and recently published django-serializable-model as an easily extensible way to serialize models, managers, and querysets. When your models extend SerializableModel, they receive an overridable .serialize method that has built-in support for all relations.

Using your example, once all of the involved models extend SerializableModel:

joins = ['object_type', 'individual']
artifact = Artifact.objects.select_related(*joins).get(pk=pk)
artifact.serialize(*joins)

Calling .serialize with the relations as arguments will have the library recurse over the related objects, calling .serialize on them as well. This returns a dictionary that looks like:

{
  'id': 1,
  'year_of_origin': 2010,
  'name': 'Dummy Title',
  'notes': '',
  'object_type_id': 1,
  'individual_id': 1,
  'object_type': { ... nested object here ... },
  'individual': { ... nested object here ... }
}

You can then call json.dumps on this dictionary to transform it to JSON.

By default, extending SerializableModel will also set the model’s manager to SerializableManager (you can extend it yourself if you’re using a custom manager) which uses SerializableQuerySet. This means you can call .serialize on a manager or queryset as well:

artifacts = Artifact.objects.select_related(*joins).all()
artifacts.serialize(*joins)

This simply calls .serialize on each model object in the queryset, returning a list of dictionaries in the same format as above.

django-serializable-model also allows you to easily override the default behavior on a per model basis, giving you the ability to do things like: add allowlists or denylists applied to each model’s .serialize, always serialize certain joins (so you don’t have to add them as arguments all the time), and more!

Answered By: agilgur5

I’m aware this topic is years old, however, I’m sharing my solution for the people still searching for an answer (during my search, I ended up here).

Please note, I was looking for a simple function which would give me nested (foreign key) objects/dictionaries (which could contain nested (foreign key) objects/dictionaries as well) within my model/queryset which I could then convert to JSON.

In my models.py, I have a custom function (not within a model class):

Models.py

def django_sub_dict(obj):
    allowed_fields = obj.allowed_fields() # pick the list containing the requested fields
    sub_dict = {}
    for field in obj._meta.fields: # go through all the fields of the model (obj)
        if field.name in allowed_fields: # be sure to only pick fields requested
            if field.is_relation: # will result in true if it's a foreign key
                sub_dict[field.name] = django_sub_dict(
                    getattr(obj, field.name)) # call this function, with a new object, the model which is being referred to by the foreign key.
            else: # not a foreign key? Just include the value (e.g., float, integer, string)
                sub_dict[field.name] = getattr(obj, field.name)
    return sub_dict # returns the dict generated

This function loops through all the fields in a models.Model object, if the models.Model is provided. I call the function within a model as follows (for completeness sake, including one entire model):

the same Models.py

class sheet_categories(models.Model):
    id = models.AutoField(primary_key=True, unique=True)
    create_date = models.DateField(auto_now_add=True)
    last_change = models.DateField(auto_now=True)
    name = models.CharField(max_length=128)
    sheet_type = models.ForeignKey(
        sheet_types, models.SET_NULL, blank=False, null=True)
    balance_sheet_sort = models.IntegerField(unique=True)

    def allowed_fields(self):
        return [
                'name',
                'sheet_type',
                'balance_sheet_sort',
                ]

    def natural_key(self):
        return django_sub_dict(self) # call the custom function (which is included in this models.py)

Note:
The nested JSON objects will only contain fields which are included in the allowed_fields of a model. Thus not including sensitive information.

To ultimately generate a JSON, I have the following view in my views.py.

views.py

class BalanceSheetData(ListView): # I believe this doesn't have to **be** a ListView.
    model = models.sheet_categories

    def get_queryset(self):
        return super().get_queryset().filter() # the filter is for future purposes. For now, not relevant

    def get(self, request, *args, **kwargs):
        context = {
            'queryset': serializers.serialize("json",
                                          self.get_queryset(),
                                          use_natural_foreign_keys=True, # this or the one below makes django include the natural_key() within a model. Not sure.
                                          use_natural_primary_keys=True, # this or the one above makes django include the natural_key() within a model. Not sure.
                                          ),
        }
        return JsonResponse(context)

This ultimately provided me with all the nested details I required in a JSON response. Although I do not share the JSON response, as this one is barely readable.

Feel free to comment.

Answered By: Jens van Hellemondt
Categories: questions Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.