kombu.exceptions.EncodeError: User is not JSON serializable

Question

I have django 1.11.5 app with celery 4.1.0 and I recived all the time:

kombu.exceptions.EncodeError: <User: testuser> is not JSON serializable

my settings.py:

CELERY_BROKER_URL = 'amqp://localhost'
CELERY_RESULT_BACKEND = 'amqp://localhost'
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_RESULT_SERIALIZER = 'json'
CELERY_TASK_SERIALIZER = 'json'
CELERY_TIMEZONE = 'Asia/Makassar'
CELERY_BEAT_SCHEDULE = {}

tasks.py

from __future__ import absolute_import, unicode_literals
from celery import task
from django.contrib.auth.models import User


@task(serializer='json')
def task_number_one():
    user = User.objects.create(username="testuser", email="[email protected]", password="pass")
    return user

I call task in the view:

def form_valid(self, form):
    form.instance.user = self.request.user
    task_number_one.delay()
    return super().form_valid(form)

Asked By: user9357277

||

Source

Answer 1

This is because you are using the JSON serializer for task serialization (as indicated by the setting CELERY_TASK_SERIALIZER = 'json'), but you are trying to return a model instance (which cannot be serialized into JSON).

You have two options:

1) Don’t pass the instance, pass the primary key of the instance and then look up the object inside your task.

2) Use the pickle task serializer instead. This will allow you to pass objects as arguments to your tasks and return them, but comes with it’s own security concerns.

Answered By: MrName

Answer 2

The error is because of Celery expecting a JSON data from your task function while you returned a User instance.

How to solve this ?
You are not using that return data anywhere, so you don’t have to return it. That is you can remove return user from the task function.
Or,
return a Json data from the task function will solve this issue as well

Solution 1

@task(serializer='json')
def task_number_one():
    user = User.objects.create(username="testuser", email="[email protected]", password="pass")

Solution 2

@task(serializer='json')

def task_number_one():
    user = User.objects.create(username="testuser", email="[email protected]", password="pass")
    # return some json data instead of `USER` instance
    return {"status": True}  # Change is here

Answered By: JPG

Answer 3

My celery task:

response = {}
...
except HTTPError as e:
        response.update(
            {
                'status': False,
                'code': e.status_code,
                'error': e.body,
            },
        )
    ...
return response

I had EncodeError(TypeError('Object of type bytes is not JSON serializable') and kombu.exceptions.EncodeError although the response is a dict that shouldn’t be a problem when JSON encoding.

It turned out that e.body is of type bytes. I changed to e.body.decode('utf-8') and the problem disappeared.

Answered By: Serhii Kushchenko

kombu.exceptions.EncodeError: User is not JSON serializable

Question:

Answers: