Developing with Django+Celery without running `celeryd`?
Question:
In development, it’s a bit of a hassle to run the celeryd as well as the Django development server. Is it possible to, for example, ask celery to run tasks synchronously during development? Or something similar?
Answers:
Yes you can do this by setting CELERY_TASK_ALWAYS_EAGER = True in your settings.
(Constant was previously called CELERY_ALWAYS_EAGER)
http://docs.celeryproject.org/en/latest/userguide/configuration.html#task-execution-settings
There’s also a custom Django test runner in django-celery that helps with CELERY_ALWAYS_EAGER.
Read more about using Celery with Django on the docs.
In version 4.0 of Celery CELERY_ALWAYS_EAGER setting was replaced by CELERY_TASK_ALWAYS_EAGER in Django settings.py or task_always_eager natively in celery conf.
Since the numerous changes in the celery configuration from version 3.x to 4.x are spread over many lines, I suggest to use the built-in settings migration tool.
celery upgrade settings --django
source:
Luckily you don’t have to manually change the files, as the celery
upgrade settings –django program should do the right thing.
http://docs.celeryproject.org/en/latest/whatsnew-4.0.html#step-4-upgrade-to-celery-4-0
After 5+ years of writing Celery tasks I have noticed a pattern I have developed that can help with testing and ease of development – I realised it is much better if my Celery tasks are thin wrappers around the regular Python functions that I typically put in myproject.impl package. Celery tasks may contain some strictly Celery related logic, like using distributed locking for an example, explicit retry logic, etc.
In development, it’s a bit of a hassle to run the celeryd as well as the Django development server. Is it possible to, for example, ask celery to run tasks synchronously during development? Or something similar?
Yes you can do this by setting CELERY_TASK_ALWAYS_EAGER = True in your settings.
(Constant was previously called CELERY_ALWAYS_EAGER)
http://docs.celeryproject.org/en/latest/userguide/configuration.html#task-execution-settings
There’s also a custom Django test runner in django-celery that helps with CELERY_ALWAYS_EAGER.
Read more about using Celery with Django on the docs.
In version 4.0 of Celery CELERY_ALWAYS_EAGER setting was replaced by CELERY_TASK_ALWAYS_EAGER in Django settings.py or task_always_eager natively in celery conf.
Since the numerous changes in the celery configuration from version 3.x to 4.x are spread over many lines, I suggest to use the built-in settings migration tool.
celery upgrade settings --django
source:
Luckily you don’t have to manually change the files, as the celery
upgrade settings –django program should do the right thing.
http://docs.celeryproject.org/en/latest/whatsnew-4.0.html#step-4-upgrade-to-celery-4-0
After 5+ years of writing Celery tasks I have noticed a pattern I have developed that can help with testing and ease of development – I realised it is much better if my Celery tasks are thin wrappers around the regular Python functions that I typically put in myproject.impl package. Celery tasks may contain some strictly Celery related logic, like using distributed locking for an example, explicit retry logic, etc.