Which is the best way to programatically terminate (cancel) a celery task
Question:
According to Celery’s documentation, we should not use the terminate option in revoke() function to cancel an executing task:
The terminate option is a last resort for administrators when a task is stuck. It’s not for terminating the task, it’s for terminating the process that’s executing the task, and that process may have already started processing another task at the point when the signal is sent, so for this reason you must never call this programmatically.
http://docs.celeryproject.org/en/latest/userguide/workers.html#revoke-revoking-tasks
So, my question is, how should I properly do if I need to cancel some running tasks programmatically?
Answers:
If you really need to kill the running task, using terminate=True in revoke() method is the proper way and your best option. The problem is that doing this has no guaranteed effect or, better said, can have negative side effects. To understand why, it’s necessary to know how revoking works and what terminate=True does. When you call revoke(), you are sending broadcast message to your workers via the broker. That means that before the time you realize you want/need to to kill the task, sending the revoke message and receiving the control command by the worker, the worker might have already finished the task, grabbed another task from the queue and started working on it. When it finally receives the command, it just kills the worker process, which, by that time, might already be working on different task.
If you know beforehand you might need to abort your tasks and can adjust the tasks code, you can take a look at abortable tasks.
revoke(task_id, terminate=True) method is redicilious. If you want to kill task’s thread, you should inherit your task class from from celery.contrib.abortable import AbortableTask.
And check self.is_aborted() flag period of time you desired.
When it’s True, exit from Thread with continiue.This is requirement in consumer.
You can set self.is_aborted() as True in producer for kill thread where ever you want.
Let me demonstrate in example
in consumer;
in producer set self.is_aborted() as True with t.abort() whereever you want;
According to Celery’s documentation, we should not use the terminate option in revoke() function to cancel an executing task:
The terminate option is a last resort for administrators when a task is stuck. It’s not for terminating the task, it’s for terminating the process that’s executing the task, and that process may have already started processing another task at the point when the signal is sent, so for this reason you must never call this programmatically.
http://docs.celeryproject.org/en/latest/userguide/workers.html#revoke-revoking-tasks
So, my question is, how should I properly do if I need to cancel some running tasks programmatically?
If you really need to kill the running task, using terminate=True in revoke() method is the proper way and your best option. The problem is that doing this has no guaranteed effect or, better said, can have negative side effects. To understand why, it’s necessary to know how revoking works and what terminate=True does. When you call revoke(), you are sending broadcast message to your workers via the broker. That means that before the time you realize you want/need to to kill the task, sending the revoke message and receiving the control command by the worker, the worker might have already finished the task, grabbed another task from the queue and started working on it. When it finally receives the command, it just kills the worker process, which, by that time, might already be working on different task.
If you know beforehand you might need to abort your tasks and can adjust the tasks code, you can take a look at abortable tasks.
revoke(task_id, terminate=True) method is redicilious. If you want to kill task’s thread, you should inherit your task class from from celery.contrib.abortable import AbortableTask.
And check self.is_aborted() flag period of time you desired.
When it’s True, exit from Thread with continiue.This is requirement in consumer.
You can set self.is_aborted() as True in producer for kill thread where ever you want.
Let me demonstrate in example
in consumer;
in producer set self.is_aborted() as True with t.abort() whereever you want;