Celery – How to send task from remote machine?
Question:
We have a server running celery workers and a Redis queue. The tasks are defined on that server.
I need to be able to call these tasks from a remote machine.
I know that it is done using send_task but I still haven’t figured out HOW? How do I tell send_task where the queue is? Where do I pass connection params (or whatever needed)? I’ve been looking for hours and all I can find is this:
from celery.execute import send_task
send_task('tasks.add')
Well, that means that I need celery on my calling machine as well. But what else do I need to set up?
Answers:
There are several ways to define routing rules, the most general of which is the custom router object. In all cases, the caller just provides a routing_key parameter in send_task, delay or apply_async and the router determines which queue to send the task into.
This may be a way:
Creating a Celery object and using send_task from that object, the object can have the configuration to find the broker.
from celery import Celery
celery = Celery()
celery.config_from_object('celeryconfig')
celery.send_task('tasks.add', (2,2))
celeryconfig is a file containing the celery configuration, there are other ways set config on the celery object.
on the remote machine, start up celery with the broker_url pointing to the machine you want to run the tasks on. Then just submit the tasks (if you have specific queues to submit to, then add the appropriate routing keys).
What you found was right.
from celery.execute import send_task
send_task('tasks.add')
If any args needed
send_task('tasks.add', kwargs={'a': 1, 'b': 2})
According to the latest update, this method is one of the app methods.
for example:
from project.celery_general import general_app
# general app is celery app you defined
general_app.send_task(name='your_task_name', *args, **kwargs)
We have a server running celery workers and a Redis queue. The tasks are defined on that server.
I need to be able to call these tasks from a remote machine.
I know that it is done using send_task but I still haven’t figured out HOW? How do I tell send_task where the queue is? Where do I pass connection params (or whatever needed)? I’ve been looking for hours and all I can find is this:
from celery.execute import send_task
send_task('tasks.add')
Well, that means that I need celery on my calling machine as well. But what else do I need to set up?
There are several ways to define routing rules, the most general of which is the custom router object. In all cases, the caller just provides a routing_key parameter in send_task, delay or apply_async and the router determines which queue to send the task into.
This may be a way:
Creating a Celery object and using send_task from that object, the object can have the configuration to find the broker.
from celery import Celery
celery = Celery()
celery.config_from_object('celeryconfig')
celery.send_task('tasks.add', (2,2))
celeryconfig is a file containing the celery configuration, there are other ways set config on the celery object.
on the remote machine, start up celery with the broker_url pointing to the machine you want to run the tasks on. Then just submit the tasks (if you have specific queues to submit to, then add the appropriate routing keys).
What you found was right.
from celery.execute import send_task
send_task('tasks.add')
If any args needed
send_task('tasks.add', kwargs={'a': 1, 'b': 2})
According to the latest update, this method is one of the app methods.
for example:
from project.celery_general import general_app
# general app is celery app you defined
general_app.send_task(name='your_task_name', *args, **kwargs)