RQ – Empty & Delete Queues
Question:
I’m using RQ, and I have a failed queue with thousands of items, and another test queue I created a while back for testing which is now empty and unused. I’m wondering how to remove all jobs from the failed queue, and delete the test queue altogether?
Apologies for the basic question, but I can’t find info on this in the RQ docs, and I’m completely new to both Redis and RQ… Thanks in advance!
Answers:
Cleanup using rq
RQ offers methods to make any queue empty:
>>> from redis import Redis
>>> from rq import Queue
>>> qfail = Queue("failed", connection=Redis())
>>> qfail.count
8
>>> qfail.empty()
8L
>>> qfail.count
0
You can do the same for test queue, if you have it still present.
Cleanup using rq-dashboard
Install rq-dashboard:
$ pip install rq-dashboard
Start it:
$ rq-dashboard
RQ Dashboard, version 0.3.4
* Running on http://0.0.0.0:9181/
Open in browser.
Select the queue
Click the red button "Empty"
And you are done.
Python function Purge jobs
If you run too old Redis, which fails on command used by RQ, you still might sucess with deleting
jobs by python code:
The code takes a name of a queue, where are job ids.
Usilg LPOP we ask for job ids by one.
Adding prefix (by default "rq:job:") to job id we have a key, where is job stored.
Using DEL on each key we purge our database job by job.
>>> import redis
>>> r = redis.StrictRedis()
>>> qname = "rq:queue:failed"
>>> def purgeq(r, qname):
... while True:
... jid = r.lpop(qname)
... if jid is None:
... break
... r.delete("rq:job:" + jid)
... print(jid)
...
>>> purge(r, qname)
a0be3624-86c1-4dc4-bb2e-2043d2734b7b
3796c312-9b02-4a77-be89-249aa7325c25
ca65f2b8-044c-41b5-b5ac-cefd56699758
896f70a7-9a35-4f6b-b122-a08513022bc5
Monitoring tool rqinfo can empty failed queue.
Just make sure you have an active virtualenv with rq installed, and run
$ rqinfo --empty-failed-queue
See rqinfo --help for more details.
– 2016 –
You can now use rq’s empty option form command line:
/path/to/rq empty queue_name
So you can use it to empty any queue not just the failed one
you can just login to redis and clear all queues
to login
user@user:~$ redis-cli
enter this command and hit enter
FLUSHALL
And you’re done
Edit: This will delete everything stored in redis
By default ‘rq’ jobs are prefixed by ‘rq:job’. So you can delete these jobs from the redis using following command,
redis-cli KEYS rq:job:* | xargs redis-cli DEL
none of the above solutions worked
failed Queue is not registered under queues
so I move all of the failed jobs to default Queue and use
rq empty queue_name –url [redis-url]
Here’s how to clear the failed job registry using django_rq:
import django_rq
from rq.registry import FailedJobRegistry
queue = django_rq.get_queue("your_queue_with_failed_jobs")
registry = FailedJobRegistry(queue=queue)
for job_id in registry.get_job_ids():
registry.remove(job_id)
– 2022 –
I was struggling with this as well and this is a piece of code which works for me.
It loops over queues name (in my case, ‘default’ and ‘low’), fetch all failed jobs for each queue and remove them
import django_rq
from rq.registry import FailedJobRegistry
from redis import Redis
from rq.job import Job
from django.conf import settings
redis = Redis(host=settings.REDIS_HOST, port=settings.REDIS_PORT)
queues = ["default", "low"]
for q in queues:
queue = django_rq.get_queue(q)
registry = FailedJobRegistry(queue=queue)
for job_id in registry.get_job_ids():
job = Job.fetch(job_id, connection=redis)
registry.remove(job)
I’m using RQ, and I have a failed queue with thousands of items, and another test queue I created a while back for testing which is now empty and unused. I’m wondering how to remove all jobs from the failed queue, and delete the test queue altogether?
Apologies for the basic question, but I can’t find info on this in the RQ docs, and I’m completely new to both Redis and RQ… Thanks in advance!
Cleanup using rq
RQ offers methods to make any queue empty:
>>> from redis import Redis
>>> from rq import Queue
>>> qfail = Queue("failed", connection=Redis())
>>> qfail.count
8
>>> qfail.empty()
8L
>>> qfail.count
0
You can do the same for test queue, if you have it still present.
Cleanup using rq-dashboard
Install rq-dashboard:
$ pip install rq-dashboard
Start it:
$ rq-dashboard
RQ Dashboard, version 0.3.4
* Running on http://0.0.0.0:9181/
Open in browser.
Select the queue
Click the red button "Empty"
And you are done.
Python function Purge jobs
If you run too old Redis, which fails on command used by RQ, you still might sucess with deleting
jobs by python code:
The code takes a name of a queue, where are job ids.
Usilg LPOP we ask for job ids by one.
Adding prefix (by default "rq:job:") to job id we have a key, where is job stored.
Using DEL on each key we purge our database job by job.
>>> import redis
>>> r = redis.StrictRedis()
>>> qname = "rq:queue:failed"
>>> def purgeq(r, qname):
... while True:
... jid = r.lpop(qname)
... if jid is None:
... break
... r.delete("rq:job:" + jid)
... print(jid)
...
>>> purge(r, qname)
a0be3624-86c1-4dc4-bb2e-2043d2734b7b
3796c312-9b02-4a77-be89-249aa7325c25
ca65f2b8-044c-41b5-b5ac-cefd56699758
896f70a7-9a35-4f6b-b122-a08513022bc5
Monitoring tool rqinfo can empty failed queue.
Just make sure you have an active virtualenv with rq installed, and run
$ rqinfo --empty-failed-queue
See rqinfo --help for more details.
– 2016 –
You can now use rq’s empty option form command line:
/path/to/rq empty queue_name
So you can use it to empty any queue not just the failed one
you can just login to redis and clear all queues
to login
user@user:~$ redis-cli
enter this command and hit enter
FLUSHALL
And you’re done
Edit: This will delete everything stored in redis
By default ‘rq’ jobs are prefixed by ‘rq:job’. So you can delete these jobs from the redis using following command,
redis-cli KEYS rq:job:* | xargs redis-cli DEL
none of the above solutions worked
failed Queue is not registered under queues
so I move all of the failed jobs to default Queue and use
rq empty queue_name –url [redis-url]
Here’s how to clear the failed job registry using django_rq:
import django_rq
from rq.registry import FailedJobRegistry
queue = django_rq.get_queue("your_queue_with_failed_jobs")
registry = FailedJobRegistry(queue=queue)
for job_id in registry.get_job_ids():
registry.remove(job_id)
– 2022 –
I was struggling with this as well and this is a piece of code which works for me.
It loops over queues name (in my case, ‘default’ and ‘low’), fetch all failed jobs for each queue and remove them
import django_rq
from rq.registry import FailedJobRegistry
from redis import Redis
from rq.job import Job
from django.conf import settings
redis = Redis(host=settings.REDIS_HOST, port=settings.REDIS_PORT)
queues = ["default", "low"]
for q in queues:
queue = django_rq.get_queue(q)
registry = FailedJobRegistry(queue=queue)
for job_id in registry.get_job_ids():
job = Job.fetch(job_id, connection=redis)
registry.remove(job)