Updating a Haystack search index with Django + Celery
Question:
In my Django project I am using Celery. I switched over a command from crontab to be a periodic task and it works well but it is just calling a method on a model. Is it possible to update my Haystack index from a periodic task as well? Has anyone done this?
/manage.py update_index
That’s the command to update the index from the Haystack documentation but I’m not sure how to call that from a task.
Answers:
the easiest way to do this would probably be to run the management command directly from python and run it in your task
from haystack.management.commands import update_index
update_index.Command().handle()
As for version 2.0.0 beta of haystack, this code should work:
from haystack.management.commands import update_index
update_index.Command().handle(using='default')
https://github.com/django-haystack/celery-haystack
I find this package to be a great, easy plug-in app to provide haystack indexing via celery. I used it in a few projects.
Also, since version 2 of the haystack you can call rebuild index from python as
from haystack.management.commands import update_index, rebuild_index
rebuild_index.Command().handle(interactive=False)
Where the “interactive” would prevent haystack asking question if you really want to rebuild index. This is equivalent to –no-input command line option.
If you use xapian as FTS backend please remember that multithreaded updates to index would result in DB Write Lock. So, the solution with celery-haystack package does attempt to spread index update into multiple workers (multiple thread) resulting in the lock with xapian.
In my Django project I am using Celery. I switched over a command from crontab to be a periodic task and it works well but it is just calling a method on a model. Is it possible to update my Haystack index from a periodic task as well? Has anyone done this?
/manage.py update_index
That’s the command to update the index from the Haystack documentation but I’m not sure how to call that from a task.
the easiest way to do this would probably be to run the management command directly from python and run it in your task
from haystack.management.commands import update_index
update_index.Command().handle()
As for version 2.0.0 beta of haystack, this code should work:
from haystack.management.commands import update_index
update_index.Command().handle(using='default')
https://github.com/django-haystack/celery-haystack
I find this package to be a great, easy plug-in app to provide haystack indexing via celery. I used it in a few projects.
Also, since version 2 of the haystack you can call rebuild index from python as
from haystack.management.commands import update_index, rebuild_index
rebuild_index.Command().handle(interactive=False)
Where the “interactive” would prevent haystack asking question if you really want to rebuild index. This is equivalent to –no-input command line option.
If you use xapian as FTS backend please remember that multithreaded updates to index would result in DB Write Lock. So, the solution with celery-haystack package does attempt to spread index update into multiple workers (multiple thread) resulting in the lock with xapian.