Celery not running (Permission Denied)
Question:
I’m not running a Django Project, this is just a simple python project and I want to demonize the celery. Initially I was running it in the shell. Celery sometimes hangs in between(known issue). Hence I need to restart it again and again. Therefore, I need to daemonize it, so that I can run a script that restarts the celery on its own via a cronjob.
I’ve looked onto various threads on SO, but with no luck.
I’ve created a user and group , both named as celery by following commands:-
sudo groupadd celery
sudo useradd -g celery celery
I created this file as: /etc/default/celeryd
CELERYD_NODES="w1"
CELERY_BIN="/usr/local/bin/celery"
CELERY_APP="tasks"
CELERYD_CHDIR="/home/cube26/Desktop/cube26/c26-quicklook"
CELERYD_OPTS="--time-limit=300 --concurrency=8 -Q BBC,BGR,FASTCOMPANY"
CELERY_CONFIG_MODULE="celeryconfig" #I dont know what this is for?
CELERYD_LOG_FILE="/var/log/celery/%n.log"
CELERYD_PID_FILE="/var/run/celery/%n.pid"
CELERY_CREATE_DIRS=1
CELERYD_USER="celery"
CELERYD_GROUP="celery"
Then downloaded this file and kept it inside /etc/init.d/celeryd.
Then chmod +x /etc/init.d/celeryd to make it executable.
Thats all I did.
and I’m still getting an error saying IOError: [Errno 13] Permission denied: '/var/log/celery/w1.log'
What wrong am I doing? Please help me rectify it.
Answers:
Based on comments, You have wrong write permissions to log file.
Please change the ownership to celery using:
chown -R celery:celery /var/log/celery/
chown -R celery:celery /var/run/celery/
-R switch is used to change permissions recursive inside directory
Tell celery where to put the pid file:
celery -A config --pidfile=${CELERYD_PID_FILE}
where CELERYD_PID_FILE is /var/www/my_site/celery.pid for example.
you have create two file for this and give the permission for you pc user name
sudo mkdir -p -m 2755 /var/run/celery
sudo mkdir -p -m 2755 /var/log/celery
sudo chown $USER:$USER /var/run/celery
sudo chown $USER:$USER /var/log/celery
instead of $USER use your user name
Tl;dr
If you’re using systemd for daemonization, your file /etc/tmpfiles.d/celery.conf should look like this:
d /run/celery 0755 myuser myusergroup -
d /var/log/celery 0755 myuser myusergroup -
A little more detail
For those using systemd, I found that I’d been using the wrong user when creating working directories for celery’s logs and once I’d updated /etc/tmpfiles.d/celery.conf with the correct user and user group, my issues were solved.
The documentation mentions:
You can also use systemd-tmpfiles in order to create working directories (for logs and pid).
file: /etc/tmpfiles.d/celery.conf
And gives the example:
d /run/celery 0755 celery celery -
d /var/log/celery 0755 celery celery -
Your file should look like this:
d /run/celery 0755 myuser myusergroup -
d /var/log/celery 0755 myuser myusergroup -
I’m not running a Django Project, this is just a simple python project and I want to demonize the celery. Initially I was running it in the shell. Celery sometimes hangs in between(known issue). Hence I need to restart it again and again. Therefore, I need to daemonize it, so that I can run a script that restarts the celery on its own via a cronjob.
I’ve looked onto various threads on SO, but with no luck.
I’ve created a user and group , both named as celery by following commands:-
sudo groupadd celery
sudo useradd -g celery celery
I created this file as: /etc/default/celeryd
CELERYD_NODES="w1"
CELERY_BIN="/usr/local/bin/celery"
CELERY_APP="tasks"
CELERYD_CHDIR="/home/cube26/Desktop/cube26/c26-quicklook"
CELERYD_OPTS="--time-limit=300 --concurrency=8 -Q BBC,BGR,FASTCOMPANY"
CELERY_CONFIG_MODULE="celeryconfig" #I dont know what this is for?
CELERYD_LOG_FILE="/var/log/celery/%n.log"
CELERYD_PID_FILE="/var/run/celery/%n.pid"
CELERY_CREATE_DIRS=1
CELERYD_USER="celery"
CELERYD_GROUP="celery"
Then downloaded this file and kept it inside /etc/init.d/celeryd.
Then chmod +x /etc/init.d/celeryd to make it executable.
Thats all I did.
and I’m still getting an error saying IOError: [Errno 13] Permission denied: '/var/log/celery/w1.log'
What wrong am I doing? Please help me rectify it.
Based on comments, You have wrong write permissions to log file.
Please change the ownership to celery using:
chown -R celery:celery /var/log/celery/
chown -R celery:celery /var/run/celery/
-R switch is used to change permissions recursive inside directory
Tell celery where to put the pid file:
celery -A config --pidfile=${CELERYD_PID_FILE}
where CELERYD_PID_FILE is /var/www/my_site/celery.pid for example.
you have create two file for this and give the permission for you pc user name
sudo mkdir -p -m 2755 /var/run/celery
sudo mkdir -p -m 2755 /var/log/celery
sudo chown $USER:$USER /var/run/celery
sudo chown $USER:$USER /var/log/celery
instead of $USER use your user name
Tl;dr
If you’re using systemd for daemonization, your file /etc/tmpfiles.d/celery.conf should look like this:
d /run/celery 0755 myuser myusergroup -
d /var/log/celery 0755 myuser myusergroup -
A little more detail
For those using systemd, I found that I’d been using the wrong user when creating working directories for celery’s logs and once I’d updated /etc/tmpfiles.d/celery.conf with the correct user and user group, my issues were solved.
The documentation mentions:
You can also use systemd-tmpfiles in order to create working directories (for logs and pid).
file: /etc/tmpfiles.d/celery.conf
And gives the example:
d /run/celery 0755 celery celery - d /var/log/celery 0755 celery celery -
Your file should look like this:
d /run/celery 0755 myuser myusergroup -
d /var/log/celery 0755 myuser myusergroup -