Running Python process with cronjob and checking it is still running every minute
Question:
I have a Python script that I’d like to run from a cronjob and then check every minute to see if it is still running and if not then start it again.
Cronjob is:
/usr/bin/python2.7 /home/mydir/public_html/myotherdir/script.py
there is some info on this but most answers don’t really detail the full process clearly, e.g.:
Using cron job to check if python script is running
e.g. in that case, it doesn’t state how to run the initial process and record the PID. It leaves me with a lot of questions unfortunately.
Therefore, could anyone give me a simple guide to how to do this?
e.g. full shell script required, what command to start the script, and so on.
Answers:
You should actually take a step back and think what and why you are doing it
Do you need to actually start your long_running program from cron? The reason i ask is that you then write another program(watcher) that starts your long_running program. I hope that you see that yo actually do not need cron to start your long_running program.
So what you have is a watcher firing every minute and starting long_running program if it is not running.
Now you need to actually understand what you are trying to do because there are many ways this can be done. If this is an exercise watcher can call ps and work out from there if it runs and start it.
in bash you can do
[ -z "$(ps -ef | grep -v grep | grep myprogname)" ] && { echo runit; }
you just replace myprogname with the name and run it with command that starts your long_running code on the back ground – perhaps
however you should use another tool. systemd, daemontools or others
It’s not that hard. First set up the crontab to run a checker every minute:
* * * * * /home/mydir/check_and_start_script.sh
Now in /home/mydir/check_and_start_script.sh,
#!/bin/bash
pid_file='/home/mydir/script.pid'
if [ ! -s "$pid_file" ] || ! kill -0 $(cat $pid_file) > /dev/null 2>&1; then
echo $$ > "$pid_file"
exec /usr/bin/python2.7 /home/mydir/public_html/myotherdir/script.py
fi
This checks to see if there’s a file with the process id of the last run of the script. If it’s there, it reads the pid and checks if the process is still running. If not, then it puts the pid of the currently running shell in the file and executes the python script in the same process, terminating the shell. Otherwise it does nothing.
Don’t forget to make the script executable
chmod 755 /home/mydir/check_and_start_script.sh
There is now a better way to do this via utility called flock, directly from cron task within a single line. flock will acquire a lock before it runs your app and release it after it is run. The format is as follows:
* * * * * /usr/bin/flock -n /tmp/fcj.lockfile <your cron job>
For your case, this’d be:
* * * * * /usr/bin/flock -n /tmp/fcj.lockfile /usr/bin/python2.7 /home/mydir/public_html/myotherdir/script.py
-n tells it to fail immediately if it cannot acquire the lock. There is also -w <seconds> option, it will fail if it cannot acquire the lock within given time frame.
* * * * /usr/bin/flock -w <seconds> /tmp/fcj.lockfile <your cron job>
In your crontab file, have this code:
command='/usr/bin/python2.7 /home/mydir/public_html/myotherdir/script.py'
*/1 * * * * pgrep -af "$command" || $command
Explanation:
Every minute, pgrep checks whether the python script is already running. If it is running, the 2nd statement will not be called since we used ||. If pgrep fails to find the python script in the existing processes, the 2nd statement will be executed and the python script will be back on work
I have a Python script that I’d like to run from a cronjob and then check every minute to see if it is still running and if not then start it again.
Cronjob is:
/usr/bin/python2.7 /home/mydir/public_html/myotherdir/script.py
there is some info on this but most answers don’t really detail the full process clearly, e.g.:
Using cron job to check if python script is running
e.g. in that case, it doesn’t state how to run the initial process and record the PID. It leaves me with a lot of questions unfortunately.
Therefore, could anyone give me a simple guide to how to do this?
e.g. full shell script required, what command to start the script, and so on.
You should actually take a step back and think what and why you are doing it
Do you need to actually start your long_running program from cron? The reason i ask is that you then write another program(watcher) that starts your long_running program. I hope that you see that yo actually do not need cron to start your long_running program.
So what you have is a watcher firing every minute and starting long_running program if it is not running.
Now you need to actually understand what you are trying to do because there are many ways this can be done. If this is an exercise watcher can call ps and work out from there if it runs and start it.
in bash you can do
[ -z "$(ps -ef | grep -v grep | grep myprogname)" ] && { echo runit; }
you just replace myprogname with the name and run it with command that starts your long_running code on the back ground – perhaps
however you should use another tool. systemd, daemontools or others
It’s not that hard. First set up the crontab to run a checker every minute:
* * * * * /home/mydir/check_and_start_script.sh
Now in /home/mydir/check_and_start_script.sh,
#!/bin/bash
pid_file='/home/mydir/script.pid'
if [ ! -s "$pid_file" ] || ! kill -0 $(cat $pid_file) > /dev/null 2>&1; then
echo $$ > "$pid_file"
exec /usr/bin/python2.7 /home/mydir/public_html/myotherdir/script.py
fi
This checks to see if there’s a file with the process id of the last run of the script. If it’s there, it reads the pid and checks if the process is still running. If not, then it puts the pid of the currently running shell in the file and executes the python script in the same process, terminating the shell. Otherwise it does nothing.
Don’t forget to make the script executable
chmod 755 /home/mydir/check_and_start_script.sh
There is now a better way to do this via utility called flock, directly from cron task within a single line. flock will acquire a lock before it runs your app and release it after it is run. The format is as follows:
* * * * * /usr/bin/flock -n /tmp/fcj.lockfile <your cron job>
For your case, this’d be:
* * * * * /usr/bin/flock -n /tmp/fcj.lockfile /usr/bin/python2.7 /home/mydir/public_html/myotherdir/script.py
-n tells it to fail immediately if it cannot acquire the lock. There is also -w <seconds> option, it will fail if it cannot acquire the lock within given time frame.
* * * * /usr/bin/flock -w <seconds> /tmp/fcj.lockfile <your cron job>
In your crontab file, have this code:
command='/usr/bin/python2.7 /home/mydir/public_html/myotherdir/script.py'
*/1 * * * * pgrep -af "$command" || $command
Explanation:
Every minute, pgrep checks whether the python script is already running. If it is running, the 2nd statement will not be called since we used ||. If pgrep fails to find the python script in the existing processes, the 2nd statement will be executed and the python script will be back on work