Running script every hour – deviating by few second/minutes

Question

I am running a script every hour (timer at 3600 seconds). But with every run the timestamp of my export file (result of my script) is later than previous one. Its almost impossible to find the sweet spot and stick to approx. the same export time. Any suggestions to have a more stable and reliable timer?

Code:

import pandas as pd
from datetime import datetime as dt
import schedule
import time

def export():
    df = pd.read_excel("filepath", sheet_name='Import Data')
    now =dt.now()
    dt_string = now.strftime("%Y%m%d %H%M%S")
    df["Minute"] = now.minute
    df["Hour"] = now.hour
    df["Day"] = now.day
    df["Month"] = now.month
    df["Year"] = now.year
    df.to_excel("filepath" + dt_string + ".xlsx", sheet_name='Export Data')

schedule.every(3600).seconds.do(export)

while 1:
    schedule.run_pending()
    time.sleep(1)

Asked By: Prmake

||

Source

Answer 1

The way to solve this is to adjust the run times based on the real clock. By real clock, I mean e.g., the UTC time, or whatever time you have on your system. The time zone is not important. What’s important is to do the scheduling based on what time it is, not based on what amount of time has elapsed since the last run.

The problem with measuring the elapsed time is that you need to consider the time it takes to take the measurements, and there can be other factors that cause delay too.

Solution 1

This was my original idea before I realised the schedule library can do this too. You may want to jump straight to Soluition 2 below, that’s simpler.

Below is an example code. It runs a method every 5 seconds. This way, the delay will not shift over time, the timing will remain constant.

You will need to adjust this to your needs. You can modify it, so it runs every hour instead of every 5 seconds, the idea is the same.

Example code:

import time
from datetime import datetime as dt
import threading
from threading import Event

def export():
    print("Do something every 5 seconds.")


run_once = Event()
run_once.clear()


def run_method(method_to_run):
    while 1:
        run_once.wait()
        run_once.clear()
        method_to_run()

thread = threading.Thread(target=run_method, args=(export,))
thread.start()

last_run = 0

while 1:
    when_to_run = int(dt.now().second / 5)
    if when_to_run > last_run:
        last_run = when_to_run
        run_once.set()
    elif when_to_run < last_run:
        last_run = when_to_run
    time.sleep(1)

Edit 1: If you want to run the task every hour, then you need to modify the following line in the above example:

    # when_to_run = int(dt.now().second / 5)
    when_to_run = int(dt.now().hour)

Edit 2: Adding Solution 2.

Solution 2

Using the schedule library’s .at() method. (Thanks @itprorh66 for the comment.)

Documentation: https://schedule.readthedocs.io/en/stable/reference.html#schedule.Job.at

Example code:

import time
import schedule


def export():
    print("Do something every 5 seconds.")


schedule.every().hour.at(":00").do(export)

while 1:
    schedule.run_pending()
    time.sleep(1)

Answered By: Tamas K

Running script every hour – deviating by few second/minutes

Question:

Answers:

Solution 1

Example code:

Solution 2

Example code: