Forecast the remaining time of a loop

Question

Good Evening,

I am trying to estimate the remaining time to the end of a loop; I’ve used:

start = datetime.now()
progress = 0

for i in range(1000): 

    #do a few calculations

    progress += 1

    stop = datetime.now()
    execution_time = stop-start 

    remaining = execution_time * ( 1000 - progress )

    print("Progress:", progress, "%, estimated", remaining, "time remaining")

But it does not seem to work properly, since it goes up to minutes, even though the loop would take 20 seconds in total, and decrease quickly when reaching the end.

How can I try to forecast the remaining time of a loop efficiently and correctly?

Asked By: Alessandro Ceccarelli

||

Source

Answer 1

Your calculation for time remaining is wrong. If it takes execution_time for progress steps. Then how much does it take for 1000 steps ?

Simple cross multiply gives you the total time. Subtract it from the time already elapsed and that will give you the time remaining.

remaining_time = execution_time * 1000 / progress - execution_time
percent_complete =  (progress  / 1000) * 100     #You can simplify this if you like
print("Progress:", percent_complete , "%, Estimated", remaining_time, "time remaining")

Also your variable execution_time_1 is never defined

Answered By: DollarAkshay

Answer 2

Rather than using datetime.datetime.now() for this sort of thing you can use time.perf_counter(), which is available in Python 3.3+. From the docs:

Return the value (in fractional seconds) of a performance counter,
i.e. a clock with the highest available resolution to measure a short
duration. It does include time elapsed during sleep and is
system-wide. The reference point of the returned value is undefined,
so that only the difference between the results of consecutive calls
is valid.

Also, you can print using a carriage return instead of a newline so that the progress reports are printed on a single line. Here’s a brief demo derived from your code.

from time import sleep, perf_counter

fmt = "  Progress: {:>3}% estimated {:>3}s remaining"
num = 1000

start = perf_counter()
for i in range(1, num + 1):
    # Simulate doing a few calculations
    sleep(0.01)

    stop = perf_counter()
    remaining = round((stop - start) * (num / i - 1))
    print(fmt.format(100 * i // num, remaining), end='r')
print()

Depending on your terminal (and Python version) you may also need to add the flush=True keyword arg to the print call in order to get the progress reports to print as they are issued.

Answered By: PM 2Ring

Answer 3

I think that in this line:

remaining = execution_time * ( 1000 - progress )

you should divide execution_time/progress, because you want to know how long it takes to complete one percent of progress.

remaining = execution_time/progress * ( 1000 - progress )

Answered By: Igor

Answer 4

Simply use tqdm package:

from tqdm import tqdm
for i in tqdm(range(10000)):
    dosomthing()

It will print everything for you:

76%|█████████████           | 7568/10000 [00:33<00:10, 229.00it/s]

Answered By: Hadij

Forecast the remaining time of a loop

Question:

Answers: