tqdm: extract time passed + time remaining?

Question

I have been going over the tqdm docs, but no matter where I look, I cannot find a method by which to extract the time passed and estimated time remaining fields (basically the center of the progress bar on each line: 00:00<00:02).

 0%|          | 0/200 [00:00<?, ?it/s]
  4%|▎         | 7/200 [00:00<00:02, 68.64it/s]
  8%|▊         | 16/200 [00:00<00:02, 72.87it/s]
 12%|█▎        | 25/200 [00:00<00:02, 77.15it/s]
 17%|█▋        | 34/200 [00:00<00:02, 79.79it/s]
 22%|██▏       | 43/200 [00:00<00:01, 79.91it/s]
 26%|██▌       | 52/200 [00:00<00:01, 80.23it/s]
 30%|███       | 61/200 [00:00<00:01, 82.13it/s]
....
100%|██████████| 200/200 [00:02<00:00, 81.22it/s]

tqdm works via essentially printing a dynamic progress bar anytime an update occurs, but is there a way to “just” print the 00:01 and 00:02 portions, so I could use them elsewhere in my Python program, such as in automatic stopping code that halts the process if it is taking too long?

Asked By: Coolio2654

||

Source

Answer 1

Edit: see the library maintainer’s answer below. Turns out, it is possible to get this information in the public API.

tqdm does not expose that information as part of its public API, and I don’t recommend trying to hack your own into it. Then you would be depending on implementation details of tqdm that might change at any time.

However, that shouldn’t stop you from writing your own. It’s easy enough to instrument a loop with a timer, and you can then abort the loop if it takes too long. Here’s a quick, rough example that still uses tqdm to provide visual feedback:

import time
from tqdm import tqdm


def long_running_function(n, timeout=5):
    start_time = time.time()

    for _ in tqdm(list(range(n))):
        time.sleep(1)  # doing some expensive work...
        elapsed_time = time.time() - start_time
        if elapsed_time > timeout:
            raise TimeoutError("long_running_function took too long!")


long_running_function(100, timeout=10)

If you run this, the function will stop its own execution after 10 seconds by raising an exception. You could catch this exception at the call site and respond to it in whatever way you deem appropriate.

If you want to be clever, you could even factor this out in a tqdm-like wrapper like this:

def timed_loop(iterator, timeout):
    start_time = time.time()
    iterator = iter(iterator)

    while True:
        elapsed_time = time.time() - start_time
        if elapsed_time > timeout:
            raise TimeoutError("long_running_function took too long!")

        try:
            yield next(iterator)
        except StopIteration:
            pass


def long_running_function(n, timeout=5):
    for _ in timed_loop(tqdm(list(range(n))), timeout=timeout):
        time.sleep(0.1)


long_running_function(100, timeout=5)

Answered By: Josh Karpel

Answer 2

tqdm objects expose some information via the public property format_dict.

from tqdm import tqdm
    
# for i in tqdm(iterable):
with tqdm(iterable) as t:
    for i in t:
        ...
        elapsed = t.format_dict['elapsed']
        elapsed_str = t.format_interval(elapsed)

Otherwise you could parse str(t).split()

Answered By: casper.dcl

Answer 3

You can get elapsed and remaining time from format_dict and some calculations.

t = tqdm(total=100)
...
elapsed = t.format_dict["elapsed"]
rate = t.format_dict["rate"]
remaining = (t.total - t.n) / rate if rate and t.total else 0  # Seconds*

Answered By: Artem Khodakov

Answer 4

Here’s the answer to the time remaining and time elapsed question:

from tqdm import tqdm
from time import sleep
    
with tqdm(total=100, bar_format="{l_bar}{bar} [ time left: {remaining}, time spent: {elapsed}]") as pbar:
        for i in loop:
            pbar.update(1)
            sleep(0.01)

If needed to be worked with or printed elsewhere:

elapsed = pbar.format_dict["elapsed"]
remains = pbar.format_dict["remaining"]

Answered By: Carlo Antonio Fernandez Benede

tqdm: extract time passed + time remaining?

Question:

Answers: