How do I wait for to finish


I have the following code, which has been simplified:

import concurrent.futures

pool = concurrent.futures.ThreadPoolExecutor(8)

def _exec(x):
    return x + x

myfuturelist =,[x for x in range(5)])

# How do I wait for my futures to finish?

for result in myfuturelist:
    # Is this how it's done?

#... stuff that should happen only after myfuturelist is
#completely resolved.
# Documentation says is asynchronous

The documentation is weak regarding Help would be great.


Asked By: Dr.Knowitall



Difference between map and submit will run jobs in parallel and wait futures to finish, collect results and return a generator. It has done the wait for you. If you set a timeout, it will wait until timeout and throw exception in generator.

map(func, *iterables, timeout=None, chunksize=1)

  • the iterables are collected immediately rather than lazily;
  • func is executed asynchronously and several calls to func may be made concurrently.

To get a list of futures and do the wait manually, you can use:

myfuturelist = [pool.submit(_exec, x) for x in range(5)]

Executor.submit will return a future object, call result on future will explicitly wait for it to finish:

myfuturelist[0].result() # wait the 1st future to finish and return the result

EDIT 2023-02-24

Although original answer is accepted, plz check mway’s and milkice’s. I’ll try to add some detail here.

wait is the better way, and it lets you control how to wait the future by parameter return_when:

  • FIRST_COMPLETED, wait until the first finishes
  • FIRST_EXCEPTION, wait until the first raises exception or all finish
  • ALL_COMPLETED, wait until all finish

It returns a tuple of finished futures and unfinished ones:

# wait first one to finish
finished_set, unfinished_set = wait(myfuturelist, return_when=FIRST_COMPLETED)
# wait all 
wait(myfuturelist, return_when=ALL_COMPLETED)

Using with is elegant, but notice that:

  • you don’t have access to those return values directly (you can workaround though, for example a nonlocal or global variable)
  • you need to close the pool, which means you can’t reuse it to save the cost of thread creation and destroy.
Answered By: CtheSky

The call to does not block until all of its tasks are complete. Use wait to do this.

from concurrent.futures import wait, ALL_COMPLETED

futures = [pool.submit(fn, args) for args in arg_list]
wait(futures, timeout=whatever, return_when=ALL_COMPLETED)  # ALL_COMPLETED is actually the default

You could also call list(results) on the generator returned by to force the evaluation (which is what you’re doing in your original example). If you’re not actually using the values returned from the tasks, though, wait is the way to go.

Answered By: mway

It’s true that will not wait for all futures to finish. Because it returns a lazy iterator like @MisterMiyagi said.

But we can accomplish this by using with:

import time

from concurrent.futures import ThreadPoolExecutor

def hello(i):

with ThreadPoolExecutor(max_workers=2) as executor:, [1, 2, 3])

# output
# 1
# 2
# 3
# finish

As you can see, finish is printed after 1,2,3. It works because Executor has a __exit__() method, code is

def __exit__(self, exc_type, exc_val, exc_tb):
    return False

the shutdown method of ThreadPoolExecutor is

def shutdown(self, wait=True, *, cancel_futures=False):
    with self._shutdown_lock:
        self._shutdown = True
        if cancel_futures:
            # Drain all work items from the queue, and then cancel their
            # associated futures.
            while True:
                    work_item = self._work_queue.get_nowait()
                except queue.Empty:
                if work_item is not None:

        # Send a wake-up to prevent threads calling
        # _work_queue.get(block=True) from permanently blocking.
    if wait:
        for t in self._threads:
shutdown.__doc__ = _base.Executor.shutdown.__doc__

So by using with, we can get the ability to wait until all futures finish.

Answered By: milkice