In Python ProcessPoolExecutor, do you need call shutdown after getting a BrokenProcessPool exception?

Question

Say I have something like this:

pool = ProcessPoolExecutor(max_workers=1)

try:
    return pool.submit(
        do_something,
            ).result()
except concurrent.futures.process.BrokenProcessPool as pool_exc:
    pool = None
    return None

Is it a bad idea to just set pool = None do I need to call shutdown?
What args should I use with shutdown? pool.shutdown(wait=False, cancel_futures=True)?

Asked By: cozos

||

Source

Answer 1

You don’t have to call shutdown as that’s the responsibility of the pool.

When the pool object is deallocated (set to None) another thread is going to terminate and join the workers, which is similar to calling shutdown(wait=True) from another thread so no work is cancelled, but since the pool is already broken, there should be no workers to join.

Even if the pool wasn’t broken, most of the work will be done in another thread, so you don’t have to explicitly call shutdown(wait=False) if you are sure there are no references to the pool anywhere in your code.

Answered By: Ahmed AEK

In Python ProcessPoolExecutor, do you need call shutdown after getting a BrokenProcessPool exception?

Question:

Answers: