process-pool

In Python ProcessPoolExecutor, do you need call shutdown after getting a BrokenProcessPool exception? Question: In Python ProcessPoolExecutor, do you need call shutdown after getting a BrokenProcessPool exception? Say I have something like this: pool = ProcessPoolExecutor(max_workers=1) try: return pool.submit( do_something, ).result() except concurrent.futures.process.BrokenProcessPool as pool_exc: pool = None return None Is it a bad idea …

best way to speed up multiprocessing code in python? Question: I am trying to mess around with matrices in python, and wanted to use multiprocessing to processes each row separately for a math operation, I have posted a minimal reproducible sample below, but keep in mind that for my actual code I do in-fact need …

Working with ProcessPoolExecutor.(Windows) Where do all the classes and functions go, that have nothing to do with multiprocessing? Question: +++ I EDITED MY QUESTION, TO REFLECT WHAT I THINK IS THE CORRECT STRUCTURE NOW, so that not necessary things don’t get loaded in the child workers.. with the help of @booboo +++ THX Where do …

Starmap combined with tqdm? Question: I am doing some parallel processing, as follows: with mp.Pool(8) as tmpPool: results = tmpPool.starmap(my_function, inputs) where inputs look like: [(1,0.2312),(5,0.52) …] i.e., tuples of an int and a float. The code runs nicely, yet I cannot seem to wrap it around a loading bar (tqdm), such as can be …

Python multiprocessing.Pool: AttributeError Question: I have a method inside a class that needs to do a lot of work in a loop, and I would like to spread the work over all of my cores. I wrote the following code, which works if I use normal map(), but with pool.map() returns an error. import multiprocessing …