How to show periodic report from queue in multiprocesssing process in Python

Question

I am using multiprocessing package in Python to start a run in a subprocess to dedicate a thread for the run so that the rest of the threads on the machine can be used for heavy computation. While the process is running, I would like to show the progress. Is there a way to access the report which is updated during the heavy computation? Here is a small example to illustrate what I would like to achieve.

import multiprocessing as _mp
def compute_squared(queue):
    number_squared = 20    
    report = {}
    def run(number_squared, report):
        for i in range(number_squared):
            j = i ** 2
            report[i] = i
   
    run(number_squared, report)
    queue.put({
        "report": report,
        "name": "compute squared"
    })

queue = _mp.Queue()
run_process = _mp.Process(
                    target=compute_squared, 
                    args=(queue, ))
run_process.start()
while run_process.is_alive():
    continue
    
final_report = queue.get()["report"]

While run_process.is_alive() I want to print the report from inside compute_squared so that I can trace the progress. I can access the final report using queue.get() but is there a way to access the intermediate reports?

Asked By: Gaurav

||

Source

Answer 1

import multiprocessing as _mp
def compute_squared(queue):
    number_squared = 20    
    report = {}
    def run(number_squared, report):
        for i in range(number_squared):
            j = i ** 2
            report[i] = i
            queue.put(report)  # Put the updated report in the queue

    run(number_squared, report)
    queue.put({
        "report": report,
        "name": "compute squared"
    })

queue = _mp.Queue()
run_process = _mp.Process(
                    target=compute_squared, 
                    args=(queue, ))
run_process.start()
while run_process.is_alive():
    # Check for updates to the report and print them
    if not queue.empty():
        report = queue.get()
        print(report)
    
final_report = queue.get()["report"]

Answered By: Raya

How to show periodic report from queue in multiprocesssing process in Python

Question:

Answers: