Ways of supress prints from joblib parallelization

Question

I have a script that contains parallelization. Can I suppress the parallelization prints with a simple contextlib suppressor or do I have to initialize a mute in the subprocesses with.

from joblib import Parallel, delayed
import os
import contextlib
def start(): 
    print('HELLO')
    def para(i): 
        print('hello')
        
    a = Parallel(n_jobs = 2)(delayed(para)(i) for i in [0,1,2])

with contextlib.redirect_stdout(open(os.devnull, "w")):            
    start()    
    
Output: 
    'hello'
    'hello'
    'hello'

Is there any way to suppress the parallelization content as well, without having to initialize it in the function it self?

Asked By: Noob Programmer

||

Source

Answer 1

The contextlib.redirect_stdout() does not redirect the native file descriptor of the standard output stream, which is inherited by a child process. So, you need to do like the following, using the os.dup2().

import sys
import os
import contextlib

@contextlib.contextmanager
def redirect_stdout_fd(file):
    stdout_fd = sys.stdout.fileno()
    stdout_fd_dup = os.dup(stdout_fd)
    os.dup2(file.fileno(), stdout_fd)
    file.close()
    try:
        yield
    finally:
        os.dup2(stdout_fd_dup, stdout_fd)
        os.close(stdout_fd_dup)

...
with redirect_stdout_fd(open(os.devnull, "w")):            
    start()

Answered By: relent95

Ways of supress prints from joblib parallelization

Question:

Answers: