Python Subprocess.run() does it auto close on completion?

Question

I apologize if this is a dumb question, however, I am not very fluent in Python yet.

In regards to the Python Subprocess function…

I’ve seen that when you use sp = subprocess.Popen(...) people close/terminate it when it’s finished running the command. Example:

sp = subprocess.Popen(['powershell.exe', '-ExecutionPolicy', 'Unrestricted', 'cp', '-r', 'ui', f'..\{name}'], stdout=subprocess.PIPE, stderr=subprocess.PIPE, cwd='UI Boiler')
        sp.wait()
        sp.terminate()

However, my question is, do you need to close any subprocess.run() functions? Or do those processes close automatically once they are finished running their commands?

The project I am working on requires a lot of those to be run and I do not wish to have 10+ shells/powershells/processes open because I didn’t close them.

Asked By: Adzeiros

||

Source

Answer 1

**Yes, on both windows and posix implementations, subprocess.run() as well as subprocess.call() will both block until completion e.g. via Process.wait() internally. Since this is a blocking call it will wait until process completion to return, so you should not need to do anything special to close processes.

To wit, here’s the relevant snippets from subprocess source in cpython-3.10 (amended for brevity):

def call(*popenargs, timeout=None, **kwargs):
    """..."""
    with Popen(*popenargs, **kwargs) as p:
        try:
            return p.wait(timeout=timeout)
        except:  # Including KeyboardInterrupt, wait handled that.
            p.kill()
            raise

# ...

def run(*popenargs,
        input=None, capture_output=False, timeout=None, check=False, **kwargs):
    """..."""
    # ...
    with Popen(*popenargs, **kwargs) as process:
        # communicate (as well as the with statement will both wait() internally
        try:
            stdout, stderr = process.communicate(input, timeout=timeout)
        except TimeoutExpired as exc:
            process.kill()
            # ... additional handling here
            raise
        except:  # Including KeyboardInterrupt, communicate handled that.
            process.kill()
            # We don't call process.wait() as .__exit__ does that for us.
            raise
        retcode = process.poll()
        if check and retcode:
            raise CalledProcessError(retcode, process.args,
                                     output=stdout, stderr=stderr)
    return CompletedProcess(process.args, retcode, stdout, stderr)

If however you want to have more control over if and when the subprocess blocks, e.g. such that you can run other code on the same thread while the other process is running, then you should use the internal sp = supbrocess.Popen() directly

As to the call to terminate() – note this would always be a no-op in your example of waiting first and then terminating without a catch. Reason being, terminate as implemented will never even bother sending a TERM signal to your subprocess because wait() is a blocking call that will not exit until the child process completes or throws an exception (e.g. on timeout). Again, if you are calling subprocesses that might hang and you want to run in the background, e.g. so you can terminate yourself if it hasn’t completed after a certain amount of time e.g. you will need to manage the subprocess yourself and subprocess.run() is probably not suitable for your needs.

A note on terminate(): subprocess.run and subprocess.call do both properly support automatically sending a kill to an erroring or timed out process, so if that’s all you need, you can stick with one of those. In fact, on windows, kill() and terminate() are identical. On posix, a SIGKILL will be sent if the subprocess throws or times out.
If on POSIX, you would want to send a SIGTERM instead so that you give the subprocess the opportunity to try to terminate gracefully or cleanup then again it’s best to interact with the Process object directly via Popen

Answered By: lemonhead

Python Subprocess.run() does it auto close on completion?

Question:

Answers: