passing data to subprocess.check_output

Question

I want to invoke a script, piping the contents of a string to its stdin and retrieving its stdout.

I don’t want to touch the real filesystem so I can’t create real temporary files for it.

using subprocess.check_output I can get whatever the script writes; how can I get the input string into its stdin though?

subprocess.check_output([script_name,"-"],stdin="this is some input")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/subprocess.py", line 537, in check_output
    process = Popen(stdout=PIPE, *popenargs, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 672, in __init__
    errread, errwrite) = self._get_handles(stdin, stdout, stderr)
  File "/usr/lib/python2.7/subprocess.py", line 1043, in _get_handles
    p2cread = stdin.fileno()
AttributeError: 'str' object has no attribute 'fileno'

Asked By: Will

||

Source

Answer 1

Use Popen.communicate instead of subprocess.check_output.

from subprocess import Popen, PIPE

p = Popen([script_name, "-"], stdin=PIPE, stdout=PIPE, stderr=PIPE)
stdout, stderr = p.communicate("this is some input")

Answered By: Fred Foo

Answer 2

In Python 3.4 and newer, you can use the input keyword parameter to send input via STDIN when using subprocess.check_output()

Quoting from the standard library documentation for subprocess.check_output():

The input argument is passed to Popen.communicate() and thus to the
subprocess’s stdin. If used it must be a byte sequence, or a string if
universal_newlines=True. When used, the internal Popen object is
automatically created with stdin=PIPE, and the stdin argument may not
be used as well.

Example:

>>> subprocess.check_output(["sed", "-e", "s/foo/bar/"],
...                         input=b"when in the course of fooman eventsn")
b'when in the course of barman eventsn'
>>> 
>>> # To send and receive strings instead of bytes,
>>> # pass in universal_newlines=True
>>> subprocess.check_output(["sed", "-e", "s/foo/bar/"],
...                         universal_newlines=True,
...                         input="when in the course of fooman eventsn")
'when in the course of barman eventsn'

Answered By: gotgenes

Answer 3

Here’s a check_output backported version for python 2.7 with input.

from subprocess import (PIPE, Popen, CalledProcessError)

def check_output_input(*popenargs, **kwargs):
    """Run command with arguments and return its output as a byte string.

    If the exit code was non-zero it raises a CalledProcessError.  The
    CalledProcessError object will have the return code in the returncode
    attribute and output in the output attribute.

    The arguments are the same as for the Popen constructor.  Example:

    >>> check_output(["ls", "-l", "/dev/null"])
    'crw-rw-rw- 1 root root 1, 3 Oct 18  2007 /dev/nulln'

    The stdout argument is not allowed as it is used internally.
    To capture standard error in the result, use stderr=STDOUT.

    >>> check_output(["/bin/sh", "-c",
    ...               "ls -l non_existent_file ; exit 0"],
    ...              stderr=STDOUT)
    'ls: non_existent_file: No such file or directoryn'

    There is an additional optional argument, "input", allowing you to
    pass a string to the subprocess's stdin.  If you use this argument
    you may not also use the Popen constructor's "stdin" argument, as
    it too will be used internally.  Example:

    >>> check_output(["sed", "-e", "s/foo/bar/"],
    ...              input=b"when in the course of fooman eventsn")
    b'when in the course of barman eventsn'

    If universal_newlines=True is passed, the return value will be a
    string rather than bytes.

    """
    if 'stdout' in kwargs:
        raise ValueError('stdout argument not allowed, it will be overridden.')
    if 'input' in kwargs:
        if 'stdin' in kwargs:
            raise ValueError('stdin and input arguments may not both be used.')
        inputdata = kwargs['input']
        del kwargs['input']
        kwargs['stdin'] = PIPE
    else:
        inputdata = None
    process = Popen(*popenargs, stdout=PIPE, **kwargs)
    try:
        output, unused_err = process.communicate(inputdata)
    except:
        process.kill()
        process.wait()
        raise
    retcode = process.poll()
    if retcode:
        cmd = kwargs.get("args")
        if cmd is None:
            cmd = popenargs[0]
        raise CalledProcessError(retcode, cmd, output=output)
    return output

Answered By: Al Conrad

passing data to subprocess.check_output

Question:

Answers: