Pipe subprocess standard output to a variable

Question

I want to run a command in pythong, using the subprocess module, and store the output in a variable. However, I do not want the command’s output to be printed to the terminal.
For this code:

def storels():
   a = subprocess.Popen("ls",shell=True)
storels()

I get the directory listing in the terminal, instead of having it stored in a. I’ve also tried:

 def storels():
       subprocess.Popen("ls > tmp",shell=True)
       a = open("./tmp")
       [Rest of Code]
 storels()

This also prints the output of ls to my terminal. I’ve even tried this command with the somewhat dated os.system method, since running ls > tmp in the terminal doesn’t print ls to the terminal at all, but stores it in tmp. However, the same thing happens.

Edit:

I get the following error after following marcog’s advice, but only when running a more complex command. cdrecord --help. Python spits this out:

Traceback (most recent call last):
  File "./install.py", line 52, in <module>
    burntrack2("hi")
  File "./install.py", line 46, in burntrack2
    a = subprocess.Popen("cdrecord --help",stdout = subprocess.PIPE)
  File "/usr/lib/python2.6/subprocess.py", line 633, in __init__
    errread, errwrite)
  File "/usr/lib/python2.6/subprocess.py", line 1139, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

Asked By: Insomaniacal

||

Source

Answer 1

To get the output of ls, use stdout=subprocess.PIPE.

>>> proc = subprocess.Popen('ls', stdout=subprocess.PIPE)
>>> output = proc.stdout.read()
>>> print output
bar
baz
foo

The command cdrecord --help outputs to stderr, so you need to pipe that indstead. You should also break up the command into a list of tokens as I’ve done below, or the alternative is to pass the shell=True argument but this fires up a fully-blown shell which can be dangerous if you don’t control the contents of the command string.

>>> proc = subprocess.Popen(['cdrecord', '--help'], stderr=subprocess.PIPE)
>>> output = proc.stderr.read()
>>> print output
Usage: wodim [options] track1...trackn
Options:
    -version    print version information and exit
    dev=target  SCSI target to use as CD/DVD-Recorder
    gracetime=# set the grace time before starting to write to #.
...

If you have a command that outputs to both stdout and stderr and you want to merge them, you can do that by piping stderr to stdout and then catching stdout.

subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)

As mentioned by Chris Morgan, you should be using proc.communicate() instead of proc.read().

>>> proc = subprocess.Popen(['cdrecord', '--help'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
>>> out, err = proc.communicate()
>>> print 'stdout:', out
stdout: 
>>> print 'stderr:', err
stderr:Usage: wodim [options] track1...trackn
Options:
    -version    print version information and exit
    dev=target  SCSI target to use as CD/DVD-Recorder
    gracetime=# set the grace time before starting to write to #.
...

Answered By: moinudin

Answer 2

With a = subprocess.Popen("cdrecord --help",stdout = subprocess.PIPE), you need to either use a list or use shell=True;

Either of these will work. The former is preferable.

a = subprocess.Popen(['cdrecord', '--help'], stdout=subprocess.PIPE)

a = subprocess.Popen('cdrecord --help', shell=True, stdout=subprocess.PIPE)

Also, instead of using Popen.stdout.read/Popen.stderr.read, you should use .communicate() (refer to the subprocess documentation for why).

proc = subprocess.Popen(['cdrecord', '--help'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = proc.communicate()

Answered By: Chris Morgan

Answer 3

If you are using python 2.7 or later, the easiest way to do this is to use the subprocess.check_output() command. Here is an example:

output = subprocess.check_output('ls')

To also redirect stderr you can use the following:

output = subprocess.check_output('ls', stderr=subprocess.STDOUT)

In the case that you want to pass parameters to the command, you can either use a list or use invoke a shell and use a single string.

output = subprocess.check_output(['ls', '-a'])
output = subprocess.check_output('ls -a', shell=True)

Answered By: amicitas

Pipe subprocess standard output to a variable

Question:

Answers: