Python – how to execute shell commands with pipe, but without 'shell=True'?

Question

I have a case to want to execute the following shell command in Python and get the output,

echo This_is_a_testing | grep -c test

I could use this python code to execute the above shell command in python,

>>> import subprocess
>>> subprocess.check_output("echo This_is_a_testing | grep -c test", shell=True)
'1n'

However, as I do not want to use the "shell=True" option, I tried the following python code,

>>> import subprocess
>>> p1 = subprocess.Popen(["echo", "This_is_a_testing"], stdout=subprocess.PIPE)
>>> p2 = subprocess.Popen(["grep", "-c", "test"], stdin=p1.stdout)
>>> p1.stdout.close()
>>> p2.communicate()
(None, None)

I wonder why the output is "None" as I have referred to the descriptions in the webpage : http://docs.python.org/library/subprocess.html#subprocess.PIPE

Had I missed some points in my code ? Any suggestion / idea ? Thanks in advance.

Asked By: user1129812

||

Source

Answer 1

From the manual:

to get anything other than None in the result tuple, you need to give
stdout=PIPE and/or stderr=PIPE

p2 = subprocess.Popen(["grep", "-c", "test"], stdin=p1.stdout, stdout=subprocess.PIPE)

Answered By: Karoly Horvath

Answer 2

Please look here:

>>> import subprocess
>>> p1 = subprocess.Popen(["echo", "This_is_a_testing"], stdout=subprocess.PIPE)
>>> p2 = subprocess.Popen(["grep", "-c", "test"], stdin=p1.stdout)
>>> 1
p1.stdout.close()
>>> p2.communicate()
(None, None)
>>>

here you get 1 as output after you write p2 = subprocess.Popen(["grep", "-c", "test"], stdin=p1.stdout), Do not ignore this output in the context of your question.

If this is what you want, then pass stdout=subprocess.PIPE as argument to the second Popen:

>>> p1 = subprocess.Popen(["echo", "This_is_a_testing"], stdout=subprocess.PIPE)
>>> p2 = subprocess.Popen(["grep", "test"], stdin=p1.stdout, stdout=subprocess.PIPE)
>>> p2.communicate()
('This_is_a_testingn', None)
>>>

Answered By: avasal

Answer 3

>>> import subprocess

>>> mycmd=subprocess.getoutput('df -h | grep home | gawk '{ print $1 }' | cut -d'/' -f3')

>>> mycmd 

'sda6'

>>>

Answered By: Дмитрий Голуб

Answer 4

While the accepted answer is correct/working, another option would be to use the Popen.communicate() method to pass something to a process’ stdin:

>>> import subprocess
>>> p2 = subprocess.Popen(["grep", "-c", "test"], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
>>> p2.communicate("This_is_a_testing")
('1n', None)
>>> print p2.returncode
0
>>>>

This resolves the need to execute another command just to redirect it’s output, if the output is already known in the python script itself.

However communicate has the side-effect, that it waits for the process to terminate. If asynchronous execution is needed/desired using two processes might be the better option.

Answered By: dpr

Answer 5

Answer is similar to mentioned earlier, with little formatting. I wanted to get exactly same output as normal shell command with pipe on python 3.

import subprocess

p1 = subprocess.Popen(["ls", "-l", "."], stdout=subprocess.PIPE)
p2 = subprocess.Popen(["grep", "May"], stdin=p1.stdout, stdout=subprocess.PIPE)


for s in (str(p2.communicate())[2:-10]).split('\n'):
    print(s)

Answered By: Naveen Singh

Python – how to execute shell commands with pipe, but without 'shell=True'?

Question:

Answers: