How do I pass a string into subprocess.Popen (using the stdin argument)?

I figured out this workaround:

>>> p = subprocess.Popen(['grep','f'],stdout=subprocess.PIPE,stdin=subprocess.PIPE)
>>> p.stdin.write(b'onentwonthreenfournfivensixn') #expects a bytes type object
>>> p.communicate()[0]
'fournfiven'
>>> p.stdin.close()

Is there a better one?

Apparently a cStringIO.StringIO object doesn’t quack close enough to
a file duck to suit subprocess.Popen

I’m afraid not. The pipe is a low-level OS concept, so it absolutely requires a file object that is represented by an OS-level file descriptor. Your workaround is the right one.

p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)    
p.stdin.write('onen')
time.sleep(0.5)
p.stdin.write('twon')
time.sleep(0.5)
p.stdin.write('threen')
time.sleep(0.5)
testresult = p.communicate()[0]
time.sleep(0.5)
print(testresult)

from subprocess import Popen, PIPE
from tempfile import SpooledTemporaryFile as tempfile
f = tempfile()
f.write('onentwonthreenfournfivensixn')
f.seek(0)
print Popen(['/bin/grep','f'],stdout=PIPE,stdin=f).stdout.read()
f.close()

"""
Ex: Dialog (2-way) with a Popen()
"""

p = subprocess.Popen('Your Command Here',
                 stdout=subprocess.PIPE,
                 stderr=subprocess.STDOUT,
                 stdin=PIPE,
                 shell=True,
                 bufsize=0)
p.stdin.write('STARTn')
out = p.stdout.readline()
while out:
  line = out
  line = line.rstrip("n")

  if "WHATEVER1" in line:
      pr = 1
      p.stdin.write('DO 1n')
      out = p.stdout.readline()
      continue

  if "WHATEVER2" in line:
      pr = 2
      p.stdin.write('DO 2n')
      out = p.stdout.readline()
      continue
"""
..........
"""

out = p.stdout.readline()

p.wait()

Beware that Popen.communicate(input=s)may give you trouble ifsis too big, because apparently the parent process will buffer it before forking the child subprocess, meaning it needs “twice as much” used memory at that point (at least according to the “under the hood” explanation and linked documentation found here). In my particular case,swas a generator that was first fully expanded and only then written tostdin so the parent process was huge right before the child was spawned,
and no memory was left to fork it:

File "/opt/local/stow/python-2.7.2/lib/python2.7/subprocess.py", line 1130, in _execute_child
self.pid = os.fork()
OSError: [Errno 12] Cannot allocate memory

I am using python3 and found out that you need to encode your string before you can pass it into stdin:

p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=PIPE)
out, err = p.communicate(input='onentwonthreenfournfivensixn'.encode())
print(out)

I’m a bit surprised nobody suggested creating a pipe, which is in my opinion the far simplest way to pass a string to stdin of a subprocess:

read, write = os.pipe()
os.write(write, "stdin input here")
os.close(write)

subprocess.check_call(['your-command'], stdin=read)

There’s a beautiful solution if you’re using Python 3.4 or better. Use the input argument instead of the stdin argument, which accepts a bytes argument:

output_bytes = subprocess.check_output(
    ["sed", "s/foo/bar/"],
    input=b"foo",
)

This works for check_output and run, but not call or check_call for some reason.

In Python 3.7+, you can also add text=True to make check_output take a string as input and return a string (instead of bytes):

output_string = subprocess.check_output(
    ["sed", "s/foo/bar/"],
    input="foo",
    text=True,
)

On Python 3.7+ do this:

my_data = "whatever you wantnshould match this f"
subprocess.run(["grep", "f"], text=True, input=my_data)

and you’ll probably want to add capture_output=True to get the output of running the command as a string.

On older versions of Python, replace text=True with universal_newlines=True:

subprocess.run(["grep", "f"], universal_newlines=True, input=my_data)

This is overkill for grep, but through my journeys I’ve learned about the Linux command expect, and the python library pexpect

• expect: dialogue with interactive programs
• pexpect: Python module for spawning child applications; controlling them; and responding to expected patterns in their output.

import pexpect
child = pexpect.spawn('grep f', timeout=10)
child.sendline('text to match')
print(child.before)

Working with interactive shell applications like ftp is trivial with pexpect

import pexpect
child = pexpect.spawn ('ftp ftp.openbsd.org')
child.expect ('Name .*: ')
child.sendline ('anonymous')
child.expect ('Password:')
child.sendline ('[email protected]')
child.expect ('ftp> ')
child.sendline ('ls /pub/OpenBSD/')
child.expect ('ftp> ')
print child.before   # Print the result of the ls command.
child.interact()     # Give control of the child to the user.