python subprocess.Popen and ssh port forwarding in the background

Question

I need to have functionality that would essentially emulate user creating a port forwrding with ssh. So, essentially that should work like that:
– execute ssh -f -N -L 10000:gateway:11000 localhost
– if there is an output from that command, show to the user and pump user’s input as a response
– finish
The code I came up with below, almost does what I need:

    ssh_proc = Popen(['ssh', '-f', '-N', '-L', '10000:gateway:11000', 'localhost'], stdin=PIPE, stdout=PIPE)
stdoutdata, stderrdata = ssh_proc.communicate()

But the problem is it never finishes. I see the command is executed and forwarding tunnel is created but communicate() is still stuck. I can break out of that with Ctrl+C and get this:

    ^CTraceback (most recent call last):
  File "sshman.py", line 278, in <module>
    add(args.remote_host, args.remote_port, args.local_port, args.local_host)
  File "sshman.py", line 125, in add
    stdoutdata, stderrdata = ssh_proc.communicate()
  File "/usr/lib64/python2.7/subprocess.py", line 740, in communicate
    return self._communicate(input)
  File "/usr/lib64/python2.7/subprocess.py", line 1256, in _communicate
    stdout, stderr = self._communicate_with_poll(input)
  File "/usr/lib64/python2.7/subprocess.py", line 1310, in _communicate_with_poll
    ready = poller.poll()
KeyboardInterrupt

Since i use -f with ssh command, it should fork the connection. Is there a way to interrupt communicate() when it is done or is there more elegant solution?

Thank you in advance for any advices/suggestions/comments.

Asked By: Sergey

||

Source

Answer 1

I guess the solution was pretty easy

ssh_proc = Popen(['ssh', '-f', '-N', '-L', '10000:gateway:11000', 'localhost'], stdin=PIPE, stdout=PIPE)
stat = ssh_proc.poll()
while stat == None:
    stat = ssh_proc.poll()

Answered By: Sergey

Answer 2

I had a relative problem when tried to set an ssh tunnel, communicate() would hang on forever, I have found that it reads from the stderr and stdout, so if your command didn’t write anything to the stderr for example, then it will hang because it’s waiting to read something from the stderr (fd = 2).
So as a solution you could use os.set_blocking(ssh_proc.stderr.fileno(), False):

# Start the process.
ssh_proc = Popen(...)
# Wait till it's done.
ssh_proc.wait()
# Disable block for read.
os.set_blocking(ssh_proc.stderr.fileno(), False)
os.set_blocking(ssh_proc.stdout.fileno(), False)
# Read stdout and stderr.
err = ssh_proc.stderr.readlines()
out = ssh_proc.stdout.readlines()
# Print stderr and stdout.
if out:
    out = out[0]
    print(out)
if err:
    err = err[0]
    print(err)
if cmd.returncode == 0:
    return 0

That way the communicate() function will not block the process, even if there is nothing to read from stdout or stderr.

Answered By: fflores

python subprocess.Popen and ssh port forwarding in the background

Question:

Answers: