How to create a SSH tunnel using Python and Paramiko?
Question:
I need to create tunneling to read information from a database. I use Paramiko, but I have not worked with tunneling yet. Please provide an example of a simple code that creates and closes a tunnel.
Answers:
I used paramiko
for some project I had a year ago, here is the part of my code where I connected with another computer/server and executed a simple python file:
import paramiko
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(hostname='...', username='...', password='...')
stdin, stdout, stderr = ssh.exec_command('python hello.py')
ssh.close()
stdin
, stdout
and sdterr
contain the inputs/outputs of the command you executed.
From here, I think you can make the connection with the database.
At work we usually create ssh tunnels forwarding ports. The way we do that is, by using the standard command ssh -L port:addr:port addr
with subprocess running in a separate thread.
I found this useful link: https://github.com/paramiko/paramiko/blob/master/demos/forward.py with an example of doing port forwarding with paramiko.
I used sshtunnel
for my projects. Example of the forwarding remote local MySQL port to the host local port:
pip install sshtunnel
python -m sshtunnel -U root -P password -L :3306 -R 127.0.0.1:3306 -p 2222 localhost
Even though this does not use paramiko, I believe it’s a very clean solution to implement (similar to @dario’s answer but without managing the thread in python).
There’s this little-mentioned feature in openssh client that allows us to control a ssh process through a unix socket, quoting man ssh
:
-M Places the ssh client into “master” mode for connection sharing. Multiple -M options places ssh
into “master” mode with confirmation required before slave connections are accepted. Refer to the
description of ControlMaster in ssh_config(5) for details.
-S ctl_path
Specifies the location of a control socket for connection sharing, or the string “none” to disable
connection sharing. Refer to the description of ControlPath and ControlMaster in ssh_config(5)
for details.
So you can start background process of ssh
(with -Nf
) and then check (or terminate) it with a another ssh
call.
I use this in a project that requires a reverse tunnel to be established
from subprocess import call, STDOUT
import os
DEVNULL = open(os.devnull, 'wb')
CONFIG = dict(
SSH_SERVER='ssh.server.com',
SSH_PORT=2222,
SSH_USER='myuser',
SSH_KEY='/path/to/user.key',
REMOTE_PORT=62222,
UNIX_SOCKET='/tmp/ssh_tunnel.sock',
KNOWN_HOSTS='/path/to/specific_known_host_to_conflicts',
)
def start():
return call(
[
'ssh', CONFIG['SSH_SERVER'],
'-Nfi', CONFIG['SSH_KEY'],
'-MS', CONFIG['UNIX_SOCKET'],
'-o', 'UserKnownHostsFile=%s' % CONFIG['KNOWN_HOSTS'],
'-o', 'ExitOnForwardFailure=yes',
'-p', str(CONFIG['SSH_PORT']),
'-l', CONFIG['SSH_USER'],
'-R', '%d:localhost:22' % CONFIG['REMOTE_PORT']
],
stdout=DEVNULL,
stderr=STDOUT
) == 0
def stop():
return __control_ssh('exit') == 0
def status():
return __control_ssh('check') == 0
def __control_ssh(command):
return call(
['ssh', '-S', CONFIG['UNIX_SOCKET'], '-O', command, 'x'],
stdout=DEVNULL,
stderr=STDOUT
)
-o ExitOnForwardFailure=yes
makes sure the ssh command will fail if the tunnel cannot be established, otherwise it will not exit.
Might I suggest trying something like pyngrok
to programmatically manage an ngrok
tunnel for you? Full disclosure, I am the developer of it. SSH example here, but it’s as easy as installing pyngrok
:
pip install pyngrok
and using it:
from pyngrok import ngrok
# <NgrokTunnel: "tcp://0.tcp.ngrok.io:12345" -> "localhost:22">
ssh_tunnel = ngrok.connect(22, "tcp")
I am adding this solution for multiple hops on multiple ports:
I have this setup:
the goal is to access the database by calling my machine on port 33306. This isn’t possible because only gateway2 is allowed to speak to the database. We cant access gateway2 because only gateway1 is allowed to speak to it.
the following is the corresponding ssh .config file:
Host gateway1
HostName gtw1_IP_address
User gtw1_user
IdentityFile "path_to_gtw1_ssh_key"
IdentitiesOnly True
Host gateway2
User gtw2_user
Hostname gtw2_IP_address
IdentityFile "path_to_gtw2_ssh_key"
IdentitiesOnly True
# mysql
LocalForward 127.0.0.1:33306 127.0.0.1:3306
ProxyCommand ssh -W %h:%p switch-cede
this is how I reproduce it in python:
from sqlalchemy import create_engine
import config
import pandas as pd
import sshtunnel
from paramiko import SSHClient
with sshtunnel.open_tunnel(
ssh_username='gtw1_user',
ssh_address_or_host=('gtw1_IP_address', 22),
remote_bind_addresses=[('gtw2_IP_address', 22), ('gtw2_IP_address', 33306)],
local_bind_addresses=[('0.0.0.0', 22), ('0.0.0.0', 33306)], #this line is optional
ssh_pkey=path_to_gtw1_ssh_key,
) as tunnel1: # tunnel1 is the tunnel between myMachine and gateway1 I believe
print('Connection to tunnel1 (86.119.30.24:22) OK...')
print(tunnel1.local_bind_ports)
with sshtunnel.open_tunnel(
ssh_address_or_host=('localhost', tunnel1.local_bind_ports[0]),
remote_bind_addresses=[('127.0.0.1', 22),('127.0.0.1', 3306)],
local_bind_addresses=[('0.0.0.0', 22), ('127.0.0.1', 33306)],
ssh_username='gtw2_user',
ssh_pkey=path_to_gtw2_ssh_key,
) as tunnel2: # tunnel2 is the tunnel between gtw1 and gtw2 I believe
print('Connection to tunnel2 (192.168.142.140:22) OK...')
print(tunnel2.local_bind_ports)
db = create_engine(
f"mysql+pymysql://{config.USER}:{config.PASSWORD}@{config.HOST}:{config.PORT}/{config.DATABASE}")
print(db)
query = "SELECT * FROM randomTable LIMIT 10;"
df = pd.read_sql(query, db)
print(df)
# note that config is a file holding the credentials to connect to the database
I need to create tunneling to read information from a database. I use Paramiko, but I have not worked with tunneling yet. Please provide an example of a simple code that creates and closes a tunnel.
I used paramiko
for some project I had a year ago, here is the part of my code where I connected with another computer/server and executed a simple python file:
import paramiko
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(hostname='...', username='...', password='...')
stdin, stdout, stderr = ssh.exec_command('python hello.py')
ssh.close()
stdin
, stdout
and sdterr
contain the inputs/outputs of the command you executed.
From here, I think you can make the connection with the database.
At work we usually create ssh tunnels forwarding ports. The way we do that is, by using the standard command ssh -L port:addr:port addr
with subprocess running in a separate thread.
I found this useful link: https://github.com/paramiko/paramiko/blob/master/demos/forward.py with an example of doing port forwarding with paramiko.
I used sshtunnel
for my projects. Example of the forwarding remote local MySQL port to the host local port:
pip install sshtunnel
python -m sshtunnel -U root -P password -L :3306 -R 127.0.0.1:3306 -p 2222 localhost
Even though this does not use paramiko, I believe it’s a very clean solution to implement (similar to @dario’s answer but without managing the thread in python).
There’s this little-mentioned feature in openssh client that allows us to control a ssh process through a unix socket, quoting man ssh
:
-M Places the ssh client into “master” mode for connection sharing. Multiple -M options places ssh
into “master” mode with confirmation required before slave connections are accepted. Refer to the
description of ControlMaster in ssh_config(5) for details.
-S ctl_path
Specifies the location of a control socket for connection sharing, or the string “none” to disable
connection sharing. Refer to the description of ControlPath and ControlMaster in ssh_config(5)
for details.
So you can start background process of ssh
(with -Nf
) and then check (or terminate) it with a another ssh
call.
I use this in a project that requires a reverse tunnel to be established
from subprocess import call, STDOUT
import os
DEVNULL = open(os.devnull, 'wb')
CONFIG = dict(
SSH_SERVER='ssh.server.com',
SSH_PORT=2222,
SSH_USER='myuser',
SSH_KEY='/path/to/user.key',
REMOTE_PORT=62222,
UNIX_SOCKET='/tmp/ssh_tunnel.sock',
KNOWN_HOSTS='/path/to/specific_known_host_to_conflicts',
)
def start():
return call(
[
'ssh', CONFIG['SSH_SERVER'],
'-Nfi', CONFIG['SSH_KEY'],
'-MS', CONFIG['UNIX_SOCKET'],
'-o', 'UserKnownHostsFile=%s' % CONFIG['KNOWN_HOSTS'],
'-o', 'ExitOnForwardFailure=yes',
'-p', str(CONFIG['SSH_PORT']),
'-l', CONFIG['SSH_USER'],
'-R', '%d:localhost:22' % CONFIG['REMOTE_PORT']
],
stdout=DEVNULL,
stderr=STDOUT
) == 0
def stop():
return __control_ssh('exit') == 0
def status():
return __control_ssh('check') == 0
def __control_ssh(command):
return call(
['ssh', '-S', CONFIG['UNIX_SOCKET'], '-O', command, 'x'],
stdout=DEVNULL,
stderr=STDOUT
)
-o ExitOnForwardFailure=yes
makes sure the ssh command will fail if the tunnel cannot be established, otherwise it will not exit.
Might I suggest trying something like pyngrok
to programmatically manage an ngrok
tunnel for you? Full disclosure, I am the developer of it. SSH example here, but it’s as easy as installing pyngrok
:
pip install pyngrok
and using it:
from pyngrok import ngrok
# <NgrokTunnel: "tcp://0.tcp.ngrok.io:12345" -> "localhost:22">
ssh_tunnel = ngrok.connect(22, "tcp")
I am adding this solution for multiple hops on multiple ports:
I have this setup:
the goal is to access the database by calling my machine on port 33306. This isn’t possible because only gateway2 is allowed to speak to the database. We cant access gateway2 because only gateway1 is allowed to speak to it.
the following is the corresponding ssh .config file:
Host gateway1
HostName gtw1_IP_address
User gtw1_user
IdentityFile "path_to_gtw1_ssh_key"
IdentitiesOnly True
Host gateway2
User gtw2_user
Hostname gtw2_IP_address
IdentityFile "path_to_gtw2_ssh_key"
IdentitiesOnly True
# mysql
LocalForward 127.0.0.1:33306 127.0.0.1:3306
ProxyCommand ssh -W %h:%p switch-cede
this is how I reproduce it in python:
from sqlalchemy import create_engine
import config
import pandas as pd
import sshtunnel
from paramiko import SSHClient
with sshtunnel.open_tunnel(
ssh_username='gtw1_user',
ssh_address_or_host=('gtw1_IP_address', 22),
remote_bind_addresses=[('gtw2_IP_address', 22), ('gtw2_IP_address', 33306)],
local_bind_addresses=[('0.0.0.0', 22), ('0.0.0.0', 33306)], #this line is optional
ssh_pkey=path_to_gtw1_ssh_key,
) as tunnel1: # tunnel1 is the tunnel between myMachine and gateway1 I believe
print('Connection to tunnel1 (86.119.30.24:22) OK...')
print(tunnel1.local_bind_ports)
with sshtunnel.open_tunnel(
ssh_address_or_host=('localhost', tunnel1.local_bind_ports[0]),
remote_bind_addresses=[('127.0.0.1', 22),('127.0.0.1', 3306)],
local_bind_addresses=[('0.0.0.0', 22), ('127.0.0.1', 33306)],
ssh_username='gtw2_user',
ssh_pkey=path_to_gtw2_ssh_key,
) as tunnel2: # tunnel2 is the tunnel between gtw1 and gtw2 I believe
print('Connection to tunnel2 (192.168.142.140:22) OK...')
print(tunnel2.local_bind_ports)
db = create_engine(
f"mysql+pymysql://{config.USER}:{config.PASSWORD}@{config.HOST}:{config.PORT}/{config.DATABASE}")
print(db)
query = "SELECT * FROM randomTable LIMIT 10;"
df = pd.read_sql(query, db)
print(df)
# note that config is a file holding the credentials to connect to the database