How to install Python packages over SSH Port Forwarding?
Question:
I am controlling a remote unit over SSH and OPENVPN.
On the remote unit I want to install some Python packages using pip
but:
- the remote company firewall allows only traffic on port 22 (and not 443, needed by
pip
);
- DNS is not installed on the remote unit;
- I cannot modify any OPENVPN settings (or I would like to avoid this option as it means to access some remote sysadmin and try to convince him that the configuration must be changed);
- all systems are Linux (Ubuntu + Debian). Non Windows involved.
Stripping down hours of attempts (I am not a system admin and my knowledge on this subject is very limited), the idea was to open an obvious SSH port forwarding:
ssh -R 9999:pypi.python.org:443 [email protected]
and then, on the remote unit play with pip install
:
pip install pymodbus==1.3.2 --proxy localhost:9999
But this command returns:
Cannot fetch index base URL https://pypi.python.org/simple/
Could not find any downloads that satisfy the requirement pymodbus==1.3.2
/root/.pip/pip.log
is:
Getting page https://pypi.python.org/simple/pymodbus/
Could not fetch URL https://pypi.python.org/simple/pymodbus/: connection error: ('Connection aborted.', BadStatusLine("''",))
Will skip URL https://pypi.python.org/simple/pymodbus/ when looking for download links for pymodbus==1.3.2
Getting page https://pypi.python.org/simple/
Could not fetch URL https://pypi.python.org/simple/: connection error: ('Connection aborted.', BadStatusLine("''",))
Will skip URL https://pypi.python.org/simple/ when looking for download links for pymodbus==1.3.2
Cannot fetch index base URL https://pypi.python.org/simple/
URLs to search for versions for pymodbus==1.3.2:
* https://pypi.python.org/simple/pymodbus/1.3.2
* https://pypi.python.org/simple/pymodbus/
Getting page https://pypi.python.org/simple/pymodbus/1.3.2
Could not fetch URL https://pypi.python.org/simple/pymodbus/1.3.2: connection error: ('Connection aborted.', BadStatusLine("''",))
Will skip URL https://pypi.python.org/simple/pymodbus/1.3.2 when looking for download links for pymodbus==1.3.2
Getting page https://pypi.python.org/simple/pymodbus/
It is obvious the remote unit cannot read the index page on pypi.pthon.org because the connection is refused.
What is the correct syntax for what I am trying to achieve?
Answers:
Proxy is going to be tricky. I suggest that you scp the pip module source file and install it locally from source. Use
pip install package —download=”/pth/to/downloaded/file” to get the package, scp it to the dest server and use pip install “/pth/to/scp/file”
It’s look like my problem. after exploration, I have found a solution.
And because in my region, pypi.python.org is slow, so I change my pip.conf and use pypi.douban.com/simple, as my index-url. this website use http protocol. so in my solution. I use 80 port as my target port.
Problem:
I have two host. host1 could connect Pypi.douban.com. and host2 couldn’t.
but I can connect host2 in host1 through ssh.
so in host2, I open a tmux session and open a ssh tunnel by local port forwarding(not remote port forwarding):
ssh -L 9999:pypi.douban.com:80 username@host1
after this redirect, I can use
pip install scikit-learn --proxy localhost:9999
to install package in host2.
I would recommend using a reverse dynamic proxy. In your case, try:
- On your local machine
ssh -R 9999 username@remotehostname
- On the remote host
pip install packageName --proxy socks5:127.0.0.1:9999
For your reference:
https://unix.stackexchange.com/questions/179270/how-to-create-reverse-dynamic-ssh-port-forwarding
I am controlling a remote unit over SSH and OPENVPN.
On the remote unit I want to install some Python packages using pip
but:
- the remote company firewall allows only traffic on port 22 (and not 443, needed by
pip
); - DNS is not installed on the remote unit;
- I cannot modify any OPENVPN settings (or I would like to avoid this option as it means to access some remote sysadmin and try to convince him that the configuration must be changed);
- all systems are Linux (Ubuntu + Debian). Non Windows involved.
Stripping down hours of attempts (I am not a system admin and my knowledge on this subject is very limited), the idea was to open an obvious SSH port forwarding:
ssh -R 9999:pypi.python.org:443 [email protected]
and then, on the remote unit play with pip install
:
pip install pymodbus==1.3.2 --proxy localhost:9999
But this command returns:
Cannot fetch index base URL https://pypi.python.org/simple/
Could not find any downloads that satisfy the requirement pymodbus==1.3.2
/root/.pip/pip.log
is:
Getting page https://pypi.python.org/simple/pymodbus/
Could not fetch URL https://pypi.python.org/simple/pymodbus/: connection error: ('Connection aborted.', BadStatusLine("''",))
Will skip URL https://pypi.python.org/simple/pymodbus/ when looking for download links for pymodbus==1.3.2
Getting page https://pypi.python.org/simple/
Could not fetch URL https://pypi.python.org/simple/: connection error: ('Connection aborted.', BadStatusLine("''",))
Will skip URL https://pypi.python.org/simple/ when looking for download links for pymodbus==1.3.2
Cannot fetch index base URL https://pypi.python.org/simple/
URLs to search for versions for pymodbus==1.3.2:
* https://pypi.python.org/simple/pymodbus/1.3.2
* https://pypi.python.org/simple/pymodbus/
Getting page https://pypi.python.org/simple/pymodbus/1.3.2
Could not fetch URL https://pypi.python.org/simple/pymodbus/1.3.2: connection error: ('Connection aborted.', BadStatusLine("''",))
Will skip URL https://pypi.python.org/simple/pymodbus/1.3.2 when looking for download links for pymodbus==1.3.2
Getting page https://pypi.python.org/simple/pymodbus/
It is obvious the remote unit cannot read the index page on pypi.pthon.org because the connection is refused.
What is the correct syntax for what I am trying to achieve?
Proxy is going to be tricky. I suggest that you scp the pip module source file and install it locally from source. Use
pip install package —download=”/pth/to/downloaded/file” to get the package, scp it to the dest server and use pip install “/pth/to/scp/file”
It’s look like my problem. after exploration, I have found a solution.
And because in my region, pypi.python.org is slow, so I change my pip.conf and use pypi.douban.com/simple, as my index-url. this website use http protocol. so in my solution. I use 80 port as my target port.
Problem:
I have two host. host1 could connect Pypi.douban.com. and host2 couldn’t.
but I can connect host2 in host1 through ssh.
so in host2, I open a tmux session and open a ssh tunnel by local port forwarding(not remote port forwarding):
ssh -L 9999:pypi.douban.com:80 username@host1
after this redirect, I can use
pip install scikit-learn --proxy localhost:9999
to install package in host2.
I would recommend using a reverse dynamic proxy. In your case, try:
- On your local machine
ssh -R 9999 username@remotehostname
- On the remote host
pip install packageName --proxy socks5:127.0.0.1:9999
For your reference:
https://unix.stackexchange.com/questions/179270/how-to-create-reverse-dynamic-ssh-port-forwarding