pip: cert failed, but curl works
Question:
We installed the our root cert on the client, and the https connection works for curl.
But if we try to use pip, it fails:
Could not fetch URL https://installserver:40443/pypi/simple/pep8/:
There was a problem confirming the ssl certificate:
<urlopen error [Errno 1] _ssl.c:499: error:14090086:SSL
routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed>
The cert is on the client. See:
(foo_fm_qti)foo_fm_qti@vis-work:~$ curl -v https://installserver:40443/pypi/simple/pep8/
* About to connect() to installserver port 40443 (#0)
* Trying 127.0.0.1... connected
* Connected to installserver (127.0.0.1) port 40443 (#0)
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs/
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS alert, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using DHE-RSA-AES256-SHA
* Server certificate:
* subject: C=DE; ST=Sachsen; L=Chemnitz; O=FOO-COM GmbH; OU=DV; CN=gray.foo-com.lan; [email protected]
* start date: 2013-09-09 10:47:50 GMT
* expire date: 2019-05-24 10:47:50 GMT
* subjectAltName: installserver matched
* issuer: C=DE; ST=Sachsen; L=Chemnitz; O=FOO-COM GmbH; CN=FOO-COM Root CA; [email protected]
* SSL certificate verify ok.
> GET /pypi/simple/pep8/ HTTP/1.1
Version: pip 1.4.1
Answers:
Unfortunately pip does not use the system certs, but curl does.
I found a solution:
pip --cert /etc/ssl/certs/FOO_Root_CA.pem install pep8
This is not nice (curl and other libraries find the cert without adding a parameter) but works.
If you don’t want to use the command line argument, you can set the cert in ~/.pip/pip.conf:
[global]
cert = /etc/ssl/certs/Foo_Root_CA.pem
My solution is downloading cacert.pem from http://curl.haxx.se/ca/cacert.pem
and add the path for cacert.pem to ~/.pip/pip.conf as guettli suggested
[global]
cert = /path/to/cacert.pem
I use:
export PIP_CERT=`python -m pip._vendor.requests.certs`
pip install pep8
PIP always validates the certificate of HTTPS connections (and all pypi packages redirect to HTTPS).
The algorithm for determining the CA file is based on 3 steps:
- Look in a list of default locations for different linux distributions
(in my case this file turned out to be out of date, as I am building on a very old linux distribution)
- If available, override the value found in (1) from a value in the pip.conf file, the environment or the command-line (in that order),
- If both (1) and (2) did not result in a value, use a bundled file
Note that pip does not use the default SSL directories and files (from ssl.get_default_verify_paths()). But only supports a bundled CA file.
PIP does support a command-line action to list the bundled file from step 3 and that is what I use for this answer.
For me, none of the config-file workarounds worked. I’m using pip 1.5.4 on Ubuntu 14.04
The command posted by @arjenve didn’t work on my system either. I get: /usr/bin/python: No module named _vendor.requests
UPDATE
An even better solution than my first workaround is installing the certificate on the system first (for me on ubuntu this would be)
sudo cp ~/my_cert.crt /usr/local/share/ca-certificates/
sudo update-ca-certificates
The previous automatically updates the bundle file (checking at the bottom of /etc/ssl/certs/ca-certificates.crt you should now see the same certificate as in my_cert.crt)
Now export that path into PIP_CERT and add it to your .bashrc:
echo export PIP_CERT=/etc/ssl/certs/ca-certificates.crt >> ~/.bashrc
OLDER WORKAROUND
My workaround was to create a bundle file from /etc/ssl/certs/ca-certificates.crt and my corporate’s crt (just concatenated both files). And then export a variable (put that on my .bashrc) like this:
export PIP_CERT=/my/path/to/the/bundle.crt
We installed the our root cert on the client, and the https connection works for curl.
But if we try to use pip, it fails:
Could not fetch URL https://installserver:40443/pypi/simple/pep8/:
There was a problem confirming the ssl certificate:
<urlopen error [Errno 1] _ssl.c:499: error:14090086:SSL
routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed>
The cert is on the client. See:
(foo_fm_qti)foo_fm_qti@vis-work:~$ curl -v https://installserver:40443/pypi/simple/pep8/
* About to connect() to installserver port 40443 (#0)
* Trying 127.0.0.1... connected
* Connected to installserver (127.0.0.1) port 40443 (#0)
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs/
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS alert, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using DHE-RSA-AES256-SHA
* Server certificate:
* subject: C=DE; ST=Sachsen; L=Chemnitz; O=FOO-COM GmbH; OU=DV; CN=gray.foo-com.lan; [email protected]
* start date: 2013-09-09 10:47:50 GMT
* expire date: 2019-05-24 10:47:50 GMT
* subjectAltName: installserver matched
* issuer: C=DE; ST=Sachsen; L=Chemnitz; O=FOO-COM GmbH; CN=FOO-COM Root CA; [email protected]
* SSL certificate verify ok.
> GET /pypi/simple/pep8/ HTTP/1.1
Version: pip 1.4.1
Unfortunately pip does not use the system certs, but curl does.
I found a solution:
pip --cert /etc/ssl/certs/FOO_Root_CA.pem install pep8
This is not nice (curl and other libraries find the cert without adding a parameter) but works.
If you don’t want to use the command line argument, you can set the cert in ~/.pip/pip.conf:
[global]
cert = /etc/ssl/certs/Foo_Root_CA.pem
My solution is downloading cacert.pem from http://curl.haxx.se/ca/cacert.pem
and add the path for cacert.pem to ~/.pip/pip.conf as guettli suggested
[global]
cert = /path/to/cacert.pem
I use:
export PIP_CERT=`python -m pip._vendor.requests.certs`
pip install pep8
PIP always validates the certificate of HTTPS connections (and all pypi packages redirect to HTTPS).
The algorithm for determining the CA file is based on 3 steps:
- Look in a list of default locations for different linux distributions
(in my case this file turned out to be out of date, as I am building on a very old linux distribution) - If available, override the value found in (1) from a value in the pip.conf file, the environment or the command-line (in that order),
- If both (1) and (2) did not result in a value, use a bundled file
Note that pip does not use the default SSL directories and files (from ssl.get_default_verify_paths()). But only supports a bundled CA file.
PIP does support a command-line action to list the bundled file from step 3 and that is what I use for this answer.
For me, none of the config-file workarounds worked. I’m using pip 1.5.4 on Ubuntu 14.04
The command posted by @arjenve didn’t work on my system either. I get: /usr/bin/python: No module named _vendor.requests
UPDATE
An even better solution than my first workaround is installing the certificate on the system first (for me on ubuntu this would be)
sudo cp ~/my_cert.crt /usr/local/share/ca-certificates/
sudo update-ca-certificates
The previous automatically updates the bundle file (checking at the bottom of /etc/ssl/certs/ca-certificates.crt you should now see the same certificate as in my_cert.crt)
Now export that path into PIP_CERT and add it to your .bashrc:
echo export PIP_CERT=/etc/ssl/certs/ca-certificates.crt >> ~/.bashrc
OLDER WORKAROUND
My workaround was to create a bundle file from /etc/ssl/certs/ca-certificates.crt and my corporate’s crt (just concatenated both files). And then export a variable (put that on my .bashrc) like this:
export PIP_CERT=/my/path/to/the/bundle.crt