Installing SSL package with PIP requires SSL package to be already installed
Question:
- CentOS 7 (strict requirement)
- Python 3.11 (strict requirement)
I had to upgrage a software and it requires now Python 3.11.
I followed instructions from Internet (https://linuxstans.com/how-to-install-python-centos/), and now Python 3.11 is installed, but cannot download anything, so all the programs that have something to do with Internet, including PIP, do not work because SSL package is not installed.
The normal way to install a Python-package is to use PIP, which doesn’t work because the SSL package I’m going to install is not installed.
I tried all the advices in internet, but they are all outdated and not working any more, because they are either not for the 3.11 version of Python or not for CentOS 7.
The error I’m getting when running the application software:
ModuleNotFoundError: No module named ‘_ssl’
When I try to install ssl with pip:
# pip install --trusted-host pypi.org ssl
WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/ssl/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/ssl/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/ssl/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/ssl/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/ssl/
Could not fetch URL https://pypi.org/simple/ssl/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/ssl/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping
ERROR: Could not find a version that satisfies the requirement ssl (from versions: none)
ERROR: No matching distribution found for ssl
WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping
I downloaded GZip files from https://pypi.org/simple/ssl/, unpacked them locally and tried to install them from local source, but PIP insists on HTTPS connection … stupid tool.
What to do?
Answers:
How to get PIP and other HTTPS-based Python programs to work after upgrading to Python 3.11:
First of all: you don’t necessarily need any magical tools like pyenv. May be pyenv would do these steps, but I’d like to understand what is happening. (Ok, I admit that make is also a "magic" tool)
Briefly describing: during compilation of Python from source code there is an option to inject OpenSSL support directly into it.
In CentOS 7 Python 2.7.5 is installed by default and couldn’t be updated to the later ones using built-in package manager. Python 3.6.8 is the latest version available in the CentOS 7 repos. 3.6 also couldn’t be updated to the later ones using the package manager.
So the only possible solution is to compile Python from source code.
- Update your
yum packages, reboot, install all the packages neccesssary to run OpenSSL and Python.
- Download the latest OpenSSL source code, unpack and compile.
- Download the latest Python source code, unpack, configure to use the compiled OpenSSL and compile with altinstall parameter. Do not remove previous Python versions! You will have more problems than benefits. I had to revert virtual machine to the latest snapshot several times, because I destroyed something completely.
Update and install yum packages
> yum update
> yum install openssl-devel bzip2-devel libffi-devel
An article suggests also to install some "Development Tools"
> yum groupinstall "Development Tools"
but this step failed for me and I was able to finish the installation without it.
Download the latest OpenSSL source code, unpack and compile
I’ve choosen /usr/src directory to do the manipulations with source code.
Download
> cd /usr/src
> wget https://ftp.openssl.org/source/openssl-1.1.1q.tar.gz --no-check-certificate
Unpack
> tar -xzvf openssl-1.1.1q.tar.gz
> cd openssl-1.1.1q
Compile
> ./config --prefix=/usr --openssldir=/etc/ssl --libdir=lib no-shared zlib-dynamic
> make
Run tests for the compiled OpenSSL
> make test
Install
> make install
Check that OpenSSL is installed
> openssl version
OpenSSL 1.1.1q 5 Jul 2022
> which openssl
/usr/bin/openssl
Download and compile Python
Download
> cd /usr/src
> wget https://www.python.org/ftp/python/3.11.0/Python-3.11.0a4.tgz
Unpack
> tar -xzf Python-3.11.0a4.tgz
> cd Python-3.11.0a4
Configure
> ./configure --enable-optimizations --with-openssl=/usr
It is important that the --with-openssl option has the same value as the --prefix option when you configured OpenSSL above!!!
Compile and install (It’s time for a cup of coffee – it takes time)
> make altinstall
Checking that Python 3.11 is installed:
> python3.11 -V
Python 3.11.0a4
If you have set symbolic links, then Python 3.11 should be callable by "python3" and/or "python" aliases
> python3 -V
Python 3.11.0a4
> python -V
Python 3.11.0a4
Also check that PIP is working and that symlink-aliases for it are there.
Now it’s time to check that your Python-based programs are working. Some of them should be installed again by PIP, because they were installed in subdirectories of previous Python versions.
After doing these manipulations I also got SSL certificates error:
<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:998)>
After running
> pip3 install certifi
the problem is gone.
- CentOS 7 (strict requirement)
- Python 3.11 (strict requirement)
I had to upgrage a software and it requires now Python 3.11.
I followed instructions from Internet (https://linuxstans.com/how-to-install-python-centos/), and now Python 3.11 is installed, but cannot download anything, so all the programs that have something to do with Internet, including PIP, do not work because SSL package is not installed.
The normal way to install a Python-package is to use PIP, which doesn’t work because the SSL package I’m going to install is not installed.
I tried all the advices in internet, but they are all outdated and not working any more, because they are either not for the 3.11 version of Python or not for CentOS 7.
The error I’m getting when running the application software:
ModuleNotFoundError: No module named ‘_ssl’
When I try to install ssl with pip:
# pip install --trusted-host pypi.org ssl
WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/ssl/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/ssl/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/ssl/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/ssl/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/ssl/
Could not fetch URL https://pypi.org/simple/ssl/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/ssl/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping
ERROR: Could not find a version that satisfies the requirement ssl (from versions: none)
ERROR: No matching distribution found for ssl
WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping
I downloaded GZip files from https://pypi.org/simple/ssl/, unpacked them locally and tried to install them from local source, but PIP insists on HTTPS connection … stupid tool.
What to do?
How to get PIP and other HTTPS-based Python programs to work after upgrading to Python 3.11:
First of all: you don’t necessarily need any magical tools like pyenv. May be pyenv would do these steps, but I’d like to understand what is happening. (Ok, I admit that make is also a "magic" tool)
Briefly describing: during compilation of Python from source code there is an option to inject OpenSSL support directly into it.
In CentOS 7 Python 2.7.5 is installed by default and couldn’t be updated to the later ones using built-in package manager. Python 3.6.8 is the latest version available in the CentOS 7 repos. 3.6 also couldn’t be updated to the later ones using the package manager.
So the only possible solution is to compile Python from source code.
- Update your
yumpackages, reboot, install all the packages neccesssary to run OpenSSL and Python. - Download the latest OpenSSL source code, unpack and compile.
- Download the latest Python source code, unpack, configure to use the compiled OpenSSL and compile with altinstall parameter. Do not remove previous Python versions! You will have more problems than benefits. I had to revert virtual machine to the latest snapshot several times, because I destroyed something completely.
Update and install yum packages
> yum update
> yum install openssl-devel bzip2-devel libffi-devel
An article suggests also to install some "Development Tools"
> yum groupinstall "Development Tools"
but this step failed for me and I was able to finish the installation without it.
Download the latest OpenSSL source code, unpack and compile
I’ve choosen /usr/src directory to do the manipulations with source code.
Download
> cd /usr/src
> wget https://ftp.openssl.org/source/openssl-1.1.1q.tar.gz --no-check-certificate
Unpack
> tar -xzvf openssl-1.1.1q.tar.gz
> cd openssl-1.1.1q
Compile
> ./config --prefix=/usr --openssldir=/etc/ssl --libdir=lib no-shared zlib-dynamic
> make
Run tests for the compiled OpenSSL
> make test
Install
> make install
Check that OpenSSL is installed
> openssl version
OpenSSL 1.1.1q 5 Jul 2022
> which openssl
/usr/bin/openssl
Download and compile Python
Download
> cd /usr/src
> wget https://www.python.org/ftp/python/3.11.0/Python-3.11.0a4.tgz
Unpack
> tar -xzf Python-3.11.0a4.tgz
> cd Python-3.11.0a4
Configure
> ./configure --enable-optimizations --with-openssl=/usr
It is important that the --with-openssl option has the same value as the --prefix option when you configured OpenSSL above!!!
Compile and install (It’s time for a cup of coffee – it takes time)
> make altinstall
Checking that Python 3.11 is installed:
> python3.11 -V
Python 3.11.0a4
If you have set symbolic links, then Python 3.11 should be callable by "python3" and/or "python" aliases
> python3 -V
Python 3.11.0a4
> python -V
Python 3.11.0a4
Also check that PIP is working and that symlink-aliases for it are there.
Now it’s time to check that your Python-based programs are working. Some of them should be installed again by PIP, because they were installed in subdirectories of previous Python versions.
After doing these manipulations I also got SSL certificates error:
<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:998)>
After running
> pip3 install certifi
the problem is gone.