Problem when installing Python from source, SSL package missing even though openssl installed

Question:

The Problem

Trying to install Python-3.11.1 from source on Zorin OS (Ubuntu16 based) I get the following errors when I try to pip install any package into a newly created venv:

python3.11 -m venv venv
source venv/bin/active
pip install numpy
WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/numpy/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/numpy/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/numpy/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/numpy/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/numpy/
Could not fetch URL https://pypi.org/simple/numpy/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/numpy/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping
ERROR: Could not find a version that satisfies the requirement numpy (from versions: none)
ERROR: No matching distribution found for numpy

Obviously, the SSL package seems to be missing, however I made sure to have both openssl and libssl-dev installed before installing python. More specifically, I made sure to have all packages installed lined out here.

The Exact Steps I Took To Install

  1. Make sure all packages that are required are installed (the once above)
  2. cd .../python-installs
  3. Download Python from python.org
  4. tar -xvzf Python-3.11.1.tgz
  5. cd Python-3.11.1 and then
./configure 
    --prefix=/opt/python/3.11.1 
    --enable-shared 
    --enable-optimizations 
    --enable-ipv6 
    --with-openssl=/usr/lib/ssl  
    --with-openssl-rpath=auto  
    LDFLAGS=-Wl,-rpath=/opt/python/3.11.1/lib,--disable-new-dtags
  1. make <- Note that I get a lot off error messages from gcc here, very similar to this, however it seems its successful at the end
  2. make altinstall

Parts of this installation process are from [1], [2]

Running python3.11 seems to work fine, however I cannot pip install anything into a venv created by Python3.11.1.

Other Possible Error Sources

Before trying to reinstall Python3.11.1, I always made sure to delete all files in the following places that were associated with Python3.11.1:

/usr/local/bin/...
/usr/local/lib/...
/usr/local/man/man1/...
/usr/local/share/man/man1/...
/usr/local/lib/pkgconfig/...
/opt/python/...

I also tried adding Python-3.11.1 to PATH by adding

PATH=/opt/python/3.11.1/bin:$PATH

to /etc/profile.d/python.sh, but it didn’t seem to do much in my case.

When configuring the python folder I am using --with-openssl=/usr/lib/ssl, though perhaps I need to use something else? I tried --with-openssl=/usr/bin/openssl but that doesn’t work because openssl is a file and not a folder and it gives me an error message and doesn’t even configure anything.

Conclusion

From my research I found that most times this error relates to the openssl library not being installed (given that python versions >= 3.10 will need it to be installed), and that installing it and reinstalling python seemed to fix the issue. However in my case it doesn’t, and I don’t know why that is.

The most likely cause is that something is wrong with my openssl configuration, but I wouldn’t know what.

Any help would be greatly appreciated.

Asked By: Racid

||
Source

Answers:

After some more research, I realized that I didn’t have libbz2-dev installed, which is obviously the first thing one should check if they get the errors above but oh well. For anyone who still finds himself struggling, here are my complete steps I took:

  1. Make sure the following libraries are installed
apt show libbz2-dev
apt show openssl
apt show libssl-dev
# Other libraries that might also be needed
apt show liblzma-dev
  1. cd .../python-installs
  2. Download the target Python version from python.org as Gzipped tar ball
  3. tar -xvzf Python-3.11.1.tgz
  4. sudo mkdir opt/python
  5. sudo mkdir opt/python/3.11.1
  6. cd Python-3.11.1 and then
./configure --prefix=/opt/python/3.11.1 
    --enable-optimizations
  1. make <- Note that I still get a lot of error messages from gcc, also get a always_inline not in line error message
  2. sudo make altinstall
  3. Add PATH=/opt/python/3.11.1/bin:$PATH to the file /etc/profile.d/python.sh
  4. reboot

Then to test if the error is gone one can for example test:

  1. python3.11 -m venv venv
  2. source venv/bin/active
  3. pip install pandas
  4. python3.11
  5. import pandas
  6. exit()

If there are no errors then everything worked out. Obviously the version needs to be changed to the actual target version of yours.

Note

If you your newly installed python version does not appear in terminal, it might be because the file /etc/profile.d/python.sh already existed with the Path to the python version (for example, if you had to install it multiple times). In that case, delete the file (or at least the PATH to the target version) and then recreate it. After rebooting it should appear in terminal.

Answered By: Racid