Building Python with SSL support in non-standard location

Try adding -Wl,-rpath,/home/fds/rms/local/lib to LDPATH.

in the Bourne shell (/bin/sh or /bin/bash):

$ LD_LIBRARY_PATH=/usr/local/lib
$ export LD_LIBRARY_PATH
$ make

in the C-shell (/bin/csh or /bin/tcsh):

% setenv LD_LIBRARY_PATH /usr/local/lib
% make

For me editing Modules/Setup was not enough as _hashlib module still ended up using the wrong OpenSSL version; and LD_LIBRARY_PATH was not taken into account at runtime on my SLES system.

What worked was to statically link the local OpenSSL to both _ssl and _hashlib by editing setup.py as per GitHub patch: eddy-geek/ python_custom_openssl.diff, and then make clean && make.

More details on why I used static links on Stack Overflow at Coredump when compiling python with a custom openssl version.

I was getting the same result until I got back to the logs for openssl.
There I saw that you need to use -fPIC when building openssl:
building ‘_ssl’ extension:

gcc -pthread -fPIC -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/ssl/include -I. -IInclude -I./Include -I/usr/local/include -I/home/feramos/Python-2.7.3/Include -I/home/feramos/Python-2.7.3 -c /home/feramos/Python-2.7.3/Modules/_ssl.c -o build/temp.linux-x86_64-2.7/home/feramos/Python-2.7.3/Modules/_ssl.o
gcc -pthread -shared build/temp.linux-x86_64-2.7/home/feramos/Python-2.7.3/Modules/_ssl.o -L/usr/local/ssl/lib -L/usr/local/lib -lssl -lcrypto -o build/lib.linux-x86_64-2.7/_ssl.so
/usr/bin/ld: /usr/local/ssl/lib/libcrypto.a(x86_64cpuid.o): relocation R_X86_64_PC32 against `OPENSSL_cpuid_setup' can not be used when making a shared object; recompile with -fPIC

openssl-0.9.8g]# .config -fPIC

then, make, make install for openssl and then build Python again.

Here is the complete process I used with Python 2.7.11.

In top level Python2.7.11 source dir:

1. Change Modules/Setup.dist, Modules/Setup : Uncomment _ssl section, comment out _socket (no-op if it’s already commented out), uncomment and set SSL appropriately (path to your new ssl lib/includes etc.)

   Note: Modules/Setup file does not exist initially but after first run it get’s content from Modules/Setup.dist i believe. Make sure changes are reflected in here before every run.

2. Apply patch: http://gist.github.com/eddy-geek/9604982
   (make distclean if previously ran make)

   ./configure --prefix=/root/.local/aks/py-ssl/python2.7 --enable-shared
   
   # modify: Makefile -> set svnversion to ""
   
   make
   
   make altinstall
   

I’m building Python2.7.13 and I see this same issue. For 2.7.13 you have to use “openssl1.0.0e” or above to make it work. I tried openssl-0.9.8g and it doesn’t work.
And somehow I can’t make it work just modifying Modules/Setup.dist so I have to manually compile that _ssl.o.
I guess this is because openssl-0.9.8g I provided is not working and it searched for system default libssl.so.10 which doesn’t work either.

I have a set of a couple of patches for static openssl and static libintl for 2 and 3 below.

For openssl (first patch) you have to set the env var OPENSSL_ROOT.

This is based on the patch from http://gist.github.com/eddy-geek/9604982 .

For Python 2.7.14:

https://gist.github.com/rkitover/2d9e5baff1f1cc4f2618dee53083bd35

https://gist.github.com/rkitover/afab7ed3ac7ce1860c43a258571c8ae1

For Python 3.6.3:

https://gist.github.com/rkitover/93d89a679705875c59275fb0a8f22b45

https://gist.github.com/rkitover/b18f19eafda3775a9652cc9cdf3ec914

For MAC OS HIGH Sierra,
and Python-3.5.6
In the above answer the openssl installation is done using source,but if you install using brew it tells where the installed package is, so if you install openssl using brew

brew install openssl

This will install openssl at /usr/local/Cellar/openssl/1.0.2o_2/ this path needs to be updated in Modules/Setup.dist at

Follow this answer which is mentioned above where the location of openssl installation is not mentioned to update the Modules/Setup.dist

In the above lines update SSL value to

SSL=/usr/local/Cellar/openssl/1.0.2o_2/

and uncomment the lines,
do a CMM and your python will get compiled with openssl.

On Linux Red Hat 7.7 x86_64, the following worked to install openssl-1.1.1d and Python-3.8.1 in my home directory (/home/unix/vangalen):

Install OpenSSL
source1 source2

cd ~
wget https://www.openssl.org/source/openssl-1.1.1d.tar.gz
tar -xzf openssl-1.1.1d.tar.gz
cd /home/unix/vangalen/openssl-1.1.1d
./config --prefix=/home/unix/vangalen/openssl --openssldir=/home/unix/vangalen/openssl
make
make test
make install

Install Python
source2 source3 source4

cd ~
wget https://www.python.org/ftp/python/3.8.1/Python-3.8.1.tgz
tar xf Python-3.8.1.tgz

Modify Python-3.8.1/Modules/Setup in a text editor. If this file doesn’t exist you may need to go through a failed run first. Uncomment lines and adjust the alias for SSL in lines 206 to 213::

_socket socketmodule.c

# Socket module helper for SSL support; you must comment out the other
# socket line above, and possibly edit the SSL variable:
SSL=/home/unix/vangalen/openssl
_ssl _ssl.c 
  -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl 
  -L$(SSL)/lib -lssl -lcrypto

cd ~/Python-3.8.1
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/home/unix/vangalen/openssl/lib
./configure --prefix=/home/unix/vangalen/py-381 --with-openssl=/home/unix/vangalen/openssl
make
make test
make install

@PeterVanGalen has a good method, and is relevant as of 2021, but I disagree with some details.

His method of modifying the Modules/Setup file results in the libssl.so and libcrypto.so libraries becoming dynamic dependencies of the python binary itself. This is not as intended — those are supposed to be dependencies of python importable .so‘s, like _ssl.cpython-39-x86_64-linux-gnu.so.

His method also doesn’t include a solution for how libssl.so and libcrypto.so will be found at runtime, rather than build time. This matters if they are in some unusual, custom path, or else python will either not run at all or be unable to import ssl. You could solve it in lots of ways (LD_LIBRARY_PATH and ld.so.conf come to mind) but I opted to use rpath, so that the .so files can always be found in their custom location when I use this python, but will otherwise stay out of the way.

This is my modification of @PeterValGalen’s approach:

# First we need openssl installed in a custom location...
tar zxf openssl-1.1.1j.tar.gz
cd openssl-1.1.1j
./config --prefix=/my/weird/path --openssldir=/my/weird/path/ssl
make
make install  # Add `sudo` if needed for permission to /my/weird/path.
cd ..

# Now the python part...
wget https://www.python.org/ftp/python/3.9.2/Python-3.9.2.tgz
tar xf Python-3.9.2.tgz
cd Python-3.9.2
LDFLAGS="${LDFLAGS} -Wl,-rpath=/my/weird/path/lib" ./configure --with-openssl=/my/weird/path
make
make install  # Add `sudo` if needed for permissions.

When done this way, the openssl libs are not dependencies of the python binary, but are dependencies of the _ssl*.so

$ ldd ./python | grep weird
# [Nothing]

$ ldd build/lib.linux-x86_64-3.9/_ssl.cpython-39-x86_64-linux-gnu.so  | grep weird
        libssl.so.1.1 => /my/weird/path/lib/libssl.so.1.1 (0x00007f733ee73000)
        libcrypto.so.1.1 => /my/weird/path/lib/libcrypto.so.1.1 (0x00007f733e9ab000)

And it works:

$ ./python -c "import ssl; print('yay')"
yay

$ ./python Lib/test/test_ssl.py
# ... lots of good stuff...
OK (skipped=10)

OpenSSL

cd /opt
sudo wget https://www.openssl.org/source/openssl-1.1.1q.tar.gz --no-check-certificate
sudo mkdir /opt/openssl
sudo tar xfvz openssl-1.1.1q.tar.gz --directory /opt/openssl
export LD_LIBRARY_PATH=/opt/openssl/lib
cd /opt/openssl/openssl-1.1.1q/
sudo ./config --prefix=/opt/openssl --openssldir=/opt/openssl/ssl
sudo make
sudo make test
sudo make install
sudo ln -s /usr/local/bin/openssl /usr/bin/openssl

Python 3.9.13

cd /usr/src 
sudo wget https://www.python.org/ftp/python/3.9.13/Python-3.9.13.tgz 
sudo tar xzf Python-3.9.13.tgz
cd /usr/src/Python-3.9.13
sudo CFLAGS="-I/opt/openssl/include/" LDFLAGS="${LDFLAGS} -Wl,-rpath=$LD_LIBRARY_PATH" ./configure --enable-optimizations --with-openssl=/opt/openssl
sudo make altinstall -j6
sudo ln -s /usr/local/bin/python3.9 /usr/bin/python3

pip3

sudo ln -s /usr/local/bin/pip3.9 /usr/bin/pip3
pip3 install --upgrade pip

Tested on Ubuntu 14.04.06