OpenSSL FIPS_mode_set not working in Python cryptography library

1. General

First of all, I want to mention that although I understand the reasons, I don’t fully agree with Cryptography‘s vision ([Cryptography]: Installation – Static Wheels). Shared libraries exist for decades and have proven their superiority. Not to mention that Python ships 2 (standard) modules (_ssl and _hashlib) that dynamically link to OpenSSL (whatever it’s on the system). As a side note, on Win, the 2 Python modules also used to link statically to OpenSSL, but starting with v3.7, they no longer do. Back to Nix: 2 OpenSSL versions are loaded into the same (Python) process. It doesn’t seem to harm, but it looks funny. And as things are today (191009), there are a bunch of .whls for v2.7 and v3.4, but none for a fairly decent (Python) environment:

I remember a similar situation a while ago: besides the 2 standard modules, M2Crypto was also used. In that situation (we shipped Python entirely), the 2 specific (FIPS capable) OpenSSL (dynamic) libs were shipped too, and all the Python modules linked to them. It worked on a variety of (desktop) environments (out of which many "exotic" ones):

• CPU architectures (LE / BE): x86, AMD64, IA64, SPARC, PPC, zSeries
• OSes (with multiple versions): Win, Lnx (RH, CentOS, OEL, SuSE, Xen, Ubuntu), Solaris, AIX, HP-UX (and as personal exercise, I added OSX)

[OpenSSL]: UserGuide-2.0.pdf – User Guide for the OpenSSL FIPS Object Module v2.0 (referenced from [OpenSSL]: FIPS-140 in case the URL changes) contains all the details needed.

Before going further, here are some terms that I’m going to use throughout the post:

• FOM – FIPS object module (fipscanister.o)

• FOME – The executable (ELF (PE on Win)) that FOM was linked in. Bear in mind, that it can be either an executable per se, either an .so (.dll). Also, if it’s included in a static lib (.a), it’s not linked (just archived). As a side note, when OpenSSL is built shared, FOME is libcrypto.so.*, while when built statically (like in this case), it’s the executable that links with libcrypto.a (e.g. openssl executable)

FOM comes on top of OpenSSL (and probably other such cryptography providers, like LibreSSL, WolfSSL, ), and it’s meant to strengthen security, (according to NIST standards), by restricting some features that otherwise would be available. One such feature example is the usage of md5 hash, which is considered weak (I’m pretty sure that sha1 will follow too, in the next version about to be released).
Here’s a (very simplified) version of what happens (at runtime):

1. FIPS mode on:

   1. Check whether the additional constraints are met:

      1. Yes: Proceed (with default functionality)

      2. No: Return with error

2. FIPS mode off:

   1. Fall back to default functionality

Part of #1.1. is the selftest. That consists of:

• Computing the FOME signature

• Comparing it to a value (that was also stored in FOME)

This happens to make sure (or drastically reduce the chances) that no one tampered (manually modifying, disassembling, …) with FOME. To have a better understanding of the signature mechanism, let’s dive into FOME build process:

1. All the FOME sources + FOM are compiled (into object files)

2. They are linked together (into FOME). This is when a normal build ends

3. The FOME signature is being computed

   1. Items from #1. + fips_premain.o are being linked into a (real, not .dll) executable (FPD)

   2. FPD is invoked against FOME (#1.). It reads FOME‘s .rodata section and computes its sha1 hash. Note that it ignores a 41 bytes zone (punches a hole) located at a specific address

4. #3.1. is repeated, but this time fips_premain.o was recompiled to also include the hash from previous step. Now it becomes clear the punched hole from previous step, it’s the place where the signature goes: (length of sha hash (40) + nul). This is the final FOME

Note: On Win, things happen just a bit differently.

I’ve managed to reproduce the problem. I’m going to start with the test script.

code00.py:

#!/usr/bin/env python

import sys

import cffi

from cryptography.hazmat.backends.openssl.backend import backend


def main(*argv):
    ffi = cffi.FFI()
    lib = backend._lib
    fmt = "OpenSSL version: {0:s}nFIPS_mode(): {1:d}nFIPS_mode_set(1): {2:d}nFIPS_mode(): {3:d}"
    print(fmt.format(ffi.string(
        lib.OPENSSL_VERSION_TEXT).decode(),
        lib.FIPS_mode(),lib.FIPS_mode_set(1), lib.FIPS_mode()
    ))
    err = lib.ERR_get_error()
    if err:
        err_fmt = "error:[{0:d}]:[{1:s}]:[{2:s}]:[{3:s}]"
        print(err_fmt.format(
            err,
            ffi.string(lib.ERR_lib_error_string(err)).decode(),
            ffi.string(lib.ERR_func_error_string(err)).decode(),
            ffi.string(lib.ERR_reason_error_string(err)).decode()
        ))
    else:
        print("Success !!!")


if __name__ == "__main__":
    print("Python {:s} {:03d}bit on {:s}n".format(" ".join(elem.strip() for elem in sys.version.split("n")),
                                                   64 if sys.maxsize > 0x100000000 else 32, sys.platform))
    rc = main(*sys.argv[1:])
    print("nDone.")
    sys.exit(rc)

2. Setup

I’ve already built FOM and a FIPS capable OpenSSL (similar to yours, but I customized their paths). The ${FIPSDIR} variable was used when building both FOM and OpenSSL.

[cfati@cfati-ubtu16x64-0:~/Work/Dev/StackOverflow/q058228435]> ~/sopr.sh
*** Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ***

[064bit-prompt]> uname -a
Linux cfati-ubtu16x64-0 4.15.0-65-generic #74~16.04.1-Ubuntu SMP Wed Sep 18 09:51:44 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
[064bit-prompt]> cat /etc/lsb-release | grep DESCR
DISTRIB_DESCRIPTION="Ubuntu 16.04.6 LTS"
[064bit-prompt]> gcc --version | grep gcc
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609
[064bit-prompt]>
[064bit-prompt]> echo ${FIPSDIR}
/home/cfati/Work/Dev/Tools/zzz_Build/OpenSSL/int/openssl-fips-2.0.16
[064bit-prompt]> tree ${FIPSDIR}
/home/cfati/Work/Dev/Tools/zzz_Build/OpenSSL/int/openssl-fips-2.0.16
├── bin
│   ├── fipsld
│   └── fips_standalone_sha1
├── include
│   └── openssl
│       ├── aes.h
│       ├── bn.h
│       ├── buffer.h
│       ├── cmac.h
│       ├── crypto.h
│       ├── des.h
│       ├── des_old.h
│       ├── dh.h
│       ├── dsa.h
│       ├── ebcdic.h
│       ├── ecdh.h
│       ├── ecdsa.h
│       ├── ec.h
│       ├── e_os2.h
│       ├── evp.h
│       ├── fips.h
│       ├── fips_rand.h
│       ├── fipssyms.h
│       ├── hmac.h
│       ├── modes.h
│       ├── opensslconf.h
│       ├── opensslv.h
│       ├── ossl_typ.h
│       ├── rsa.h
│       ├── sha.h
│       └── symhacks.h
└── lib
    ├── fipscanister.o
    ├── fipscanister.o.sha1
    ├── fips_premain.c
    └── fips_premain.c.sha1

4 directories, 32 files
[064bit-prompt]>
[064bit-prompt]> echo ${OPENSSL_DIR}
/home/cfati/Work/Dev/Tools/openssl-1.0.2t-fips-2.0.16-static
[064bit-prompt]> tree ${OPENSSL_DIR}/bin ${OPENSSL_DIR}/lib
/home/cfati/Work/Dev/Tools/openssl-1.0.2t-fips-2.0.16-static/bin
├── c_rehash
└── openssl
/home/cfati/Work/Dev/Tools/openssl-1.0.2t-fips-2.0.16-static/lib
├── engines
├── libcrypto.a
├── libssl.a
└── pkgconfig
    ├── libcrypto.pc
    ├── libssl.pc
    └── openssl.pc

2 directories, 7 files
[064bit-prompt]>
[064bit-prompt]> ldd ${OPENSSL_DIR}/bin/openssl
    linux-vdso.so.1 =>  (0x00007ffeec045000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f12c19c2000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f12c15f8000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f12c1bc6000)
[064bit-prompt]>
[064bit-prompt]> ${OPENSSL_DIR}/bin/openssl version
OpenSSL 1.0.2t-fips  10 Sep 2019
[064bit-prompt]> ${OPENSSL_DIR}/bin/openssl sha1 ./code00.py
SHA1(./code00.py)= ff122260b025103dbc03316e3d3e26cd683e7a12
[064bit-prompt]> ${OPENSSL_DIR}/bin/openssl md5 ./code00.py
MD5(./code00.py)= eac85e46734260c1bfcceb89d6a3bd32
[064bit-prompt]> OPENSSL_FIPS=1 ${OPENSSL_DIR}/bin/openssl sha1 ./code00.py
SHA1(./code00.py)= ff122260b025103dbc03316e3d3e26cd683e7a12
[064bit-prompt]> OPENSSL_FIPS=1 ${OPENSSL_DIR}/bin/openssl md5 ./code00.py
Error setting digest md5
140584610875032:error:060A80A3:digital envelope routines:FIPS_DIGESTINIT:disabled for fips:fips_md.c:180:

3. Cryptography module

[064bit-prompt]> ls
code00.py  cryptography-2.7.tar.gz
[064bit-prompt]> mkdir build
[064bit-prompt]> cd build
[064bit-prompt]>
[064bit-prompt]> CFLAGS="-I${OPENSSL_DIR}/include -DOPENSSL_FIPS=1" LDFLAGS="-L${OPENSSL_DIR}/lib" python3 -m pip wheel --no-cache --no-binary :all: ../cryptography-2.7.tar.gz
Processing /home/cfati/Work/Dev/StackOverflow/q058228435/cryptography-2.7.tar.gz
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting asn1crypto>=0.21.0 (from cryptography==2.7)
  Downloading https://files.pythonhosted.org/packages/d1/e2/c518f2bc5805668803ebf0659628b0e9d77ca981308c7e9e5564b30b8337/asn1crypto-1.0.1.tar.gz (115kB)
     |████████████████████████████████| 122kB 801kB/s
Collecting cffi!=1.11.3,>=1.8 (from cryptography==2.7)
  Downloading https://files.pythonhosted.org/packages/93/1a/ab8c62b5838722f29f3daffcc8d4bd61844aa9b5f437341cc890ceee483b/cffi-1.12.3.tar.gz (456kB)
     |████████████████████████████████| 460kB 1.8MB/s
Collecting six>=1.4.1 (from cryptography==2.7)
  Downloading https://files.pythonhosted.org/packages/dd/bf/4138e7bfb757de47d1f4b6994648ec67a51efe58fa907c1e11e350cddfca/six-1.12.0.tar.gz
Collecting pycparser (from cffi!=1.11.3,>=1.8->cryptography==2.7)
  Downloading https://files.pythonhosted.org/packages/68/9e/49196946aee219aead1290e00d1e7fdeab8567783e83e1b9ab5585e6206a/pycparser-2.19.tar.gz (158kB)
     |████████████████████████████████| 163kB 4.5MB/s
Building wheels for collected packages: cryptography, asn1crypto, cffi, six, pycparser
  Building wheel for cryptography (PEP 517) ... done
  Stored in directory: /home/cfati/Work/Dev/StackOverflow/q058228435/build
  Building wheel for asn1crypto (setup.py) ... done
  Stored in directory: /home/cfati/Work/Dev/StackOverflow/q058228435/build
  Building wheel for cffi (setup.py) ... done
  Stored in directory: /home/cfati/Work/Dev/StackOverflow/q058228435/build
  Building wheel for six (setup.py) ... done
  Stored in directory: /home/cfati/Work/Dev/StackOverflow/q058228435/build
  Building wheel for pycparser (setup.py) ... done
  Stored in directory: /home/cfati/Work/Dev/StackOverflow/q058228435/build
Successfully built cryptography asn1crypto cffi six pycparser
WARNING: You are using pip version 19.1.1, however version 19.2.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
[064bit-prompt]>
[064bit-prompt]> ls
asn1crypto-1.0.1-py3-none-any.whl        cryptography-2.7-cp35-cp35m-linux_x86_64.whl  six-1.12.0-py2.py3-none-any.whl
cffi-1.12.3-cp35-cp35m-linux_x86_64.whl  pycparser-2.19-py2.py3-none-any.whl
[064bit-prompt]>
[064bit-prompt]> for f in $(ls *.whl); do unzip ${f} > /dev/null; done
[064bit-prompt]> ls
asn1crypto                         cffi-1.12.3-cp35-cp35m-linux_x86_64.whl        cryptography-2.7-cp35-cp35m-linux_x86_64.whl  pycparser-2.19-py2.py3-none-any.whl
asn1crypto-1.0.1.dist-info         cffi-1.12.3.dist-info                          cryptography-2.7.dist-info                    six-1.12.0.dist-info
asn1crypto-1.0.1-py3-none-any.whl  _cffi_backend.cpython-35m-x86_64-linux-gnu.so  pycparser                                     six-1.12.0-py2.py3-none-any.whl
cffi                               cryptography                                   pycparser-2.19.dist-info                      six.py
[064bit-prompt]> PYTHONPATH=.:${PYTHONPATH} python3 ../code00.py
Python 3.5.2 (default, Jul 10 2019, 11:58:48) [GCC 5.4.0 20160609] 64bit on linux

OpenSSL version: OpenSSL 1.0.2t-fips  10 Sep 2019
FIPS_mode(): 0
FIPS_mode_set(1): 0
FIPS_mode(): 0
error:[755413103]:[FIPS routines]:[FIPS_check_incore_fingerprint]:[fingerprint does not match]

Done.

As seen, I’m pretty much where you are.

4. Deeper dive

After long (and some might consider painful) hours of debugging, trials, …, I reached to a conclusion. Considering that:

• Tampering with anything from FOM (contents of ${FIPSDIR}), will no loner qualify as FIPS validated. To be frank, neither does this, as there are specific instructions that when building FOM, only a sys admin should copy the artifacts in a secure location …., bla, bla, bla. This seems paranoid to me, but these are the facts. As a remark, back in 2013, when we 1^st came in contact with FIPS (probably to prevent any possible attack (e.g. MITM)), the FOM sources CD was shipped from USA to ROU :)))
• Default Python version is statically (again 🙂 ) built, meaning that ${PYTHONCORE} (the Python interpreter) resides in the python executable, rather than in a .so (libpython*.so*) that can be linked to (and that the python executable does in case of shared builds)
• Cryptography‘s _openssl extension module (_openssl.abi*.so) needs symbols from ${PYTHONCORE} (e.g. PyLong_FromLong), but that’s OK since at the moment it will be loaded into the (Python) process (launched from the aforementioned executable), it will find them (this is a common practice on Nix)
• Build step #3.2.: the executable (FPD – which must run) doesn’t find the symbols, so it fails

This is a deadlock (whatever room one constraint leaves, is closed by others), so IT SIMPLY CAN’T BE DONE !!! (at least, at this time). Period!!! X(

5. Alternative

I was going to suggest this as an elegant alternative (including the OpenSSL .sos (with any (.so) client having rpath set to "there") in the .whl, next to _openssl.abi3.so which links to them), but apparently this is the only way (that I’ve found, at least).

The 1^st step, is to build a shared OpenSSL version (FOME will be libcrypto.so.*).

[064bit-prompt]> ls
code00.py  cryptography-2.7.tar.gz
[064bit-prompt]> export OPENSSL_DIR=/home/cfati/Work/Dev/Tools/openssl-1.0.2t-fips-2.0.16
[064bit-prompt]> ldd ${OPENSSL_DIR}/bin/openssl
        linux-vdso.so.1 =>  (0x00007ffe62faf000)
        libssl.so.1.0.0 => /home/cfati/Work/Dev/Tools/openssl-1.0.2t-fips-2.0.16/lib/libssl.so.1.0.0 (0x00007fe33c06f000)
        libcrypto.so.1.0.0 => /home/cfati/Work/Dev/Tools/openssl-1.0.2t-fips-2.0.16/lib/libcrypto.so.1.0.0 (0x00007fe33bb92000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fe33b7c8000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fe33b5c4000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fe33c2e3000)
[064bit-prompt]>
[064bit-prompt]> ${OPENSSL_DIR}/bin/openssl version
OpenSSL 1.0.2t-fips  10 Sep 2019
[064bit-prompt]> ${OPENSSL_DIR}/bin/openssl md5 ./code00.py
MD5(./code00.py)= eac85e46734260c1bfcceb89d6a3bd32
[064bit-prompt]> OPENSSL_FIPS=1 ${OPENSSL_DIR}/bin/openssl md5 ./code00.py
Error setting digest md5
139796140275352:error:060A80A3:digital envelope routines:FIPS_DIGESTINIT:disabled for fips:fips_md.c:180

After another session of deep diving (lots of failed attempts), I was able to get it working. However, took a lot of actions:

• Manual interventions
• Hacks
• (Lame) workarounds (gainarii)

I’m afraid that if I’d put everything here it would well exceed the 30K chars limit ([SE.Meta]: Knowing Your Limits: What is the maximum length of a question title, post, image and links used?).

However, I published the .whl at [GitHub]: CristiFati/Prebuilt-Binaries – (master) Prebuilt-Binaries/Cryptography/v2.7. So far it’s for Python 3.5 (64bit) only, as it’s the default version that comes on Ubuntu 16. If you use another (newer) version, just let me know, and I’ll get it (maybe build it myself), and build the .whl for that version too (I am going to do it anyway).

After replacing the original .whl with the one built by me:

[cfati@cfati-ubtu16x64-0:~/Work/Dev/StackOverflow/q058228435/build]> ll
total 2936
drwxrwxr-x 2 cfati cfati    4096 Oct  9 21:40 .
drwxrwxr-x 4 cfati cfati    4096 Oct  9 21:28 ..
-rw-rw-r-- 1 cfati cfati  108067 Oct  9 08:43 asn1crypto-1.0.1-py3-none-any.whl
-rw-rw-r-- 1 cfati cfati  318045 Oct  9 08:43 cffi-1.12.3-cp35-cp35m-linux_x86_64.whl
-rw-rw-r-- 1 cfati cfati 2438739 Oct  9 21:40 cryptography-2.7-cp35-cp35m-linux_x86_64.whl
-rw-rw-r-- 1 cfati cfati  112066 Oct  9 08:43 pycparser-2.19-py2.py3-none-any.whl
-rw-rw-r-- 1 cfati cfati   12099 Oct  9 08:43 six-1.12.0-py2.py3-none-any.whl
[cfati@cfati-ubtu16x64-0:~/Work/Dev/StackOverflow/q058228435/build]> for f in $(ls *.whl); do unzip ${f} > /dev/null; done
[cfati@cfati-ubtu16x64-0:~/Work/Dev/StackOverflow/q058228435/build]> ls
asn1crypto                         cffi-1.12.3-cp35-cp35m-linux_x86_64.whl        cryptography-2.7-cp35-cp35m-linux_x86_64.whl  pycparser-2.19-py2.py3-none-any.whl
asn1crypto-1.0.1.dist-info         cffi-1.12.3.dist-info                          cryptography-2.7.dist-info                    six-1.12.0.dist-info
asn1crypto-1.0.1-py3-none-any.whl  _cffi_backend.cpython-35m-x86_64-linux-gnu.so  pycparser                                     six-1.12.0-py2.py3-none-any.whl
cffi                               cryptography                                   pycparser-2.19.dist-info                      six.py
[cfati@cfati-ubtu16x64-0:~/Work/Dev/StackOverflow/q058228435/build]> PYTHONPATH=.:${PYTHONPATH} python3 ../code00.py
Python 3.5.2 (default, Jul 10 2019, 11:58:48) [GCC 5.4.0 20160609] 64bit on linux

OpenSSL version: OpenSSL 1.0.2t-fips  10 Sep 2019
FIPS_mode(): 0
FIPS_mode_set(1): 1
FIPS_mode(): 1
Success !!!

Done.

Related (more or less) posts:

• [SO]: Unable to build a working FIPS capable OpenSSL on HP-UX

• [SO]: How to enable FIPS mode for libcrypto and libssl packaged with Python? (@CristiFati’s answer)

• [SO]: How to implement FIPS_mode() and FIPS_mode_set() in Python 3.6’s ssl module? (@CristiFati’s answer)