Can't receive peer certificate in Python client using OpenSSL's ssl.SSLContext()

After a number of attempts, some failed, some partial successful, I found a way that should work (didn’t test it with self signed certificates, though). Also, I wiped out everything from the previous attempts.

There are 2 necessary steps:

1. Get the server certificate using [Python 3.Docs]: (ssl.get_server_certificate(addr, ssl_version=PROTOCOL_TLS, ca_certs=None), which returns it as a PEM encoded string (e.g.: ours – pretty printed):

   '-----BEGIN CERTIFICATE-----'
   'MIIIPjCCByagAwIBAgIICG/ofYt2G48wDQYJKoZIhvcNAQELBQAwSTELMAkGA1UE'
   'BhMCVVMxEzARBgNVBAoTCkdvb2dsZSBJbmMxJTAjBgNVBAMTHEdvb2dsZSBJbnRl'
   
   ...
   
   'L2KuOvWZ40sTVCJdWPUMtT9VP7VHfLNTFft/IhR+bUPkr33xjOa0Idq6cL89oufn'
   '-----END CERTIFICATE-----'
   

2. Decode the certificate using (!!!undocumented!!!) ssl._ssl._test_decode_cert (present in Python 3 / Python 2)

3. Due to the fact that ssl._ssl._test_decode_cert can only read the certificate from a file, 2 additional steps are needed:

   • Save the certificate from #1. in a temporary file (before #2., obviously)

   • Delete that file when done with it

I would like to emphasize [Python 3.Docs]: SSLSocket.getpeercert(binary_form=False), which contains lots of info (that I missed the last time(s)).
Also, I found out about ssl._ssl._test_decode_cert, by looking at SSLSocket.getpeercert implementation ("${PYTHON_SRC_DIR}/Modules/_ssl.c").

code00.py:

#!/usr/bin/env python

import itertools
import os
import socket
import ssl
import sys


def _get_tmp_cert_file_name(host, port):
    return os.path.join(os.path.dirname(os.path.abspath(__file__)), "_".join(("cert", host, str(port), str(os.getpid()), ".crt")))


def _decode_cert(cert_pem, tmp_cert_file_name):
    #print(tmp_cert_file_name)
    with open(tmp_cert_file_name, "w") as fout:
        fout.write(cert_pem)
    try:
        return ssl._ssl._test_decode_cert(tmp_cert_file_name)
    except Exception as e:
        print("Error decoding certificate:", e)
        return dict()
    finally:
        os.unlink(tmp_cert_file_name)


def get_srv_cert_0(host, port=443):
    try:
        cert_pem = ssl.get_server_certificate((host, port))
    except Exception as e:
        print("Error getting certificate:", e)
        return dict()
    tmp_cert_file_name = _get_tmp_cert_file_name(host, port)
    return _decode_cert(cert_pem, tmp_cert_file_name)


def get_srv_cert_1(host, port=443):
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    context = ssl.SSLContext()
    ssl_sock = context.wrap_socket(sock, server_hostname=host)
    try:
        ssl_sock.connect((host, port))
    except Exception as e:
        print("Error connecting:n", e)
        return dict()
    try:
        cert_der = ssl_sock.getpeercert(True)
    except Exception as e:
        print("Error getting cert:n", e)
        return dict()
    tmp_cert_file_name = _get_tmp_cert_file_name(host, port)
    return _decode_cert(ssl.DER_cert_to_PEM_cert(cert_der), tmp_cert_file_name)


def main(*argv):
    domain = "google.com"
    if argv:
        print("Using custom method")
        get_srv_cert_func = get_srv_cert_1
    else:
        print("Using regular method")
        get_srv_cert_func = get_srv_cert_0

    cert = get_srv_cert_func(domain)
    print("====== peer's certificate ======")
    try:
        print("Issued To:", dict(itertools.chain(*cert["subject"]))["commonName"])
        print("Issued By:", dict(itertools.chain(*cert["issuer"]))["commonName"])
        print("Valid From:", cert["notBefore"])
        print("Valid To:", cert["notAfter"])
        if (cert == None):
            print("no certificate")
    except Exception as e:
        print("Error getting certificate:", e)


if __name__ == "__main__":
    print("Python {:s} {:03d}bit on {:s}n".format(" ".join(elem.strip() for elem in sys.version.split("n")),
                                                   64 if sys.maxsize > 0x100000000 else 32, sys.platform))
    rc = main(*sys.argv[1:])
    print("nDone.")
    sys.exit(rc)

Notes:

• _get_tmp_cert_file_name: generates the temporary file name (located in the same dir as the script) that will store the certificate

• _decode_cert: saves the certificate in the file, then decodes the file and returns the resulting dict

• get_srv_cert_0: gets the certificate form server, then decodes it

• get_srv_cert_1: same thing that get_srv_cert_0 does, but "manually"

  • Its advantage is controlling the SSL context creation / manipulation (which I think was the main point of the question)

• main:

  • Gets the server certificate using one of the 2 methods above (based on an argument being / not being passed to the script)

  • Prints certificate data (your code with some small corrections)

Output:

(py35x64_test) e:WorkDevStackOverflowq050055935> "e:WorkDevVEnvspy35x64_testScriptspython.exe" ./code00.py
Python 3.5.4 (v3.5.4:3f56838, Aug  8 2017, 02:17:05) [MSC v.1900 64 bit (AMD64)] 064bit on win32

Using regular method
====== peer's certificate ======
Issued To: *.google.com
Issued By: Google Internet Authority G2
Valid From: Apr 10 18:58:05 2018 GMT
Valid To: Jul  3 18:33:00 2018 GMT

Done.

(py35x64_test) e:WorkDevStackOverflowq050055935> "e:WorkDevVEnvspy35x64_testScriptspython.exe" ./code00.py 1
Python 3.5.4 (v3.5.4:3f56838, Aug  8 2017, 02:17:05) [MSC v.1900 64 bit (AMD64)] 064bit on win32

Using custom method
====== peer's certificate ======
Issued To: *.google.com
Issued By: Google Internet Authority G2
Valid From: Apr 10 18:55:13 2018 GMT
Valid To: Jul  3 18:33:00 2018 GMT

Done.

Check [SO]: How can I decode a SSL certificate using python? (@CristiFati’s answer) for the decoding part only.

It looks like Python doesn’t parse/store the client’s cert when ssl.CERT_NONE is set which is the default for SSLContext

import socket
import ssl

dest = ("www.google.com", 443)

sock = socket.socket(socket.AF_INET)
ctx = ssl.SSLContext()

# You will either need to add the self-signed certs to the OS cert store (varies by OS)
# See https://docs.python.org/3/library/ssl.html#ssl.SSLContext.load_verify_locations for loading non-defaults
ctx.set_default_verify_paths()

ctx.verify_mode = ssl.CERT_REQUIRED
ssock = ctx.wrap_socket(sock, server_hostname=dest[0])
result = ssock.connect(dest)
print(ssock.getpeercert())

See https://github.com/python/cpython/blob/main/Modules/_ssl.c#L1823
It’s doing bitwise & so when verify mode is CERT_NONE (0) then the code block is skipped. CERT_OPTIONAL=1 or CERT_REQUIRED=2 is needed for the cert parsing/populating to work (otherwise it just sets an empty dict)

For your self-signed certs, you should either import them into the OS certificate authority store or use the method SSLContext.load_verify_locations to load the self-signed cert (since it’s self-signed, you can load the server/leaf cert as the CA cert)

If you go the OS-store path, you can probably just use the default context. I think on Windows there’s a per-user CA store (not positive, though). I don’t think such a thing exists on Linux so you’d need to add it system-wide. Not sure about macOS or other operating systems