Why do I get [SSL: CERTIFICATE_VERIFY_FAILED] in Python when ssl setup looks OK?

Question:

I’m working on a Python app to comunicate with a service running on localhost via secure websocket protocol.
Here is a sample code:

import json
import asyncio
import websockets
import ssl
import certifi


ssl_context = ssl.create_default_context()
ssl_context.load_verify_locations(certifi.where())
ssl_context.load_default_certs()

query =  {
    "jsonrpc": "2.0",
    "method": "queryHeadsets",
    "params": {},
    "id": 1
    }
json = json.dumps(query)

async def query(json):

    async with websockets.connect("wss://emotivcortex.com:54321") as ws:
        await ws.send(json)
        response = await ws.recv()
        print(response)

asyncio.get_event_loop().run_until_complete(query(json))

The problem is that the ssl handsake keeps failing with the following error:

ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)

I’m running Windows 10, Python 3.7.3 64-bit

$pip list
Package    Version
---------- --------
certifi    2019.3.9
pip        19.0.3
setuptools 40.8.0
websockets 7.0

I have inspected the certificate presented by the service. It seems valid and signed by COMODO.
I’ve tried:

ssl_context = ssl.create_default_context()
ssl_context.load_verify_locations(certifi.where())
print(ssl.get_default_verify_paths())
print(ssl_context.cert_store_stats())
ssl_context.load_default_certs()
print(ssl_context.get_ca_certs())

and found out that there are several COMODO CA certificates avaible for python. Yet I still get the error.

Here is the full error message if it helps:

SSL handshake failed on verifying the certificate
protocol: <asyncio.sslproto.SSLProtocol object at 0x0000020C11283048>
transport: <_SelectorSocketTransport fd=508 read=polling write=<idle, bufsize=0>>
Traceback (most recent call last):
  File "C:UsersMatyas2Pythonlibasynciosslproto.py", line 625, in _on_handshake_complete
    raise handshake_exc
  File "C:UsersMatyas2Pythonlibasynciosslproto.py", line 189, in feed_ssldata
    self._sslobj.do_handshake()
  File "C:UsersMatyas2Pythonlibssl.py", line 763, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)
SSL error in data received
protocol: <asyncio.sslproto.SSLProtocol object at 0x0000020C11283048>
transport: <_SelectorSocketTransport closing fd=508 read=idle write=<idle, bufsize=0>>
Traceback (most recent call last):
  File "C:UsersMatyas2Pythonlibasynciosslproto.py", line 526, in data_received
    ssldata, appdata = self._sslpipe.feed_ssldata(data)
  File "C:UsersMatyas2Pythonlibasynciosslproto.py", line 189, in feed_ssldata
    self._sslobj.do_handshake()
  File "C:UsersMatyas2Pythonlibssl.py", line 763, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)
Traceback (most recent call last):
  File "test.py", line 37, in <module>
    asyncio.get_event_loop().run_until_complete(query(json))
  File "C:UsersMatyas2Pythonlibasynciobase_events.py", line 584, in run_until_complete
    return future.result()
  File "test.py", line 32, in query
    async with websockets.connect("wss://emotivcortex.com:54321") as ws:
  File "C:UsersMatyas2Pythonlibsite-packageswebsocketspy35client.py", line 2, in __aenter__
    return await self
  File "C:UsersMatyas2Pythonlibsite-packageswebsocketspy35client.py", line 12, in __await_impl__
    transport, protocol = await self._creating_connection
  File "C:UsersMatyas2Pythonlibasynciobase_events.py", line 986, in create_connection
    ssl_handshake_timeout=ssl_handshake_timeout)
  File "C:UsersMatyas2Pythonlibasynciobase_events.py", line 1014, in _create_connection_transport
    await waiter
  File "C:UsersMatyas2Pythonlibasynciosslproto.py", line 526, in data_received
    ssldata, appdata = self._sslpipe.feed_ssldata(data)
  File "C:UsersMatyas2Pythonlibasynciosslproto.py", line 189, in feed_ssldata
    self._sslobj.do_handshake()
  File "C:UsersMatyas2Pythonlibssl.py", line 763, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)

SSL conection to servers in the Internet works fine.
What am I missing?
What am I doing wrong?

I will gladly provide aditional information if needed.

EDIT:
The certificate is for emotivcortex.com and issued by COMODO RSA Domain Validation Secure Server CA, so I assume it’s not a self-signed certificate.
OpenSSL:

$python -c "import ssl; print(ssl.OPENSSL_VERSION)"
OpenSSL 1.1.0j  20 Nov 2018
Asked By: digitalchemy

||

Answers:

The problem is caused by a missing intermediate CA certificate.

By inspecting the certificate presented by the service in OpenSSL I found out that the certificate was issued by "COMODO RSA Domain Validation Secure Server CA". The CA certificate of this particular authority is actually not present in the CA bundle of the python package certifi (there are different COMODO… certificates).

Solution

Manually download the missing certificate in PEM format from the CA’s webpage and add it to the CA bundle used in your code.

Also, there is a mistake in the app code:
When calling the function websockets.connect(), pass a keyword argument ssl=ssl_context so the CA bundle specified earlier is actually used.
The correct code looks like this:

import json
import asyncio
import websockets
import ssl
import certifi


ssl_context = ssl.create_default_context()
ssl_context.load_verify_locations(certifi.where())


query =  {
    "jsonrpc": "2.0",
    "method": "queryHeadsets",
    "params": {},
    "id": 1
    }
json = json.dumps(query)

async def query(json):
    
    async with websockets.connect("wss://emotivcortex.com:54321", ssl=ssl_context) as ws:
        await ws.send(json)
        response = await ws.recv()
        print(response)

asyncio.get_event_loop().run_until_complete(query(json))

Many thanks to larsks and Steffen Ullrich for pointing me in the right direction.

Answered By: digitalchemy