How to make Python use CA certificates from Mac OS TrustStore?

This is also a problem in Python 3.6 with MacOS Sierrra. I know your use case is different. But I stumbled upon this thread while investigating this problem. So if anyone is also having this article is worth checking out:

http://www.cdotson.com/2017/01/sslerror-with-python-3-6-x-on-macos-sierra/

In a nutshell: Python 3.6 does not rely on MacOS’ openSSL anymore. It comes with its own openSSL bundled and doesn’t have access on MacOS’ root certificates.

You have two options:

Run an install command shipped with Python 3.6

cd /Applications/Python 3.6/
./Install Certificates.command

or

Install the certifi package with

pip install certifi

I chose the first option and it worked.

For me /Applications/Python 3.6/./Install Certificates command fails on pip certifi install. I am on mac High Sierra and use python3 so pip somewhat fails and I have to use pip3 instead.

So here what I did:

1. Manually ran pip3 install --update certify in a shell
2. Remove the install certifi line from the command script
3. Reran the script and everything was fine.

Note that you will end up with a cert.pem symbolic link in: /Library/Frameworks/Python.framework/Versions/3.6/etc/openssl/

As an update and datapoint, I ran into this issue running Python 3.7.0 on macOS 10.13.4:

$ ipython
Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 26 2018, 23:26:24)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.0.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import bokeh.sampledata

In [2]: bokeh.sampledata.download()
Using data directory: /Users/me/.bokeh/data

...
SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1045)

Instructions for solving the problem are in /Applications/Python 3.7/ReadMe.rtf

Following the suggestion there and running /Applications/Python 3.7/Install Certificates.command solved the problem:

From the terminal:

$ /Applications/Python 3.7/Install Certificates.command

Re-starting IPython…

$ ipython
>>> import bokeh.sampledata

>>> bokeh.sampledata.download()
Using data directory: /Users/me/.bokeh/data
Downloading: CGM.csv (1589982 bytes)
   1589982 [100.00%]
...

Mac brew install python env.

$ python3
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 16:52:21) 
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import certifi
>>> certifi.where()
'/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/certifi/cacert.pem'
>>> 

Or from the command line:

$ python -m certifi

then need link cacert.pem as cert.pem

$ ln -s /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/certifi/cacert.pem cert.pem
$ pwd
/Library/Frameworks/Python.framework/Versions/3.7/etc/openssl

rehash

then work fine.

Run this to set the appropriate variables. This is a combination of the answers that have already been given here. Put it in your ~/.bash_profile to make it permanent.

CERT_PATH=$(python -m certifi)
export SSL_CERT_FILE=${CERT_PATH}
export REQUESTS_CA_BUNDLE=${CERT_PATH}

I see a lot of answers out there recommend to turn off certificate validation or to use certifi.where.
While turning off SSL is obvious risk. certifi.where is also a risk, mainly if you intend to make this code a production code that will run in a customer env.

PEP describing why it is wrong.
ssl.create_default_context is well integrate with linux and windows trustStore. the problem is, as in your case with mac.
I solve this by loading the certificates using the integrated security commandline tool

def create_macos_ssl_context():
    import subprocess
    import ssl
    import tempfile
    ctx = ssl.create_default_context()
    macos_ca_certs = subprocess.run(["security", "find-certificate", "-a", "-p",
                                     "/System/Library/Keychains/SystemRootCertificates.keychain"],
                                    stdout=subprocess.PIPE).stdout

    with tempfile.NamedTemporaryFile('w+b') as tmp_file:
        tmp_file.write(macos_ca_certs)
        ctx.load_verify_locations(tmp_file.name)
    print(ctx.get_ca_certs())

Note that this gives you the systemRoot certificates. if you need the user than simply change the value in the security command

Solution for MacOS or Linux with latest Python versions installed either as standalone or via port or brew

Download certificates from Certifi project at https://github.com/certifi/python-certifi/blob/master/certifi/cacert.pem. FYI Certifi is a 3rd party library that provides Mozilla’s curated collection of Root Certificates for validating the trustworthiness of SSL certificates while verifying the identity of TLS hosts.

Then add to your ~/.zshrc on the latest MacOS or ~/.bash_profile or similar:

export SSL_CERT_FILE=/pathtodownloadedfile/cacert.pem
export REQUESTS_CA_BUNDLE=/pathtodownloadedfile/cacert.pem

This was intended as an edit to an existing question, but since the queue was already full, posting as a separate answer.

Tested on MacOS 12.3.1 with python 3.10 installed with MacPorts.

If you prefer to trust root CA according to your OS, export them from System Roots keychain into a single file:

security export -t certs -f pemseq -k /System/Library/Keychains/SystemRootCertificates.keychain -o bundleCA.pem

If, additionally, you want to trust some internal self-signed CAs, export them, too. They are probably stored under the System keychain:

security export -t certs -f pemseq -k /Library/Keychains/System.keychain -o selfSignedCAbundle.pem

Merge the two files:

cat bundleCA.pem selfSignedCAbundle.pem >> allCAbundle.pem

Export as bash variable

export REQUESTS_CA_BUNDLE=/path/to/allCAbundle.pem

Consider adding the last code snippet to your .bash_profile

Note that REQUESTS_CA_BUNDLE works only for a single file, and does not for a directory.

In my case only installing "Install Certificates.command" solved this issue using MAC OS

Update SSL certificate with certifi (MacOS only)

All we would have to do is to run command with the following piece of code:

- Press "command + space" button or open Spotlight
- type "Install Certificates.command" 

What this command does is update our system’s SSL certificate directory for MacOS.