Cartopy can't download shapefiles because a certificate is not trusted
Question:
I need to use cartopy to plot a thing.
Running the example from their website
import cartopy.crs as ccrs
import matplotlib.pyplot as plt
ax = plt.axes(projection=ccrs.PlateCarree())
ax.coastlines()
plt.show()
I get a wonderfully long error related to a certificate that can’t be verified when cartopy tries to download a shape file. (See the errors below and ignore the file paths. I removed the stuff specific to my hard drive. It’s in a virtual environment).
If I go to the URL it is trying to download (https://naciscdn.org/naturalearth/110m/physical/ne_110m_coastline.zip), I can download the zip file. Inspecting the cert shows that it is distributed by Let’s Encrypt and is valid for several more months. After I got the error, I even went and downloaded the certificate, added it to my system key chain and set the trust level to always trust.
I’m using Python 3.7.3 on macOS Catalina 10.15.4.
Anyone know how I could fix this?
/env/lib/python3.7/site-packages/cartopy/io/__init__.py:260: DownloadWarning: Downloading: https://naciscdn.org/naturalearth/110m/physical/ne_110m_coastline.zip
warnings.warn('Downloading: {}'.format(url), DownloadWarning)
Traceback (most recent call last):
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 1317, in do_open
encode_chunked=req.has_header('Transfer-encoding'))
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/http/client.py", line 1229, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/http/client.py", line 1275, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/http/client.py", line 1224, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/http/client.py", line 1016, in _send_output
self.send(msg)
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/http/client.py", line 956, in send
self.connect()
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/http/client.py", line 1392, in connect
server_hostname=server_hostname)
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/ssl.py", line 412, in wrap_socket
session=session
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/ssl.py", line 853, in _create
self.do_handshake()
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/ssl.py", line 1117, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/env/lib/python3.7/site-packages/matplotlib/backends/backend_macosx.py", line 74, in _draw
self.figure.draw(renderer)
File "/env/lib/python3.7/site-packages/matplotlib/artist.py", line 38, in draw_wrapper
return draw(artist, renderer, *args, **kwargs)
File "/env/lib/python3.7/site-packages/matplotlib/figure.py", line 1736, in draw
renderer, self, artists, self.suppressComposite)
File "/env/lib/python3.7/site-packages/matplotlib/image.py", line 137, in _draw_list_compositing_images
a.draw(renderer)
File "/env/lib/python3.7/site-packages/matplotlib/artist.py", line 38, in draw_wrapper
return draw(artist, renderer, *args, **kwargs)
File "/env/lib/python3.7/site-packages/cartopy/mpl/geoaxes.py", line 479, in draw
return matplotlib.axes.Axes.draw(self, renderer=renderer, **kwargs)
File "/env/lib/python3.7/site-packages/matplotlib/artist.py", line 38, in draw_wrapper
return draw(artist, renderer, *args, **kwargs)
File "/env/lib/python3.7/site-packages/matplotlib/axes/_base.py", line 2630, in draw
mimage._draw_list_compositing_images(renderer, self, artists)
File "/env/lib/python3.7/site-packages/matplotlib/image.py", line 137, in _draw_list_compositing_images
a.draw(renderer)
File "/env/lib/python3.7/site-packages/matplotlib/artist.py", line 38, in draw_wrapper
return draw(artist, renderer, *args, **kwargs)
File "/env/lib/python3.7/site-packages/cartopy/mpl/feature_artist.py", line 155, in draw
geoms = self._feature.intersecting_geometries(extent)
File "/env/lib/python3.7/site-packages/cartopy/feature/__init__.py", line 302, in intersecting_geometries
return super(NaturalEarthFeature, self).intersecting_geometries(extent)
File "/env/lib/python3.7/site-packages/cartopy/feature/__init__.py", line 110, in intersecting_geometries
return (geom for geom in self.geometries() if
File "/env/lib/python3.7/site-packages/cartopy/feature/__init__.py", line 286, in geometries
name=self.name)
File "/env/lib/python3.7/site-packages/cartopy/io/shapereader.py", line 295, in natural_earth
return ne_downloader.path(format_dict)
File "/env/lib/python3.7/site-packages/cartopy/io/__init__.py", line 222, in path
result_path = self.acquire_resource(target_path, format_dict)
File "/env/lib/python3.7/site-packages/cartopy/io/shapereader.py", line 350, in acquire_resource
shapefile_online = self._urlopen(url)
File "/env/lib/python3.7/site-packages/cartopy/io/__init__.py", line 261, in _urlopen
return urlopen(url)
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 525, in open
response = self._open(req, data)
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 543, in _open
'_open', req)
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 1360, in https_open
context=self._context, check_hostname=self._check_hostname)
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 1319, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)>
Answers:
I had a very similar problem. I solved it following this post.
In my case, I did not need root privileges as I was using a conda environment.
So I got the directory with certificates:
$ dirname $(echo $REQUESTS_CA_BUNDLE)
/etc/ssl/certs
And the directory in which to create the link from openssl_capath from:
$ python -c 'import ssl; print(ssl.get_default_verify_paths())'
DefaultVerifyPaths(cafile='/home/<user>/miniconda3/envs/<env_name>/ssl/cert.pem', capath='/home/<user>/miniconda3/envs/<env_name>/ssl/certs', openssl_cafile_env='SSL_CERT_FILE', openssl_cafile='/home/<user>/miniconda3/envs/<env_name>/ssl/cert.pem', openssl_capath_env='SSL_CERT_DIR', openssl_capath='/home/<user>/miniconda3/envs/<env_name>/ssl/certs')
So the solution was:
ln -s /etc/ssl/certs/ /home/<user>/miniconda3/envs/<env_name>/ssl/
(replace with your user and environment name).
(Ubuntu 20.04.3 LTS, Python 3.10.2, cartopy 0.20.2, Spyder IDE 5.2.2)
I had this problem in Jupyter Hub, where my conda environment variables were, for some reason, not properly initialized.
I used this workaround to set Env vars in the notebook, relative to the currently activated conda environment:
import sys, os
from pathlib import Path
env_base_path = Path(sys.executable).parents[1]
os.environ["SSL_CERT_FILE"] =
str(env_base_path / 'ssl' / 'cert.pem')
os.environ["SSL_CERT_DIR"] =
str(env_base_path / 'ssl' / 'certs')
os.environ["REQUESTS_CA_BUNDLE"] =
str(env_base_path / 'ssl' / 'cert.pem')
I need to use cartopy to plot a thing.
Running the example from their website
import cartopy.crs as ccrs
import matplotlib.pyplot as plt
ax = plt.axes(projection=ccrs.PlateCarree())
ax.coastlines()
plt.show()
I get a wonderfully long error related to a certificate that can’t be verified when cartopy tries to download a shape file. (See the errors below and ignore the file paths. I removed the stuff specific to my hard drive. It’s in a virtual environment).
If I go to the URL it is trying to download (https://naciscdn.org/naturalearth/110m/physical/ne_110m_coastline.zip), I can download the zip file. Inspecting the cert shows that it is distributed by Let’s Encrypt and is valid for several more months. After I got the error, I even went and downloaded the certificate, added it to my system key chain and set the trust level to always trust.
I’m using Python 3.7.3 on macOS Catalina 10.15.4.
Anyone know how I could fix this?
/env/lib/python3.7/site-packages/cartopy/io/__init__.py:260: DownloadWarning: Downloading: https://naciscdn.org/naturalearth/110m/physical/ne_110m_coastline.zip
warnings.warn('Downloading: {}'.format(url), DownloadWarning)
Traceback (most recent call last):
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 1317, in do_open
encode_chunked=req.has_header('Transfer-encoding'))
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/http/client.py", line 1229, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/http/client.py", line 1275, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/http/client.py", line 1224, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/http/client.py", line 1016, in _send_output
self.send(msg)
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/http/client.py", line 956, in send
self.connect()
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/http/client.py", line 1392, in connect
server_hostname=server_hostname)
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/ssl.py", line 412, in wrap_socket
session=session
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/ssl.py", line 853, in _create
self.do_handshake()
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/ssl.py", line 1117, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/env/lib/python3.7/site-packages/matplotlib/backends/backend_macosx.py", line 74, in _draw
self.figure.draw(renderer)
File "/env/lib/python3.7/site-packages/matplotlib/artist.py", line 38, in draw_wrapper
return draw(artist, renderer, *args, **kwargs)
File "/env/lib/python3.7/site-packages/matplotlib/figure.py", line 1736, in draw
renderer, self, artists, self.suppressComposite)
File "/env/lib/python3.7/site-packages/matplotlib/image.py", line 137, in _draw_list_compositing_images
a.draw(renderer)
File "/env/lib/python3.7/site-packages/matplotlib/artist.py", line 38, in draw_wrapper
return draw(artist, renderer, *args, **kwargs)
File "/env/lib/python3.7/site-packages/cartopy/mpl/geoaxes.py", line 479, in draw
return matplotlib.axes.Axes.draw(self, renderer=renderer, **kwargs)
File "/env/lib/python3.7/site-packages/matplotlib/artist.py", line 38, in draw_wrapper
return draw(artist, renderer, *args, **kwargs)
File "/env/lib/python3.7/site-packages/matplotlib/axes/_base.py", line 2630, in draw
mimage._draw_list_compositing_images(renderer, self, artists)
File "/env/lib/python3.7/site-packages/matplotlib/image.py", line 137, in _draw_list_compositing_images
a.draw(renderer)
File "/env/lib/python3.7/site-packages/matplotlib/artist.py", line 38, in draw_wrapper
return draw(artist, renderer, *args, **kwargs)
File "/env/lib/python3.7/site-packages/cartopy/mpl/feature_artist.py", line 155, in draw
geoms = self._feature.intersecting_geometries(extent)
File "/env/lib/python3.7/site-packages/cartopy/feature/__init__.py", line 302, in intersecting_geometries
return super(NaturalEarthFeature, self).intersecting_geometries(extent)
File "/env/lib/python3.7/site-packages/cartopy/feature/__init__.py", line 110, in intersecting_geometries
return (geom for geom in self.geometries() if
File "/env/lib/python3.7/site-packages/cartopy/feature/__init__.py", line 286, in geometries
name=self.name)
File "/env/lib/python3.7/site-packages/cartopy/io/shapereader.py", line 295, in natural_earth
return ne_downloader.path(format_dict)
File "/env/lib/python3.7/site-packages/cartopy/io/__init__.py", line 222, in path
result_path = self.acquire_resource(target_path, format_dict)
File "/env/lib/python3.7/site-packages/cartopy/io/shapereader.py", line 350, in acquire_resource
shapefile_online = self._urlopen(url)
File "/env/lib/python3.7/site-packages/cartopy/io/__init__.py", line 261, in _urlopen
return urlopen(url)
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 525, in open
response = self._open(req, data)
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 543, in _open
'_open', req)
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 1360, in https_open
context=self._context, check_hostname=self._check_hostname)
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/urllib/request.py", line 1319, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)>
I had a very similar problem. I solved it following this post.
In my case, I did not need root privileges as I was using a conda environment.
So I got the directory with certificates:
$ dirname $(echo $REQUESTS_CA_BUNDLE)
/etc/ssl/certs
And the directory in which to create the link from openssl_capath from:
$ python -c 'import ssl; print(ssl.get_default_verify_paths())'
DefaultVerifyPaths(cafile='/home/<user>/miniconda3/envs/<env_name>/ssl/cert.pem', capath='/home/<user>/miniconda3/envs/<env_name>/ssl/certs', openssl_cafile_env='SSL_CERT_FILE', openssl_cafile='/home/<user>/miniconda3/envs/<env_name>/ssl/cert.pem', openssl_capath_env='SSL_CERT_DIR', openssl_capath='/home/<user>/miniconda3/envs/<env_name>/ssl/certs')
So the solution was:
ln -s /etc/ssl/certs/ /home/<user>/miniconda3/envs/<env_name>/ssl/
(replace with your user and environment name).
(Ubuntu 20.04.3 LTS, Python 3.10.2, cartopy 0.20.2, Spyder IDE 5.2.2)
I had this problem in Jupyter Hub, where my conda environment variables were, for some reason, not properly initialized.
I used this workaround to set Env vars in the notebook, relative to the currently activated conda environment:
import sys, os
from pathlib import Path
env_base_path = Path(sys.executable).parents[1]
os.environ["SSL_CERT_FILE"] =
str(env_base_path / 'ssl' / 'cert.pem')
os.environ["SSL_CERT_DIR"] =
str(env_base_path / 'ssl' / 'certs')
os.environ["REQUESTS_CA_BUNDLE"] =
str(env_base_path / 'ssl' / 'cert.pem')