urllib.requiest.urlopen error: certificate verify failed on python Virtual Environment
Question:
Im building a Web Scapper, when testing on venv -> [SSL: CERTIFICATE_VERIFY_FAILED]
But, when I’m testing on ipython shell -> Perfectly good
I wondering what the root problem is?
Thanks for your help!`from urllib.request import urlopen
from bs4 import BeautifulSoup
import subprocess
html = urlopen(‘http://www.pythonscraping.com/pages/page3.html’)
bs = BeautifulSoup(html, ‘lxml’)
html_tags = bs.prettify()`
Answers:
Try installing pip install certifi with your venv and add another valid certificate from any website with https:
Also this is how you add a certificate to certifi library:
I just understand why it could leads into this problems!
If your guys using an virtual env (such as: create a by using python3 -m venv .venv -> it will absolutely isolate (easy to understand right?)
But once it isolate from the current machine -> the ip address still not change!
However the dependencies with others are cut down already -> This could lead to the about situation!
but when your guys create a venv (by conda) -> it setting more detailed -> so it might by not crash into the problem.
If somethings wrong, im happy to get the replies for this one!
PS: Python Automation Cookbook – packs helped me to understand this problem deeply! and ChatGPT: network dependencies and lib in venv.
Im building a Web Scapper, when testing on venv -> [SSL: CERTIFICATE_VERIFY_FAILED]
But, when I’m testing on ipython shell -> Perfectly good
I wondering what the root problem is?
Thanks for your help!`from urllib.request import urlopen
from bs4 import BeautifulSoup
import subprocess
html = urlopen(‘http://www.pythonscraping.com/pages/page3.html’)
bs = BeautifulSoup(html, ‘lxml’)
html_tags = bs.prettify()`
Try installing pip install certifi with your venv and add another valid certificate from any website with https:
Also this is how you add a certificate to certifi library:
I just understand why it could leads into this problems!
If your guys using an virtual env (such as: create a by using python3 -m venv .venv -> it will absolutely isolate (easy to understand right?)
But once it isolate from the current machine -> the ip address still not change!
However the dependencies with others are cut down already -> This could lead to the about situation!
but when your guys create a venv (by conda) -> it setting more detailed -> so it might by not crash into the problem.
If somethings wrong, im happy to get the replies for this one!
PS: Python Automation Cookbook – packs helped me to understand this problem deeply! and ChatGPT: network dependencies and lib in venv.