handshake failed; returned -1, SSL error code 1, net_error -201
Question:
i am trying to do web scraping using python using selenium but whenever i run the code i am getting the
error
[4824:524:0818/154954.605:ERROR:ssl_client_socket_impl.cc(959)] handshake failed; returned -1, SSL error code 1, net_error -201
[4824:524:0818/154954.614:ERROR:ssl_client_socket_impl.cc(959)] handshake failed; returned -1, SSL error code 1, net_error -201
[4824:524:0818/154954.721:ERROR:ssl_client_socket_impl.cc(959)] handshake failed; returned -1, SSL error code 1, net_error -201
[4824:524:0818/154954.730:ERROR:ssl_client_socket_impl.cc(959)] handshake failed; returned -1, SSL error code 1, net_error -201
Empty DataFrame
Columns: [Rank, Country, Total Cases, New Cases, Deaths, New Deaths, Recovered, Active Cases, Critical]
Index: []
my code i am trying to use selenium to go at the website called worldometer and extract the data from the table present at their website using pandas. i have used selenium before to access other websites but it didn’t give error then. I am using python version 3.6.8
i tried the fixes like installing OpenSSl but it didn’t install
i also tried other fixes like adding –ignore-certificate-errors and –ignore-ssl-errors but that didn’t work also
import pandas as pd
import time
# Covid 19 Webscraper
browser = webdriver.Chrome('C:\webdrivers\chromedriver.exe')
# opening sites
browser.get("https://www.worldometers.info/coronavirus/")
time.sleep(15)
#creating Data Frame
df = pd.DataFrame(columns=['Rank','Country','Total Cases','New Cases','Deaths','New Deaths','Recovered','Active Cases','Critical'])
# finding xpath and info
for i in browser.find_elements_by_xpath("//*[@id='main_table_countries_today']/tbody/tr"):
td_list = i.find_elements_by_tag_name('td')
row = []
for td in td_list:
row.append(td.text)
data={}
for j in range(len(df.columns)):
data[df.columns[j]] = row[j]
df.append(data, ignore_index=True)
print(df)
Answers:
its look like your browser store don’t have certificate required by website. Please use chrome options as below:
options = webdriver.ChromeOptions()
options.add_argument("--ignore-certificate-error")
options.add_argument("--ignore-ssl-errors")
browser = webdriver.Chrome('C:\webdrivers\chromedriver.exe',options=options)
browser.get("https://www.worldometers.info/coronavirus/")
With Capabilites:
caps = webdriver.DesiredCapabilities.CHROME.copy()
caps['acceptInsecureCerts'] = True
caps['acceptSslCerts'] = True
driver = webdriver.Chrome(desired_capabilities=caps)
As mentioned in this answer:
The error is know to the [Chromium] developer team (or broadly speaking, it has been reported on multiple occasions).
So, as mentioned in this answer (Explanatory note: I changed the code to Python 3 and Selenium 4. The important line is marked with ), you can set Chromium log-level
to hide error:
You can restrict Chromium’s log level to 3, so that only fatal errors are logged. Please bear in mind that you won’t see any other error-related messages which might cause mayhem in production! The code looks like this:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
service_current = Service(r"C:pathtochromedriver.exe")
options_current = Options()
options_current.binary_location = r"C:pathtochrome.exe"
options_current.add_argument("log-level=3") #
browser = webdriver.Chrome(service=service_current, options=options_current)
Chromium log levels are:
Description
Value
INFO
0
WARNING
1
LOG_ERROR
2
LOG_FATAL
3
i am trying to do web scraping using python using selenium but whenever i run the code i am getting the
error
[4824:524:0818/154954.605:ERROR:ssl_client_socket_impl.cc(959)] handshake failed; returned -1, SSL error code 1, net_error -201
[4824:524:0818/154954.614:ERROR:ssl_client_socket_impl.cc(959)] handshake failed; returned -1, SSL error code 1, net_error -201
[4824:524:0818/154954.721:ERROR:ssl_client_socket_impl.cc(959)] handshake failed; returned -1, SSL error code 1, net_error -201
[4824:524:0818/154954.730:ERROR:ssl_client_socket_impl.cc(959)] handshake failed; returned -1, SSL error code 1, net_error -201
Empty DataFrame
Columns: [Rank, Country, Total Cases, New Cases, Deaths, New Deaths, Recovered, Active Cases, Critical]
Index: []
my code i am trying to use selenium to go at the website called worldometer and extract the data from the table present at their website using pandas. i have used selenium before to access other websites but it didn’t give error then. I am using python version 3.6.8
i tried the fixes like installing OpenSSl but it didn’t install
i also tried other fixes like adding –ignore-certificate-errors and –ignore-ssl-errors but that didn’t work also
import pandas as pd
import time
# Covid 19 Webscraper
browser = webdriver.Chrome('C:\webdrivers\chromedriver.exe')
# opening sites
browser.get("https://www.worldometers.info/coronavirus/")
time.sleep(15)
#creating Data Frame
df = pd.DataFrame(columns=['Rank','Country','Total Cases','New Cases','Deaths','New Deaths','Recovered','Active Cases','Critical'])
# finding xpath and info
for i in browser.find_elements_by_xpath("//*[@id='main_table_countries_today']/tbody/tr"):
td_list = i.find_elements_by_tag_name('td')
row = []
for td in td_list:
row.append(td.text)
data={}
for j in range(len(df.columns)):
data[df.columns[j]] = row[j]
df.append(data, ignore_index=True)
print(df)
its look like your browser store don’t have certificate required by website. Please use chrome options as below:
options = webdriver.ChromeOptions()
options.add_argument("--ignore-certificate-error")
options.add_argument("--ignore-ssl-errors")
browser = webdriver.Chrome('C:\webdrivers\chromedriver.exe',options=options)
browser.get("https://www.worldometers.info/coronavirus/")
With Capabilites:
caps = webdriver.DesiredCapabilities.CHROME.copy()
caps['acceptInsecureCerts'] = True
caps['acceptSslCerts'] = True
driver = webdriver.Chrome(desired_capabilities=caps)
As mentioned in this answer:
The error is know to the [Chromium] developer team (or broadly speaking, it has been reported on multiple occasions).
So, as mentioned in this answer (Explanatory note: I changed the code to Python 3 and Selenium 4. The important line is marked with ), you can set Chromium log-level
to hide error:
You can restrict Chromium’s log level to 3, so that only fatal errors are logged. Please bear in mind that you won’t see any other error-related messages which might cause mayhem in production! The code looks like this:
from selenium import webdriver from selenium.webdriver.chrome.options import Options from selenium.webdriver.chrome.service import Service service_current = Service(r"C:pathtochromedriver.exe") options_current = Options() options_current.binary_location = r"C:pathtochrome.exe" options_current.add_argument("log-level=3") # browser = webdriver.Chrome(service=service_current, options=options_current)
Chromium log levels are:
Description Value INFO 0 WARNING 1 LOG_ERROR 2 LOG_FATAL 3