Accessing Tables Using BeautifulSoup
Question:
from bs4 import BeautifulSoup
import requests
url_link = "https://en.wikipedia.org/wiki/List_of_states_and_territories_of_the_United_States"
source = requests.get(url_link).text
soup = BeautifulSoup(source, "lxml")
my_table = soup.find("table", class_ = "wikitable sortable plainrowheaders jquery-tablesorter")
from bs4 import BeautifulSoup
import requests
url_link = "https://en.wikipedia.org/wiki/List_of_states_and_territories_of_the_United_States"
source = requests.get(url_link).text
soup = BeautifulSoup(source, "lxml")
my_table = soup.find("table", class_ = "wikitable sortable plainrowheaders")
I was wondering whether anybody knows why the first block of code gives me a none object, but the second block of code returns what I want. Is it something to do with hyphenated class names? Thanks in advance:)!
Answers:
No table in the HTML source (do a view source) has the class jquery-tablesorter and thus the first find() does not return any hits. Remember that BeautifulSoup does not run client side javascript the way Selenium does and thus classes like jquery-tablesorter that are applied by javascript libraries (or any other javascript functionality) will not be present to BeautifulSoup.
from bs4 import BeautifulSoup
import requests
url_link = "https://en.wikipedia.org/wiki/List_of_states_and_territories_of_the_United_States"
source = requests.get(url_link).text
soup = BeautifulSoup(source, "lxml")
my_table = soup.find("table", class_ = "wikitable sortable plainrowheaders jquery-tablesorter")
from bs4 import BeautifulSoup
import requests
url_link = "https://en.wikipedia.org/wiki/List_of_states_and_territories_of_the_United_States"
source = requests.get(url_link).text
soup = BeautifulSoup(source, "lxml")
my_table = soup.find("table", class_ = "wikitable sortable plainrowheaders")
I was wondering whether anybody knows why the first block of code gives me a none object, but the second block of code returns what I want. Is it something to do with hyphenated class names? Thanks in advance:)!
No table in the HTML source (do a view source) has the class jquery-tablesorter and thus the first find() does not return any hits. Remember that BeautifulSoup does not run client side javascript the way Selenium does and thus classes like jquery-tablesorter that are applied by javascript libraries (or any other javascript functionality) will not be present to BeautifulSoup.