Web-scraping with NBA.com

Question

I am trying to scrape a page on NBA.com, but the table that I’m looking for can’t be found. How can I find this HTML content within a request. I’m trying to do this for study purposes.

import requests
from bs4 import BeautifulSoup

url = 'https://www.nba.com/stats/player/203076/boxscores-traditional?Season=2022-23'
data = requests.get(url)
info = "Davis"  

with open ('{}.html'.format(info), 'w+', encoding="utf-8") as f:
    f.write(data.text)

# We read the file we created
with open ("{}.html".format(info), encoding="utf-8") as f:
    page = f.read()

# We use the BeautifulSoup library 
soup = BeautifulSoup(page,  'html.parser')
stats = soup.find_all('table', class_='Crom_table__p1iZz')

I expected to see the table class Crom_table__p1iZz and its tr contents

Asked By: ivan

||

Source

Answer 1

It is not possible to retrieve the table you are looking for using request-based scraping, at least not the way you want to do it.

If you can find a request that the site does to fetch the data you want then it might be possible, however I have not checked out the site fully, you would have to do that on your own..

Otherwise, the table is loaded using Javascript, you will need to use something such as selenium or chrome webdriver.

Answered By: S. C.

Web-scraping with NBA.com

Question:

Answers: