Python Beautiful Soup and Dealing with a Non-Existent Table (for Dynamic Webpages)

Question

I’m using the following commands to successfully scrape data from a table on multiple webpages through an iteration function:

 Sales = soup.find('table', id="tblSales")
 SaleData = Sales.find_all('tr')
    for tr in SaleData:
        td = tr.find_all('td')
        SaleData = [i.text for i in td]
        print(SaleData)

The issue is that sometimes the table doesn’t exist on the page (there are no "sales" so there is no table inserted into the page). So python throws an error and stops. When the table doesn’t exist, the webpage has the following html:

<div>
<span id="ucSaleInfo_lblNoSales">
    <p style="text-align:center"><em>No Sales Information Available</em></p>
</span>                                            
</div>

Is there a way to deal with non-existent tables in the soup?

Asked By: Humbaba

||

Source

Answer 1

The simplest way is to just check if Sales exists

if Sales := soup.find('table', id="tblSales"):
    SaleData = Sales.find_all('tr')
        for tr in SaleData:
            td = tr.find_all('td')
            SaleData = [i.text for i in td]
            print(SaleData)

OR (essentially same thing)

Sales = soup.find('table', id="tblSales")
if Sales:
    SaleData = Sales.find_all('tr')
        for tr in SaleData:
            td = tr.find_all('td')
            SaleData = [i.text for i in td]
            print(SaleData)

Answered By: OneMadGypsy

Python Beautiful Soup and Dealing with a Non-Existent Table (for Dynamic Webpages)

Question:

Answers: