Webscrape non-unique tags

Question

I am trying to scrape (https://slashdot.org/software/p/MyCase/) for

"$39.00/month/user" and the "
Basic Plan: $39 per user/month (billed annually) or $49 per user/month (billed monthly)
Pro Plan: $59 per user/month (billed annually) or $69 per user/month (billed monthly)
Advanced Plan: $79 per user/month (billed annually) or $89 per user/month (billed monthly"; however, I don’t know how to grab them as the tag for them both (and others) is the same: "div" class="field-value".

How would I grab these separately with beautifulsoup.

Asked By: BQuist

||

Source

Answer 1

You can grab the required tags by the text of the label (for example, there are many ways):

import requests
from bs4 import BeautifulSoup


url = "https://slashdot.org/software/p/MyCase/"
soup = BeautifulSoup(requests.get(url).content, "html.parser")

pricing_1 = soup.select_one(
    '.field-label:-soup-contains("Pricing Starts At") + .field-value'
)
pricing_2 = soup.select_one(
    '.field-label:-soup-contains("Pricing Information") + .field-value'
)

print(pricing_1.text)
print(pricing_2.text)

Prints:

$39.00/month/user
Basic Plan: $39 per user/month (billed annually) or $49 per user/month (billed monthly)
Pro Plan: $59 per user/month (billed annually) or $69 per user/month (billed monthly)
Advanced Plan: $79 per user/month (billed annually) or $89 per user/month (billed monthly

Answered By: Andrej Kesely

Webscrape non-unique tags

Question:

Answers: