beautifulsoup

How do I create a list from a webpage?

How do I create a list from a webpage? Question: I am attempting to create a list of words from the website text. I would like to be able to randomise the word that is produced from this list using random. I hope this makes sense. import random as r from bs4 import BeautifulSoup import …

Total answers: 2

BeautifulSoup .find() only works on certain links

BeautifulSoup .find() only works on certain links Question: I have made a very simple web scraper which iterates through a list of links and scrapes the text and dates from them and outputs this into a text file. So far everything has been working fine but I have received an error which only occurs for …

Total answers: 1

How to extract price from web page using Beautiful Soup?

How to extract price from web page using Beautiful Soup? Question: import requests from bs4 import BeautifulSoup URL="https://shop.beobasta.rs/proizvod/smrznuti-spanac/" header={"User Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36 OPR/105.0.0.0","Accept-Language":"en-US,en;q=0.9"} response=requests.get(URL,headers=header) soup=BeautifulSoup(response.text,’html.parser’) price_element = soup.find("span",class_="woocommerce-Price-amount amount") print(price_element) I’m trying to extract price from this website but the only thing I’m getting is None. …

Total answers: 2

Using Python with Beautifulsoup to find span class data-automation-id value

Using Python with Beautifulsoup to find span class data-automation-id value Question: Could someone please tell me how to use beautifulsoup to extract the value ($100.00) from a data-automation-id="header-account-balance"? I can see how to find elements by class (How to find elements by class) and id (Beautiful Soup and extracting a div and its contents by …

Total answers: 1

How to iteratively retrieve the right information from beautiful soup elements?

How to iteratively retrieve the right information from beautiful soup elements? Question: I try to retrieve information from EZB press releases. To do so I use BeautifulSoup. Since the structure (HTML) of the press releases is changing over time, it is difficult to retrieve the date of the press releases with a single selector. Hence …

Total answers: 2

Scraping using BeautifulSoup print an empty output

Scraping using BeautifulSoup print an empty output Question: I’m trying to scrape a website. I want to print all the elements with the following class name, class=product-size-info__main-label The code is the following: from bs4 import BeautifulSoup with open("MadeInItaly.html", "r") as f: doc= BeautifulSoup (f, "html.parser") tags = doc.find_all(class_="product-size-info__main-label") print(tags) Result: [XS, XS, S, M, L, …

Total answers: 2

python Error : AttributeError: 'NoneType' object has no attribute 'find_all'

python Error : AttributeError: 'NoneType' object has no attribute 'find_all' Question: I have a requirement to read a table from confluence page and store it into a Json or csv file. In order to achieve that requirement, I am using python scripting. When I ran my code and try to print the confluence page content, …

Total answers: 2

How can I scrape a specific URL from a webpage using BeautifulSoup?

How can I scrape a specific URL from a webpage using BeautifulSoup? Question: I’m writing a Python script that parses HTML (a classifieds website) and sends me email notifications on specific products and price points. Everything works here except for the "listing_url" capture, which I want displayed in the email so I can click on …

Total answers: 1

Web scraping reviews from Amazon only returns data for the first page

Web scraping reviews from Amazon only returns data for the first page Question: I am trying to scrape reviews from Amazon. The reviews can appear on multiple pages to scrape more than one page I construct a list of links which I later scrape separately: # Construct list of links to scrape multiple pages links …

Total answers: 3