beautifulsoup

How do I write all my BeautifulSoup data from a website to a text file? Python

How do I write all my BeautifulSoup data from a website to a text file? Python Question: I am trying to read data from open insider and put it into an easy to read text file. Here is my code so far: from bs4 import BeautifulSoup import requests page = requests.get("http://openinsider.com/top-insider-purchases-of-the-month") ”’print(page.status_code) checks to see …

Total answers: 1

How do I separate text after using BeautifulSoup in order to plot?

How do I separate text after using BeautifulSoup in order to plot? Question: I am trying to make a program that scrapes the data from open insider and take that data and plot it. Open insider shows what insiders of the company are buying or selling the stock. I want to be able to show, …

Total answers: 3

BeautifulSoup find a href in marquee

BeautifulSoup find a href in marquee Question: I’m using bs4 to scrape links from a scrolling marquee. I’m able to get the marquee data, which is returned as a bs4 resultSet element. However, I cannot seem to access the href’s within the data. I’m sure I’m missing something as I’m new to web scraping, and …

Total answers: 1

How to get the ID of an element with class name with BS4

How to get the ID of an element with class name with BS4 Question: I have a site where there are multiple li elements whos ID I need but I only have the class name. I also need the IDs to be put into a list The html: <ul class="price-list"> <li class="price-box" id="200"></li> <li class="price-box" …

Total answers: 2

BS4 not displaying text in Flask

BS4 not displaying text in Flask Question: I’m learning Python(Flask) and BeautifulSoup. For my first project I just wanted to wanted to get a video name from YT and display it on the homepage of my web app. An error returns: AttributeError: ‘NoneType’ object has no attribute ‘text’ import requests from flask import Blueprint, render_template …

Total answers: 1

reviews of a firm

reviews of a firm Question: My goal is to scrape the entire reviews of this firm. I tried manipulating @Driftr95 codes: def extract(pg): headers = {‘user-agent’ : ‘Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36′} url = f’https://www.glassdoor.com/Reviews/3M-Reviews-E446_P{pg}.htm?filter.iso3Language=eng’ # f’https://www.glassdoor.com/Reviews/Google-Engineering-Reviews-EI_IE9079.0,6_DEPT1007_IP{pg}.htm?sort.sortType=RD&sort.ascending=false&filter.iso3Language=eng’ r = requests.get(url, headers, timeout=(3.05, 27)) soup = BeautifulSoup(r.content, ‘html.parser’)# this …

Total answers: 1

How to dynamically find the nearest specific parent of a selected element?

How to dynamically find the nearest specific parent of a selected element? Question: I want to parse many html pages and remove a div that contains the text "Message", using beautifulsoup html.parser and python. The div has no name or id, so pointing to it is not possible. I am able to do this for …

Total answers: 1

Can't scrape table BeautifulSoup

Can't scrape table BeautifulSoup Question: I’m trying to scrape the following table from this URL: https://baseballsavant.mlb.com/leaderboard/outs_above_average?type=Fielder&startYear=2022&endYear=2022&split=no&team=&range=year&min=10&pos=of&roles=&viz=show This is my code: import requests from bs4 import BeautifulSoup url = "https://baseballsavant.mlb.com/leaderboard/outs_above_average?type=Fielder&startYear=2022&endYear=2022&split=no&team=&range=year&min=10&pos=of&roles=&viz=show" r = requests.get(url) soup = BeautifulSoup(r.content, "lxml") table = soup.find("table") for row in table.findAll("tr"): print([i.text for i in row.findAll("td")]) However, my variable table returns None, even …

Total answers: 1

How to get the a href link from under the div class? using beautiful soup

How to get the a href link from under the div class? using beautiful soup Question: I am trying to scrape the href attribute from links from a page, but I end up with [] as the output The HTML code is My desired output is: https://www.pigiame.co.ke/listings/nissan-latio-2016-36000-kms-5300124 Asked By: Abduls || Source Answers: You can …

Total answers: 1