web scrapping issue for item availability

Question

I’m new to web scrapping and was trying to learn by starting small project of tracking availablity of an item from online store. Website itself
So far I have just was able to get class part of what I need to get, but not quite sure how to get the text from it for item Name, Availability and Price.
I’d like your advice on what is the least effort solution to get this information from it. Or should I just treat it like a string and slice until I get my desired three values?

my code

import requests from bs4 
import BeautifulSoup



productURL = "https://allbest.kz/p98634475-oneplus-9rt-12256gb.html"
result = requests.get(productURL)
doc = BeautifulSoup(result.text, "html.parser")
classname = doc.select("[class~=b-sticky-panel__holder]")
print(classname)

Output:

[<div class="b-sticky-panel__holder"><span class="b-sticky-panel__product-name" data-qaid="sticky_product_name">OnePlus 9RT 12/256Gb 5G Black</span><span class="b-sticky-panel__product-status" data-qaid="product_status_sticky_panel" title="Нет в наличии">Нет в наличии</span><div class="b-sticky-panel__cost" data-qaid="product_price_sticky_panel"><span class="b-sticky-panel__price">213 400 <span class="notranslate">Тг.</span></span></div></div>]

FYI: Here in the output "Нет в наличии" basically means "Out of stock".

Desired result:

Name = "OnePlus 9RT 12/256Gb 5G Black"
Availability = "Out of stock"
Price = 213 400

Asked By: Nurbek Kuantyrov

||

Source

Answer 1

select returns a list of elements based on your selector, so you had a list with one element, and such you index the first element as you only have one. followed by selecting again to find the children tags that contain the information you need. Something like this.
You can ignore looking for the panel holder all together and just go directly for the tags that contain the information you need.

productURL = "https://allbest.kz/p98634475-oneplus-9rt-12256gb.html"
result = requests.get(productURL)
doc = BeautifulSoup(result.text, "html.parser")

name = doc.select("[class~=b-sticky-panel__product-name]")[0].text
availability = doc.select("[class~=b-sticky-panel__product-status]")[0].text
price = doc.select("[class~=b-sticky-panel__price]")[0].text

print(f"Name = {name}")
print(f"Availability = {availability}")
print(f"Price = {price}")

Output:

Name = OnePlus 9RT 12/256Gb 5G Black
Availability = Нет в наличии
Price = 213 400 Тг.

Answered By: DeltaHaxor

Answer 2

The following code snippet will get you the info you’re looking for:

import requests
from bs4 import BeautifulSoup

headers= {'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36'}

url = 'https://allbest.kz/p98634475-oneplus-9rt-12256gb.html'
r = requests.get(url, headers=headers)
soup = BeautifulSoup(r.text, 'html.parser')

availability = soup.select_one('li[data-qaid="presence_data"]').get_text(strip=True)
name = soup.select_one('span[data-qaid="product_name"]').get_text(strip=True)
price = soup.select_one('span[data-qaid="product_price"]').get_text(strip=True)
print('Name =', name)
print('Availability =', availability)
print('Price =', price)

This will print out in terminal:

Name = OnePlus 9RT 12/256Gb 5G Black
Availability = Нет в наличии
Price = 213 400

BeautifulSoup docs: https://www.crummy.com/software/BeautifulSoup/bs4/doc/#

Requests docs: https://requests.readthedocs.io/en/latest/

Answered By: platipus_on_fire

web scrapping issue for item availability

Question:

Answers: