How to extract price from web page using Beautiful Soup?

Question

import requests
from bs4 import BeautifulSoup

URL="https://shop.beobasta.rs/proizvod/smrznuti-spanac/"
header={"User Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36 OPR/105.0.0.0","Accept-Language":"en-US,en;q=0.9"}
response=requests.get(URL,headers=header)
soup=BeautifulSoup(response.text,'html.parser')
price_element = soup.find("span",class_="woocommerce-Price-amount amount")
print(price_element)

I’m trying to extract price from this website but the only thing I’m getting is None. What am I doing wrong?

Asked By: luna28blue

||

Source

Answer 1

If you check your soup with:

print(soup)
#outpout
<title>400 Bad Request</title>
</head><body>
<h1>Bad Request</h1>
<p>Your browser sent a request that this server could not understand.<br/>

This is because inrequests.get(URL,headers=header) your header is not compatible with that website.

If you for example would just use:

requests.get(URL) it would probably work already.

Answered By: tetris programming

Answer 2

It has to be User-Agent, not User Agent – and this makes all problem.

But code works also without User-Agent.

Or even with fake "User-Agent": "Mozilla/5.0"

Answered By: furas

How to extract price from web page using Beautiful Soup?

Question:

Answers: