Python Web Scraping same class

Question

I am looking for some help, so i am pretty bad at web scraping, i am still learning the basics and stuff. So i am developing a application, you can put your question in the app, and it will fetch the answer(s) from google and return/print() the answer(s). So when you enter a question in google like "what is a letter?" google returns two explanations:

a character representing one or more of the sounds used in speech; any of the symbols of an alphabet.
"a capital letter"
a written, typed, or printed communication, sent in an envelope by post or messenger.
"he sent a letter to Mrs Falconer"

now… Both got the same class when inspecting the element. Which makes it impossible to print() both explanations out. Because when i enter the class, which both explanations are having, it only prints out the first(1.) one, which i don’t really understand, and is there any way to print both out even though they are having the same class?
Here is my code:

import requests
from bs4 import BeautifulSoup

search = input("Search: ")
URL = "https://www.google.co.in/search?q=" + search

headers = {
'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36 Edg/89.0.774.57'
}

page = requests.get(URL, headers=headers)
soup = BeautifulSoup(page.content, 'html.parser') 
result = soup.find(class_="LTKOO sY7ric").get_text() 
print(result)

Asked By: Pete

||

Source

Answer 1

This will give you all the text of those classes.

txts = [ x.get_text() for x in soup.find_all(class_="LTKOO sY7ric")]
print(txts)

Answered By: Dean Van Greunen

Answer 2

You can just run a for loop iterating over soup checking for every element with the required class name, then print out the text from the class

for(ele in soup.find_all(class_="LTKOO sY7ric")):
    print(ele.get_text())

Answered By: yashsh

Answer 3

The loop will help you extract all possible values:

import requests
from bs4 import BeautifulSoup

search = input("Search: ")
URL = "https://www.google.co.in/search?q=" + search

headers = {
'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36 Edg/89.0.774.57'
}

page = requests.get(URL, headers=headers)
soup = BeautifulSoup(page.content, 'html.parser') 


results = []

for ele in soup.find_all(class_="LTKOO sY7ric"):
    try:
        result = ele.find(class_="LTKOO sY7ric").text.strip()
        
    except AttributeError:
        result = 'no  data'
        
    results.append(result)
        
print(results)

I hope this helps.

Answered By: Farid Mammadaliyev

Python Web Scraping same class

Question:

Answers: