Getting None value after I put '.text' in 'find_all' funtion even though I set the tag and class correctly

Question

Hello I am new to the field of Data science & currently I am doing project of web scraping. I comes to a problem where I getting error of ‘none type object has no attribute text’.I just print out the funtion output without the ‘.text’ getting none value as output anyone please check the code and see where I am going wrong I already put tag and class accordingly.

import requests
from bs4 import BeautifulSoup
from csv import writer
url = 'https://www.99acres.com/search/property/buy/pune? city=19&preference=S&area_unit=1&res_com=R'
R = requests.get(url)
soup = BeautifulSoup(R.content, 'html.parser')
lists = soup.find_all('table', class_='srpTuple__tableFSL')

for list in lists:
   title = list.find('h2', class_='srpTuple__tupleTitleOverflow').text
   Location = list.find('a', class_='srpTuple__dFlex').text
   price = list.find('td', class_ = 'srpTuple__col title_semiBold ')
   info = [title,Location,price]
   print(info)

Here is the Output :

['2 BHK Apartment in Moshi', 'Flower City', None]
['3 BHK Apartment in Kondhwa', 'Mannat Towers', None]

Asked By: Syed

||

Source

Answer 1

Your search for the price field is not finding a result, you should search directly by the <td id=''> field as it will be more accurate.

You won’t be able to call .text against a find() that fails to find an object as it will return None.

import requests
from bs4 import BeautifulSoup
from csv import writer
url = 'https://www.99acres.com/search/property/buy/pune?city=19&preference=S&area_unit=1&res_com=R'
R = requests.get(url)
soup = BeautifulSoup(R.content, 'html.parser')
lists = soup.find_all('table', class_='srpTuple__tableFSL')

for l in lists:
   title = l.find('h2', class_='srpTuple__tupleTitleOverflow').text
   Location = l.find('a', class_='srpTuple__dFlex').text
   price = l.find('td', id='srp_tuple_price').text
   info = [title,Location,price]
   print(info)

['2 BHK Apartment in Kiwale', 'Maithili Square', '₹ 40.31  - 52.37 L']
['2 BHK Apartment in Lohegaon', 'Nivasa Enchante', '₹ 57.17  - 65.14 L']

If you suspect you may encounter times without a value, you can always check if find() found a result before calling .text.

   title = l.find('h2', class_='srpTuple__tupleTitleOverflow').text
   Location = l.find('a', class_='srpTuple__dFlex').text
   price = l.find('td', id='srp_tuple_price')

   if price:
     price = price.text
   else:
     price = 'N/A'
     
   info = [title,Location,price]
   print(info)

Answered By: PacketLoss

Getting None value after I put '.text' in 'find_all' funtion even though I set the tag and class correctly

Question:

Answers: