Beautiful Soup Error pulling back data | Discord.py

Question

I am relatively new when it comes to BS4 but I am trying to create a discord bot that pulls back the information for this page but I am struggling with inputing the right data path. Currently when I run my command I am getting back this error: AttributeError: 'list' object has no attribute 'find_all' if anyone has any insight on what I might be doing wrong that would be great.

Thanks

@bot.slash_command(name="dailyreading",description="Shows Daily Reading",guild_ids=[testingServerID])
async def dailyreading(ctx):
 embed=Embed(title="Daily Reading".format(ctx), description="https://bible.usccb.org/daily-bible-reading")
 page = requests.get("https://bible.usccb.org/daily-bible-reading")
 soup = BeautifulSoup(page.content, 'html.parser')
 reading = soup.find("h3",class_="content-body")
 readingtext = soup.find("div",class_="content-body")
 reading=[]
 readingtext=[]
 for i in reading.find_all("a"):
   reading.append(i.get_text(strip=True,separator=" "))
   embed.add_field(name="Reading:", value=reading,inline=False)
 for s in readingtext.find_all("a"):
   readingtext.append(s.get_text(strip=True,separator=" "))
   embed.add_field(name="Context:", value=readingtext, inline=False)
 await ctx.send(embed=embed)

Asked By: Jinxed

||

Source

Answer 1

Main issue in my opinion is, that you first set:

reading = soup.find("h3",class_="content-body")
readingtext = soup.find("div",class_="content-body")

and then directly afterwards to empty lists:

reading=[]
readingtext=[]

Better rename your lists that should store your results:

reading_list=[]
readingtext_list=[]

or skip the variables in for-loops and iterate ResultSet directly:

reading=[]
readingtext=[]

for i in soup.select('h3.content-body a'):
    reading.append(i.get_text(strip=True,separator=" "))
    embed.add_field(name="Reading:", value=reading,inline=False)

for s in soup.select('div.content-body a'):
    readingtext.append(s.get_text(strip=True,separator=" "))
    embed.add_field(name="Context:", value=readingtext, inline=False)

Answered By: HedgeHog

Answer 2

You are getting AttributeError: ‘list’ object has no attribute ‘find_all because

reading = soup.find("h3",class_="content-body")

and

readingtext = soup.find("div",class_="content-body")

both selection select single element and after that you are iterating over them but your previous selection is not a list that’s why you are getting such errors.

Example:

page = requests.get("https://bible.usccb.org/daily-bible-reading")
soup = BeautifulSoup(page.content, 'html.parser')

reading=[]
readingtext=[]
for i in soup.select("div.address a"):
    reading.append(i.get_text(strip=True))

for j in soup.select("div.content-body p"):
    readingtext.append(j.get_text(strip=True))


print(reading)

print(readingtext)

Output:

['EZ 28:1-10', 'DEUTERONOMY 32:26-27AB, 27CD-28, 30, 35CD-36AB', '2 COR 8:9', 'MT 19:23-30']

['The word of the LORD came to me: Son of man,say to the prince of Tyre:Thus says the Lord GOD:', 'Because you are haughty of heart,you say, “A god am I!I occupy a godly thronein the heart of the sea!”—And yet you are a man, and not a god,however you may think yourself like a god.Oh yes, you are wiser than Daniel,there is no secret that is beyond you.By your wisdom and your intelligenceyou have made riches for yourself;You have put gold and silverinto your treasuries.By your great wisdom applied to your tradingyou have heaped up your riches;your heart has grown haughty from your riches–therefore thus says the Lord GOD:Because you have thought yourselfto have the mind of a god,Therefore I will bring against youforeigners, the most barbarous of nations.They shall draw their swordsagainst your beauteous 
wisdom,they shall run them through your splendid apparel.They shall thrust you down to the pit, there to diea bloodied corpse, in the heart of the sea.Will you then say, “I am a god!”when you face your murderers?No, you are man, not a god,handed over to those who will slay you.You shall die the death of the uncircumcisedat the hands of foreigners,for I have spoken, says the Lord GOD.', '', '']

Answered By: F.Hoque

Beautiful Soup Error pulling back data | Discord.py

Question:

Answers: