Retrieving attribute values through iterations using BeautifulSoup

Question

I’m scraping an html saved on a file with the following code:

from bs4 import BeautifulSoup as bs

path_xml = r"..."

content = []

with open(path_xml, "r") as file:
    content = file.readlines()

content = "".join(content)
bs_content = bs(content, "html.parser")

bilder = bs_content.find_all("bilder")

def get_str_bild(match):
    test = match.findChildren("b")

    for x in range(len(test)): # here is the problem (not giving me all elements in test)
 
        return test[x].get("d")

for b in bilder:
    if b.b: 
        print(get_str_bild(b))

Output:

L3357U00_002120.jpg
L3357U00_002140.jpg
L3357U00_002160.jpg

Basically, there are 3 positions in the xml file where I have children of the node "bilder". Each block looks like this:

<Bilder>
    <B Nr="1" D="L3357U00_002120.jpg"/>
    <B Nr="2" D="L3357U00_002120.jpg"/>
    <B Nr="3" D="L3357U00_002120.jpg"/>
    <B Nr="4" D="L3357U00_002120.jpg"/>
    <B Nr="9" D="L3357U00_002120.jpg"/>
    <B Nr="1" D="L3357U00_002130.jpg"/>
    <B Nr="2" D="L3357U00_002130.jpg"/>
    <B Nr="3" D="L3357U00_002130.jpg"/>
    <B Nr="4" D="L3357U00_002130.jpg"/>
    <B Nr="9" D="L3357U00_002130.jpg"/>
</Bilder>

Currently it only returns the first picture of each block and I want to return all of them.

What am I doing wrong here?

Asked By: maggTech

||

Source

Answer 1

You need to fix get_str_bild(match) function. It currently returns the first d attribute.

Replace you function with this:

def get_str_bild(match):
    test = match.find_all("b")
    
    elements = []
    for x in range(len(test)):
        elements.append(test[x].get("d"))

    return elements

Answered By: Refet

Answer 2

You’re missing the cycle on bs of your bilders. You can remove your function and simplify your code as follows:

pic_1 = "L3357U00_002120.jpg"

bs_content = bs(content, "html.parser")
for i, builder in enumerate(bs_content.find_all("bilder")):
    print(f'builder {i}')
    for b in bilder.find_all('b'):
        if b['nr'] == pic_1:
            print(b['d'])
            #break

Answered By: lemon

Retrieving attribute values through iterations using BeautifulSoup

Question:

Answers: