Extracting data from a dictionary in bs4?

Question

My code:

import requests
from bs4 import BeautifulSoup

name = input("Player's name: ")
url = f'https://www.futwiz.com/en/searches/career22/{name}'

r = requests.get(url)
print(r.content)

soup = BeautifulSoup(r.text,'html.parser')
results = soup.find(attrs={"lineid"})
print(results)

I can’t get the data from the dictionary.
r.text looks like this:

b'[{"lineid":"438","cardname":" Hakimi","name":"Achraf
Hakimi","urlname":"achraf-hakimi","rating":"85","potential":"89","club":"73","pid":"235212","position":"RB","nation":"129","league":"16","acceleration":"93","agility":"84","balance":"79","jumping":"80","reactions":"84","sprintspeed":"95","stamina":"93","strength":"74","aggression":"75","positioning":"79","tactaware":"78","vision":"78","ballcontrol":"83","crossing":"86","curve":"77","dribbling":"81","finishing":"75","fkacc":"74","headingacc":"68","longpass":"73","longshot":"73","marking":"75","penalties":"60","shortpass":"81","shotpower":"80","slidetackle":"76","standingtackle":"78","volleys":"74","fname":"Achraf","lname":"Hakimi","dob":"22","height":"181","foot":"Right","traits":null,"teamname":"Paris
Saint-Germain","leaguename":"France Ligue 1
(1)","nationname":"Morocco","skillmoves":"3","weakfoot":"4"}]’

I’m trying to access "lineid" and get "438"

Asked By: papi

||

Source

Answer 1

BeautifulSoup is a HTML parser, the response you’re receiving is a JSON.

This script could help you:

import json
import requests

name = input("Player's name: ")
url = f'https://www.futwiz.com/en/searches/career22/{name}'

r = requests.get(url)
results = [line["lineid"] for line in json.loads(r.text)]
print(results)

Answered By: bugless-soup

Answer 2

import requests
import pandas as pd

name = input("Player's name: ")
url = f'https://www.futwiz.com/en/searches/career22/{name}'
#print(url)
r = requests.get(url)
#print(r.content)

df=pd.DataFrame(r.json())
print(df)

Output:

lineid      cardname  ... skillmoves weakfoot
0       6         Messi  ...          4        4
1      12   Lewandowski  ...          4        4
2       7       Ronaldo  ...          5        4
3      58       Benzema  ...          4        4
4      35        Mbappe  ...          5        4
5      33         Salah  ...          4        3
6      16        Bruyne  ...          4        5
7      14            Jr  ...          5        5
8      17         Oblak  ...          1        3
9     112         Kante  ...          2        3
10     60         Neuer  ...          1        4
11     31          Dijk  ...          2        3
12    106          Kane  ...          3        5
13    159           Son  ...          4        5
14    111       Kimmich  ...          3        4
15     63      Casemiro  ...          2        3
16     62      Courtois  ...          1        3
17    108       Ederson  ...          1        3
18    287    Donnarumma  ...          1        3
19     34       Alisson  ...          1        3
20     32          Mane  ...          4        4
21    342      Goretzka  ...          3        4
22    277    Marquinhos  ...          3        3
23    595          Dias  ...          2        4
24     30        Stegen  ...          1        4

[25 rows x 50 columns]

Answered By: F.Hoque

Extracting data from a dictionary in bs4?

Question:

Answers: