xpath selector in bs4 and xml doesn't work

Question

i want to get the rank data from this link https://zapper.fi/nft/ethereum/0x2a6eec51ee67a941fa50c1fbc4ca8853604461b8/616
but the returned result for me is []

my code is as follow :

import requests
from bs4 import BeautifulSoup
from lxml import etree
url = 'https://zapper.fi/nft/ethereum/0x2a6eec51ee67a941fa50c1fbc4ca8853604461b8/616'
webpage = requests.get(url, headers=HEADERS)

soup = BeautifulSoup(webpage.content, "html.parser")
dom = etree.HTML(str(soup))
rank = dom.xpath('/html/body/div[1]/div[2]/div[3]/main/div/div/div/div/div[2]/div[1]/div[2]/div[2]/div[1]/div[2]')

enter image description here

how can i solve this

thanks in advance

Asked By: hasti hakhamanesh

||

Source

Answer 1

This is one way to obtain that information:

import requests
from bs4 import BeautifulSoup as bs
import json
import pandas as pd

url = 'https://zapper.fi/nft/ethereum/0x2a6eec51ee67a941fa50c1fbc4ca8853604461b8/616'
r = requests.get(url)
soup = bs(r.text, 'html.parser')
json_obj = json.loads(soup.select_one('script[id="__NEXT_DATA__"]').text)
df = pd.json_normalize(json_obj['props']['pageProps']['nftToken'])
print(df[['rarityRank', 'collection.supply']])

Result in terminal:

    rarityRank  collection.supply
0   746 6000

Answered By: Barry the Platipus

xpath selector in bs4 and xml doesn't work

Question:

Answers: