How do I convert an array of strings from an API pull to a DataFrame with pandas?

Question

With the API I am using, it gives me a list of 15 terms that are ‘trending’ within a given category.

I want to use this data in PowerBI as part of a dashboard, but PowerBI only accepts data in the form of a Pandas dataframe.

I’ve tried a few things but I think the issue comes down to me not understanding how the data is actually being presented to me in the first place to then convert it.

The code that pulls from the API:

import requests
import pandas as pd

url = "https://api.newswhip.com/v1/trendingEntities?key=XXX" #APIkey removed for obvious reasons

payload = {
    "filters": ["categories: 33"],
    "from": None,
    "to": None
}
headers = {
    "accept": "application/json",
    "content-type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

The response to which is:

{"trendingEntities":["Charlton","Spanish Super Cup","North London Derby","Bruno Fernandes","Rodrygo","Weghorst","Facundo Pellistri","Pierre-Emerick Aubameyang","North London","Kobbie Mainoo","Angers","Sergio Busquets","FIFA Best","Aaron Ramsdale","Kai Havertz"]}

Ideally, the final output would be:

   0
0  Charlton
1  Spanish Super Cup
2  North London Derby
3  Bruno Fernandes
...
12 FIFA Best
13 Aaron Ramsdale
14 Kai Havertz

Do you have any ideas on how to split the array and convert it to a dataframe? I’ve tried a few things, from blindly asking Pandas to convert the data raw, to using dicts and concats – I didn’t save the code I tried for these, but I’m going to keep working on this and I’ll update the question as I try new methods.

Asked By: RogueSergeant

||

Source

Answer 1

Assuming you’re using the requests library, you can use:

pd.DataFrame(response.json())

This outputs:

             trendingEntities
0                    Charlton
1           Spanish Super Cup
2          North London Derby
3             Bruno Fernandes
4                     Rodrygo
5                    Weghorst
6           Facundo Pellistri
7   Pierre-Emerick Aubameyang
8                North London
9               Kobbie Mainoo
10                     Angers
11            Sergio Busquets
12                  FIFA Best
13             Aaron Ramsdale
14                Kai Havertz

Answered By: BrokenBenchmark

Answer 2

You can run a loop and make a list and append it into list

    a = []
    data = response.json()
    for item in data['trendingEntities']
            name = item
            alldata = [name]
            a.append(alldata)
    pd.DataFrame(a).to_excel('111.xlsx')

Answered By: Scraping Solution

How do I convert an array of strings from an API pull to a DataFrame with pandas?

Question:

Answers: