Convert API to Pandas DataFrame

Question

I’d like to convert the API call into a pandas data frame.

At the moment, the API is very unorganised and I’d like to incorporate pandas to make it easier to read/edit/manipulate.

I have attempted the following:

r = requests.get('http://api.football-data.org/v1/competitions/398/teams')
x = r.json()
df = pd.read_json(x)
print df

But receive:

TypeError: Expected String or Unicode

Asked By: BF99

||

Source

Answer 1

read_json function expects a string. You are providing a JSON object (parsed using requests library’s json method). What you need to do is to convert the object back to a string using json.dumps method:

import json 

r = requests.get('http://api.football-data.org/v1/competitions/398/teams')
x = r.json()
df = pd.read_json(json.dumps(x))

Or even better, just get the buffer from request object directly and do not convert it to an object.

r = requests.get('http://api.football-data.org/v1/competitions/398/teams')
df = pd.read_json(r.text)

Answered By: bman

Answer 2

pd.read_json expects a string. However, r.json() returns a dict object.

In your case, you should explore the structure of the returned JSON object by looking at x.keys(). This will yield ['count', '_links', 'teams']. You are probably interested in the ‘teams’ field.

As such, you should do the following:

r = requests.get('http://api.football-data.org/v1/competitions/398/teams')
x = r.json()
df = pd.DataFrame(x['teams'])
print df

Answered By: Rishabh Srivastava

Answer 3

This simple solution worked for me (the api link in question is not accessible by me)

df=pd.read_json('https://api.coinmarketcap.com/v1/ticker/?limit=10')
df.head()


   24h_volume_usd  available_supply            id  last_updated  
0     12465900000          16812425       bitcoin    1516379664   
1      4827670000          97080757      ethereum    1516379652   
2      5091970000       38739142811        ripple    1516379641   
3       862348000          16920150  bitcoin-cash    1516379657   
4       678044000       25927070538       cardano    1516379659   

   market_cap_usd    max_supply          name  percent_change_1h  
0    198285740450  2.100000e+07       Bitcoin               0.88   
1    103477408544           NaN      Ethereum               0.02   
2     62593157388  1.000000e+11        Ripple              -0.63   
3     30726992400  2.100000e+07  Bitcoin Cash               0.41   
4     17206681852  4.500000e+10       Cardano               0.56   

   percent_change_24h  percent_change_7d  price_btc     price_usd  rank  
0               -0.37             -15.41   1.000000  11794.000000     1   
1               -0.92             -15.24   0.090786   1065.890000     2   
2               -1.39             -20.53   0.000138      1.615760     3   
3               -2.43             -29.81   0.154675   1816.000000     4   
4               -4.15             -18.47   0.000057      0.663657     5   

  symbol  total_supply  
0    BTC      16812425  
1    ETH      97080757  
2    XRP   99993093880  
3    BCH      16920150  
4    ADA   31112483745

Answered By: usct01

Convert API to Pandas DataFrame

Question:

Answers: