Create a pandas dataframe from a nested json file

Question

I have the following json file:

{
    "data": {
        "start_date": "2022-10-01",
        "end_date": "2022-10-04",
        "cur": "EUR",
        "prizes": {
            "2022-10-01": {
                "coffee": 0.1448939471560284,
                "usd": 1
            },
            "2022-10-02": {
                "coffee": 0.14487923291390148,
                "usd":1
            },
            "2022-10-03": {
                "coffee": 0.1454857922753868,
                "usd": 1
            }
        }
    }
}

I want to create a dataframe which looks like this, (so without the usd column):

              coffee  
2022-10-01  0.144894  
2022-10-02  0.144879  
2022-10-03  0.145486

This is what I tried:

path = r'C:UsersGeoDesktopjson_filescoffee.json'

df = pd.read_json(path)
df = pd.DataFrame(df['data']['prizes']['2022-10-01']['coffee']).T
print(df)

This is what I received:

   raise ValueError("DataFrame constructor not properly called!")
ValueError: DataFrame constructor not properly called!

Asked By: Erling_Haaland

||

Source

Answer 1

Assuming that the json is stored in the variable json

json = {
    "data": {
        "start_date": "2022-10-01",
        "end_date": "2022-10-04",
        "cur": "EUR",
        "prizes": {
            "2022-10-01": {
                "coffee": 0.1448939471560284,
                "usd": 1
            },
            "2022-10-02": {
                "coffee": 0.14487923291390148,
                "usd":1
            },
            "2022-10-03": {
                "coffee": 0.1454857922753868,
                "usd": 1
            }
        }
    }
}

The following will do the work

df = pd.DataFrame(json['data']['prizes']).T.drop('usd', axis=1)

[Out]:

              coffee
2022-10-01  0.144894
2022-10-02  0.144879
2022-10-03  0.145486

Answered By: Gonçalo Peres

Answer 2

One option assuming d the input variable:

df = (pd.DataFrame
        .from_dict(d['data']['prizes'], orient='index')
        .drop(columns='usd', errors='ignore')  # or [['coffee']]
      )

output:

              coffee
2022-10-01  0.144894
2022-10-02  0.144879
2022-10-03  0.145486

Answered By: mozway

Answer 3

Assume your json file is loaded into dictionary as data

df = pd.DataFrame(data['data']['prizes']).drop('usd').T

print(df)

              coffee
2022-10-01  0.144894
2022-10-02  0.144879
2022-10-03  0.145486

Answered By: Ynjxsjmh

Answer 4

dic=json.load(path)
df=pd.DataFrame(dic)
print(type(df.loc['prizes']))
md=pd.DataFrame(df.loc['prizes'][0])
md=md.transpose()
del md['usd']
print(md)

Answered By: Pradhumn charde

Create a pandas dataframe from a nested json file

Question:

Answers: