Trying to make a pandas dataframe from a dictionary in a list of dictionaries

Question

I have JSON data from a website, where I am trying to create a pandas dataframe from the data. It seems like I have a list of dictionaries that is nested in a dictionary and I am not sure what to do. My goal was to create key,value pairs and then make them into a dataframe.

import requests
import pandas as pd

search_url = 'https://data.europa.eu/api/hub/statistics/data/num-datasets'
response = requests.get(search_url)

root=response.json()

print(root)

I was able to get the data into my notebook, but I am not sure the best way to get data out of the dictionaries into lists to create a dataframe.

I tried to use pd.json_normalize(), but it didn’t work.

The output looks like this:

{'name': 'count',
 'stats': [{'date': '2019-08-01', 'count': 877625.0},
  {'date': '2019-09-01', 'count': 895697.0},
  {'date': '2020-10-01', 'count': 1161894.0},
  {'date': '2020-11-01', 'count': 1205046.0},
  {'date': '2020-12-01', 'count': 1184899.0},
  {'date': '2023-01-01', 'count': 1503404.0}]}

My goal is to have two columns in a pd.Dataframe:

Date
Count

Asked By: natguy8

||

Source

Answer 1

d={'name': 'count',
 'stats': [{'date': '2019-08-01', 'count': 877625.0},
  {'date': '2019-09-01', 'count': 895697.0},
  {'date': '2020-10-01', 'count': 1161894.0},
  {'date': '2020-11-01', 'count': 1205046.0},
  {'date': '2020-12-01', 'count': 1184899.0},
  {'date': '2023-01-01', 'count': 1503404.0}]}
pd.DataFrame(d['stats'])
Out[274]: 
         date      count
0  2019-08-01   877625.0
1  2019-09-01   895697.0
2  2020-10-01  1161894.0
3  2020-11-01  1205046.0
4  2020-12-01  1184899.0
5  2023-01-01  1503404.0

Answered By: Surjit Samra

Trying to make a pandas dataframe from a dictionary in a list of dictionaries

Question:

Answers: