How to use python to convert nested dictionary to pandas DataFrame

Question

Here is my dict looks like:

 {'ItemA': {'2022-09': {'Qty': 0, 'Amount': 0, 'Cost': 0}, '2022-08': {'Qty': 0, 'Amount': 0, 'Cost': 0},'ItemB': {'2022-09': {'Qty': 5.0, 'Amount': 10, 'Cost': 5}, '2022-08': {'Qty': 2.0, 'Amount': 8, 'Cost': 4}}}}

I want to convert this dict to pandas dataframe looks like this:

Name	2022-09 Qty	2022-09 Amount	…	2022-08 Cost
ItemA	0	0	…	0
ItemB	5	10	…	4

I have tried pd.DataFrame.from_dict, but it’s not work very well.

Asked By: ddd

||

Source

Answer 1

You can use from_records, it gets you most of the way there (note, I had to modify your input dict as it had a missing } , see below):

records = {}
for name in D:
   records[name] = pd.DataFrame.from_records(D[name]).unstack()

result = pd.DataFrame.from_records(records).transpose()
#result:
#      2022-08             2022-09            
#          Qty Amount Cost     Qty Amount Cost
#ItemA     0.0    0.0  0.0     0.0    0.0  0.0
#ItemB     2.0    8.0  4.0     5.0   10.0  5.0

If you want your specific columns names, you can follow examples like the one here:

result.columns = result.columns.map(" ".join)
result.index.name = "Name"
result.reset_index(inplace=True)
# result
#    Name  2022-08 Qty  ...  2022-09 Amount  2022-09 Cost
#0  ItemA          0.0  ...             0.0           0.0
#1  ItemB          2.0  ...            10.0           5.0

Note, I think you had a typo in your original dict, this is what I think you meant to input:

D={'ItemA': {'2022-09': {'Qty': 0, 'Amount': 0, 'Cost': 0}, '2022-08': {'Qty': 0, 'Amount': 0, 'Cost': 0}},'ItemB': {'2022-09': {'Qty': 5.0, 'Amount': 10, 'Cost': 5}, '2022-08': {'Qty': 2.0, 'Amount': 8, 'Cost': 4}}}
import json
print(json.dumps(D,indent=1))
#{
# "ItemA": {
#  "2022-09": {
#   "Qty": 0,
#   "Amount": 0,
#   "Cost": 0
#  },
#  "2022-08": {
#   "Qty": 0,
#   "Amount": 0,
#   "Cost": 0
#  }
# },
# "ItemB": {
#  "2022-09": {
#   "Qty": 5.0,
#   "Amount": 10,
#   "Cost": 5
#  },
#  "2022-08": {
#   "Qty": 2.0,
#   "Amount": 8,
#   "Cost": 4
#  }
# }
#}

Answered By: dermen

How to use python to convert nested dictionary to pandas DataFrame

Question:

Answers: