How to unpack a list of dicts into a dataframe

Question:

I have a pandas dataframe that looks like this:

enter image description here

    market_hash_name                                  prices
0   10 Year Birthday Sticker Capsule    [{'market': 'gamerpay', 'price': 88, 'quantity...
1   1st Lieutenant Farlow | SWAT    [{'market': 'skinbid', 'price': 311, 'quantity...
2   2020 RMR Challengers    [{'market': 'skinbid', 'price': 15, 'quantity'...
3   2020 RMR Contenders [{'market': 'skinbid', 'price': 14, 'quantity'...
4   2020 RMR Legends    [{'market': 'skinbid', 'price': 14, 'quantity'...

And I’m trying to unpack the "prices" column, which contains a list of dictionaries for each row, into new columns with the market being the column name and then having the corresponding prices as the data for each row so that it looks somewhat like this:

    market_hash_name                      gamerpay        skinbid        csgofloat  
0   10 Year Birthday Sticker Capsule         88             86              75

Each list of dictionaries in the "prices" column, can contain up to 22 dictionaries with prices from different markets.

I have tried using pd.Series() which gave me this result:

enter image description here and I also got the same result using pd.json_normalize()

Any help is much appreciated!

Asked By: vaccuate

||
Source

Answers:

Assuming a list of valid dictionaries in "prices", use json_normalize on the column after explode, then pivot and join back to the original:

s = df['prices'].explode()
out = (df.drop(columns='prices')
         .join(pd.json_normalize(s).set_index(s.index).pivot(columns='market', values='price'))
      )

Example:

                   market_hash_name  gamerpay  skinbid
0  10 Year Birthday Sticker Capsule      88.0     86.0
1      1st Lieutenant Farlow | SWAT       NaN    311.0
2              2020 RMR Challengers       NaN     15.0
3               2020 RMR Contenders       NaN     14.0
4                  2020 RMR Legends       NaN     14.0
Answered By: mozway