json_normalized from csv to dataframe

Question

I have this data in a csv file:

cardinality|create_time|properties 382|2022-04-05 16:32:43.000+0000|{'birthday': '542217600000', 'first_name': 'Char', 'gender': 'Male', 'last_name': 'Mander', 'nick_name': ''} 400|2022-02-07 14:20:59.000+0000|{'birthday': '967046400000', 'first_name': 'ABC', 'gender': 'Male', 'last_name': 'ZXY', 'nick_name': ''} 132|2021-08-09 14:01:30.000+0000|{'birthday': '739900800000', 'first_name': 'Test', 'gender': 'Male', 'last_name': 'Tickles', 'nick_name': ''}

Im trying to json_normalize ‘properties’ but im encountering an empty data frame after json_normalize.

This is the complete steps that I am doing:

with open ('input.csv' ,'r') as data1: 
    df1 = pd.read_csv(data1, encoding='utf-8', dtype= 'object', sep='|') 
df1=df1['properties']
print(pd.json_normalize(df1))

It results into below

Empty DataFrame
Columns: []
Index: [0, 1, 2]

im trying to achieve this output

       birthday first_name gender last_name nick_name
0  542217600000       Char   Male    Mander
1  967046400000        ABC   Male       ZXY
2  739900800000       Test   Male   tickles

Asked By: chichi

||

Source

Answer 1

You need to pass the properties column in json_normalize as a list of dictionaries, however, it is currently a string. You can convert the properties column to a list of dictionaries using ast.literal_eval from the ast module.

try this code

import ast
import pandas as pd

with open ('input.csv' ,'r') as data1: 
    df1 = pd.read_csv(data1, encoding='utf-8', dtype= 'object', sep='|') 
df1['properties'] = df1['properties'].apply(ast.literal_eval)
df1 = pd.json_normalize(df1, 'properties')
print(df1)

Answered By: OB Ayoub

json_normalized from csv to dataframe

Question:

Answers: