How to load a pandas column from a csv file with lists as lists and not as strings

Question

when I write a list in pandas when I read it, its dtype is string and not array, is there any way to write a column of list in such a way that be again array type when we read it?

Here is what I mean:

d=[['d',['A','B','C']],['p',['F','G']]]

df=pd.DataFrame(d)

df.to_csv('file.csv')

when I run the following code,

pd.read_csv('file.csv')['1'].values[0]

the output is:

 "['A', 'B', 'C']"

but I want this:

  ['A', 'B', 'C']

Asked By: user15649753

||

Source

Answer 1

one solution would be to run it through literal_eval.

you make a dictionary with the column name as key and the converter function as value. and pass that into read_csv with the keyword converters

Note that if your column has mixed data (also strings and other stuff) you might want to write a custom function which filters and converts the different types

from ast import literal_eval
df1 = pd.read_csv('file.csv', converters={'1': literal_eval})
df1

output:

type(df1["1"][0])

output:

Answered By: n4321d

How to load a pandas column from a csv file with lists as lists and not as strings

Question:

Answers: