Pandas DataFrame to List of Lists


It’s easy to turn a list of lists into a pandas dataframe:

import pandas as pd
df = pd.DataFrame([[1,2,3],[3,4,5]])

But how do I turn df back into a list of lists?

lol = df.what_to_do_now?
print lol
# [[1,2,3],[3,4,5]]
Asked By: bumpkin



You could access the underlying array and call its tolist method:

>>> df = pd.DataFrame([[1,2,3],[3,4,5]])
>>> lol = df.values.tolist()
>>> lol
[[1L, 2L, 3L], [3L, 4L, 5L]]
Answered By: DSM

I don’t know if it will fit your needs, but you can also do:

>>> lol = df.values
>>> lol
array([[1, 2, 3],
       [3, 4, 5]])

This is just a numpy array from the ndarray module, which lets you do all the usual numpy array things.

Answered By: aps

If the data has column and index labels that you want to preserve, there are a few options.

Example data:

>>> df = pd.DataFrame([[1,2,3],[3,4,5]], 
       columns=('first', 'second', 'third'), 
       index=('alpha', 'beta')) 
>>> df
       first  second  third
alpha      1       2      3
beta       3       4      5

The tolist() method described in other answers is useful but yields only the core data – which may not be enough, depending on your needs.

>>> df.values.tolist()
[[1, 2, 3], [3, 4, 5]]

One approach is to convert the DataFrame to json using df.to_json() and then parse it again. This is cumbersome but does have some advantages, because the to_json() method has some useful options.

>>> df.to_json()

>>> df.to_json(orient='split')

Cumbersome but may be useful.

The good news is that it’s pretty straightforward to build lists for the columns and rows:

>>> columns = [] + [i for i in df.columns]
>>> rows = [[i for i in row] for row in df.itertuples()]

This yields:

>>> print(f"columns: {columns}nrows: {rows}") 
columns: [None, 'first', 'second', 'third']
rows: [['alpha', 1, 2, 3], ['beta', 3, 4, 5]]

If the None as the name of the index is bothersome, rename it:

df = df.rename_axis('stage')


>>> columns = [] + [i for i in df.columns]
>>> print(f"columns: {columns}nrows: {rows}") 

columns: ['stage', 'first', 'second', 'third']
rows: [['alpha', 1, 2, 3], ['beta', 3, 4, 5]]
Answered By: Andrew E

I wanted to preserve the index, so I adapted the original answer to this solution:

list_df = df.reset_index().values.tolist()

Now you can paste it somewhere else (e.g. to paste into a Stack Overflow question) and latter recreate it:

pd.Dataframe(list_df, columns=['name1', ...])
pd.set_index(['name1'], inplace=True)
Answered By: neves

Maybe something changed but this gave back a list of ndarrays which did what I needed.

Answered By: Ian Rubenstein

Note: I have seen many cases on Stack Overflow where converting a Pandas Series or DataFrame to a NumPy array or plain Python lists is entirely unecessary. If you’re new to the library, consider double-checking whether the functionality you need is already offered by those Pandas objects.

To quote a comment by @jpp:

In practice, there’s often no need to convert the NumPy array into a list of lists.

If a Pandas DataFrame/Series won’t work, you can use the built-in DataFrame.to_numpy and Series.to_numpy methods.

Answered By: AMC

We can use the DataFrame.iterrows() function to iterate over each of the rows of the given Dataframe and construct a list out of the data of each row:

# Empty list 
row_list =[] 

# Iterate over each row 
for index, rows in df.iterrows(): 
    # Create list for the current row 
    my_list =[rows.Date, rows.Event, rows.Cost] 

    # append the list to the final list 

# Print 

We can successfully extract each row of the given data frame into a list

Answered By: Ram Prajapati

This is very simple:

import numpy as np

list_of_lists = np.array(df)
Answered By: Tms91

"df.values" returns a numpy array. This does not preserve the data types. An integer might be converted to a float.

df.iterrows() returns a series which also does not guarantee to preserve the data types. See:

The code below converts to a list of list and preserves the data types:

rows = [list(row) for row in df.itertuples()]
Answered By: e1i45

If you wish to convert a Pandas DataFrame to a table (list of lists) and include the header column this should work:

import pandas as pd
def dfToTable(df:pd.DataFrame) -> list:
    return [list(df.columns)] + df.values.tolist()

Usage (in REPL):

>>> df = pd.DataFrame(
             , columns=["c1", "c2", "c3"])
>>> df
     c1    c2    c3
0  r1c1  r1c2  r1c3
1  r2c1  r2c2  r3c3
>>> dfToTable(df)
[['c1', 'c2', 'c3'], ['r1c1', 'r1c2', 'r1c3'], ['r2c1', 'r2c2', 'r3c3']]
Answered By: Timothy C. Quinn
  1. The solutions presented so far suffer from a "reinventing the wheel" approach. Quoting @AMC:

If you’re new to the library, consider double-checking whether the functionality you need is already offered by those Pandas objects.

  1. If you convert a dataframe to a list of lists you will lose information – namely the index and columns names.

My solution: use to_dict()

dict_of_lists = df.to_dict(orient='split')

This will give you a dictionary with three lists: index, columns, data. If you decide you really don’t need the columns and index names, you get the data with

Answered By: Thorsten

I had this problem: how do I get the headers of the df to be in row 0 for writing them to row 1 in the excel (using xlsxwriter)? None of the proposed solutions worked, but they pointed me in the right direction. I just needed one line more of code

# get csv data
df = pd.read_csv(filename)

# combine column headers and list of lists of values
lol = [df.columns.tolist()] + df.values.tolist()
Answered By: Jondalar

Not quite relate to the issue but another flavor with same expectation

converting data frame series into list of lists to plot the chart using create_distplot in Plotly

Categories: questions Tags: ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.