Using map to convert pandas dataframe to list

Question

I am using map to convert some columns in a dataframe to list of dicts. Here is a MWE illustrating my question.

import pandas as pd

df = pd.DataFrame()
df['Col1'] = [197, 1600, 1200]
  
df['Col2'] = [297, 2600, 2200]
  
df['Col1_a'] = [198, 1599, 1199]

df['Col2_a'] = [296, 2599, 2199]

print(df)

The output is

   Col1  Col2  Col1_a  Col2_a
0   197   297     198     296
1  1600  2600    1599    2599
2  1200  2200    1199    2199

Now say I want to extract only those columns whose name ends with a suffix "_a". One way to do it is the following –

list_col = ["Col1","Col2"]
cols_w_suffix = map(lambda x: x + '_a', list_col)

print(df[cols_w_suffix].to_dict('records'))

[{'Col1_a': 198, 'Col2_a': 296}, {'Col1_a': 1599, 'Col2_a': 2599}, {'Col1_a': 1199, 'Col2_a': 2199}]

This is expected answer. However, if I try to print the same expression again, I get an empty dataframe.

print(df[cols_w_suffix].to_dict('records'))

[]

Why does it evaluate to an empty dataframe? I think I am missing something about the behavior of map. Because when I directly pass the column names, the output is still as expected.

df[["Col1_a","Col2_a"]].to_dict('records')

[{'Col1_a': 198, 'Col2_a': 296}, {'Col1_a': 1599, 'Col2_a': 2599}, {'Col1_a': 1199, 'Col2_a': 2199}]

Asked By: honeybadger

||

Source

Answer 1

Your map generator is exhausted.

Use cols_w_suffix = list(map(lambda x: x + '_a', list_col)) or a list comprehension cols_w_suffix = [f'{x}_a' for x in list_col].

That said, a better method to select the columns would be:

df.filter(regex='_a$')

Or:

df.loc[:, df.columns.str.endswith('_a')]

Answered By: mozway

Using map to convert pandas dataframe to list

Question:

Answers: