output NaN value when using apply function to a dataframe with index

Question

I am trying to use apply function to create 2 new columns. when dataframe has index, it doesn’t wokr, the new columns have values of NaN. If dataframe has no index, then it works. Could you please help? Thanks


def calc_test(row):

    a=row['col1']+row['col2']
    b=row['col1']/row['col2']
    return (a,b)

df_test_dict={'col1':[1,2,3,4,5],'col2':[10,20,30,40,50]}
df_test=pd.DataFrame(df_test_dict)
df_test.index=['a1','b1','c1','d1','e1']
df_test

    col1    col2
a1  1   10
b1  2   20
c1  3   30
d1  4   40
e1  5   50

Now I use apply function, the new creately columns have values of NaN. Thanks for your help.

df_test[['a','b']] = pd.DataFrame(df_test.apply(lambda row:calc_test(row),axis=1).tolist())
df_test


col1    col2    a   b
a1  1   10  NaN NaN
b1  2   20  NaN NaN
c1  3   30  NaN NaN
d1  4   40  NaN NaN
e1  5   50  NaN Na

Asked By: roudan

||

Source

Answer 1

You are wrapping the return of the apply as a DataFrame which has a default indexing of [0, 1, 2, 3, 4] which don’t exist in your original DataFrame’s index. You can see this by looking at the output of pd.DataFrame(df_test.apply(lambda row:calc_test(row),axis=1).tolist()).

Simply remove the pd.DataFrame() to fix this problem.

df_test[['a', 'b']] = df_test.apply(lambda row:calc_test(row),axis=1).tolist()

Answered By: Michael Cao

Answer 2

When using apply, you may use the result_type ='expand' argument to expand the output of your function as columns of a pandas Dataframe:

df_test[['a','b']]=df_test.apply(lambda row:calc_test(row),axis=1, result_type ='expand')

This returns:

    col1    col2    a   b
a1  1   10  11.0    0.1
b1  2   20  22.0    0.1
c1  3   30  33.0    0.1
d1  4   40  44.0    0.1
e1  5   50  55.0    0.1

Answered By: Sheldon

output NaN value when using apply function to a dataframe with index

Question:

Answers: