fillna multiple columns of a dataframe with corresponding columns of another dataframe pandas

Question

There is a dataframe df_1 which has some nan values. These nan values should be filled by values from another dataframe df_2 with correspond to same column and row.

df_1 = pd.DataFrame([
                   [0.1, 2, 55, 0,np.nan],
                   [0.2, 4, np.nan, 1,99],
                   [0.3, np.nan, 22, 5,88],
                   [0.4, np.nan, np.nan, 4,77]
                   ],
                   columns=list('ABCDE'))

df_2 = pd.DataFrame([
    [0.1, 2, 55, 0.5],
    [0.2, 4, 6, 1],
    [0.3, 7, 22, 5],
],
    columns=list('ABCD'))

The output is expected as:

    A     B      C      D     E
0   0.1  2.0    55.0    0    NaN
1   0.2  4.0    6.0     1   99.0
2   0.3  7.0    22.0    5   88.0
3   0.4  NaN    NaN     4   77.0

I tried with df_1 = df_1.fillna(df_2). But it does not fill up the nans. Is it any way to fix it?

Asked By: EMT

||

Source

Answer 1

For me your solution return expected ouput, maybe problem is NaNs are not missing values but strings, so need df_1 = df_1.replace('NaN', np.nan) first:

df = df_1.fillna(df_2)
print (df)
     A    B     C  D     E
0  0.1  2.0  55.0  0   NaN
1  0.2  4.0   6.0  1  99.0
2  0.3  7.0  22.0  5  88.0
3  0.4  NaN   NaN  4  77.0

df = df_1.combine_first(df_2)
print (df)
     A    B     C    D     E
0  0.1  2.0  55.0  0.0   NaN
1  0.2  4.0   6.0  1.0  99.0
2  0.3  7.0  22.0  5.0  88.0
3  0.4  NaN   NaN  4.0  77.0

Answered By: jezrael

Answer 2

Another way to do it

df_1[df_1.isnull()] = df_2
print(df_1)


     A    B     C  D     E
0  0.1  2.0  55.0  0   NaN
1  0.2  4.0   6.0  1  99.0
2  0.3  7.0  22.0  5  88.0
3  0.4  NaN   NaN  4  77.0

Answered By: delirium78

fillna multiple columns of a dataframe with corresponding columns of another dataframe pandas

Question:

Answers: