Pandas replace NaN values with zeros after pivot operation

Question

I have a DataFrame like this:

Year    Month     Day      Rain (mm)

2021      1        1          15

2021      1        2          NaN

2021      1        3          12

And so on (there are multiple years). I have used pivot_table function to convert the DataFrame into this:

Year         2021     2020    2019    2018   2017
Month  Day
1       1     15
        2     NaN
        3     12

I used:

df = df.pivot_table(index=['Month', 'Day'], columns='Year',
                 values='Rain (mm)', aggfunc='first')

Now I would like to replace all NaN values and also possible -1 values with zeros from every column (by columns I mean years) but I have not been able to do so. I have tried:

df = df.fillna(0)

And also:

df.loc[df['Rain (mm)'] == NaN, 'Rain (mm)'] = 0

But neither won’t work, no error message/exception, dataframe just remains unchanged. What I’m doing wrong? Any advise is highly appreciated.

Asked By: Mr. Engineer

||

Source

Answer 1

I think problem is NaN are strings, so cannot replace them, so first try convert values to numeric:

df['Rain (mm)'] = pd.to_numeric(df['Rain (mm)'], errors='coerce')

df = df.pivot_table(index=['Month', 'Day'], columns='Year',
             values='Rain (mm)', aggfunc='first').fillna(0)

Answered By: jezrael

Pandas replace NaN values with zeros after pivot operation

Question:

Answers: