Replace NAN with Dictionary Value for a column in Pandas using Replace() or fillna() in Python

Question

I’m new to python and I’m trying to use fillna() functionality and facing some problem.
I have a DataFrame called Temp_Data_DF which has two columns like below:

Temp_Data_DF:
A  B
1  NAN
2  NAN
3  {'KEY':1,'VALUE':2}

I want to replace all NAN with Dict value and resulted dataframe should be like this:

Temp_Data_DF:
A  B
1  {'KEY':1,'VALUE':2}
2  {'KEY':1,'VALUE':2}
3  {'KEY':1,'VALUE':2}

I tried the below code:

Bvalue = {'KEY':1,'VALUE':2}
Temp_Data_DF['B']=Temp_Data_DF['B'].fillna(Bvalue)

But its not replacing the NAN with desired value
Any help will be appreciated.

I was refering to below link.

Link:Pandas dataframe fillna() only some columns in place

Asked By: sindhu

||

Source

Answer 1

You can fillna by Series created by dictionary:

Bvalue = {'KEY':10,'VALUE':20}
Temp_Data_DF['B']=Temp_Data_DF['B'].fillna(pd.Series([Bvalue], index=Temp_Data_DF.index))
print (Temp_Data_DF)
   A                         B
0  1  {'VALUE': 20, 'KEY': 10}
1  2  {'VALUE': 20, 'KEY': 10}
2  3    {'VALUE': 2, 'KEY': 1}

Detail:

print (pd.Series([Bvalue], index=Temp_Data_DF.index))
0    {'VALUE': 20, 'KEY': 10}
1    {'VALUE': 20, 'KEY': 10}
2    {'VALUE': 20, 'KEY': 10}
dtype: object

How it working:

Idea is create new Series with same size like original Series filled by dictionary, so if use fillna by another Series it working nice.

Another solution: Idea is use NaN != NaN, so if use if-else in Series.apply it replace too:

Bvalue = {'KEY':10,'VALUE':20}
Temp_Data_DF['B']=Temp_Data_DF['B'].apply(lambda x: x if x == x else Bvalue)
print (Temp_Data_DF)
   A                         B
0  1  {'KEY': 10, 'VALUE': 20}
1  2  {'KEY': 10, 'VALUE': 20}
2  3  {'KEY': 10, 'VALUE': 20}

Answered By: jezrael

Answer 2

I had a similar problem but @jezrael’s approach was not working for me. I managed to get it work by creating a series from a list of the default dict.

Temp_Data_DF['B'] = Temp_Data_DF['B'].fillna(pd.Series([{'KEY':1,'VALUE':2}] * Temp_Data_DF.shape[0]))

Answered By: QwertyHarry

Replace NAN with Dictionary Value for a column in Pandas using Replace() or fillna() in Python

Question:

Answers: