How to overwrite a subset of a column in pandas with a dictionary

Question

I have a full dataframe (no NaNs) with some wrong cells.
I created a dictionary that has as some identifier as keys and the correct value as value.
I would like to overwrite only the cells in the column of the values that match the key in the column of the keys, leaving the rest as it is.

This is a minimal working example.
I want to overwrite the values in test_df['B'] with the ones from dictionary.values() according to the identifier stored in test_df['A'] and dictionary.keys().

test_df = pd.DataFrame({
    'A' : [1,2,3,4], 
    'B' : ['one','two','c','d'],
    'C' : ['a','b','c','d']
})

dictionary = {
    3:'three',
    4:'four',
}

wanted_result = pd.DataFrame({
    'A' : [1,2,3,4], 
    'B' : ['one', 'two','three','four'],
    'C' : ['a','b','c','d']
})

One possible solution I tried is the following.

def correct_df(dictionary, df):
    for k,v in dictionary.items():
        df.loc[df['A']==k, 'B'] = v
    return df

correct_df(dictionary, test_df)

I did obtain what I was expecting but I do not like this solution because iterating over the keys might not be so good when dictionary is big or when there are multiple values in ‘A’ matching one key of the dictionary.

Do you have any other possible solution that is faster than this one?

Asked By: Mirk

||

Source

Answer 1

One idea for mapping only matched values, this should be faster if only few matching rows with Series.map and Series.isin in DataFrame.loc:

def correct_df(dictionary, df):
    m = df['A'].isin(dictionary.keys())
    df.loc[m, 'B'] = df.loc[m, 'A'].map(dictionary)
    return df

df = correct_df(dictionary, test_df)
print (df)
   A      B  C
0  1    one  a
1  2    two  b
2  3  three  c
3  4   four  d

Another idea if matching most rows – mapping by Series.map and replace missing values (non matched) to original column by Series.fillna:

def correct_df(dictionary, df):

    df['B'] = df['A'].map(dictionary).fillna(df['B'])
    return df

df = correct_df(dictionary, test_df)
print (df)
   A      B  C
0  1    one  a
1  2    two  b
2  3  three  c
3  4   four  d

Answered By: jezrael

How to overwrite a subset of a column in pandas with a dictionary

Question:

Answers: