ValueError: Series.replace cannot use dict-value and non-None to_replace

Question:

Code:

h1=df[df["native-country"]!="?"]
f1=h1.mode()
df['native-country'] = df['native-country'].replace("?",df['native-country'].mode())

Error:

ValueError: Series.replace cannot use dict-value and non-None to_replace

I dont know why im getting this error hope someone could help me

Asked By: Vishnu Venkat18

||

Answers:

My simplistic, recommended answer is to use:

mode = df[df["native-country"]!="?"]["native-country"].mode()[0]
df['native-country'] = df['native-country'].replace("?", mode)

but this comes with a major caveat and further explanation below.


More detailed explanation:

df['native-country'].mode() will return a Series, not an individual value. This is because there can be more than one mode. Consider the following:

d = {'native-country': ['?', None, 'Spain', 'Germany', 'Greece']}
df = pd.DataFrame(d)
mode = df['native-country'].mode()

Inspecting mode reveals that there are actually multiple mode values, since the most common element of the Series is any element that occurs once:

0          ?
1    Germany
2     Greece
3      Spain
dtype: object

It’s also worth noting that the None value is excluded by default. Even in the case where there is a single most common value, Series.mode() returns a single element Series:

d = {'native-country': ['?', None, 'Spain', 'Germany', 'Spain']}
df = pd.DataFrame(d)
mode = df['native-country'].mode()

This gives mode as:

0    Spain
dtype: object

My very simplistic approach just uses the first value of the returned Series as the value used to replace, but you’ll have to decide for yourself if you want a more complex logic for the case where there may be multiple modes.


Full, copy-pasteable code example:

import pandas as pd


d = {'native-country': ['?', None, 'Spain', 'Germany', 'Spain']}
df = pd.DataFrame(d)

print(df)

mode = df[df["native-country"]!="?"]["native-country"].mode()[0]
df['native-country'] = df['native-country'].replace("?", mode)

print(df)
Answered By: Frodnar

Hello I had the same error pupping up.

But by the time I added the sqared brakeds it was solved, as follows

X_val[‘V1’]=X_val[‘V1’].replace([‘X_val.V1>13’],-0.70,inplace=True)

then for yours:
.replace(["?"],mode)

Answered By: Galo