Any way to get mappings of a label encoder in Python pandas?
Question:
I am converting strings to categorical values in my dataset using the following piece of code.
data['weekday'] = pd.Categorical.from_array(data.weekday).labels
For eg,
index weekday
0 Sunday
1 Sunday
2 Wednesday
3 Monday
4 Monday
5 Thursday
6 Tuesday
After encoding the weekday, my dataset appears like this:
index weekday
0 3
1 3
2 6
3 1
4 1
5 4
6 5
Is there any way I can know that Sunday has been mapped to 3, Wednesday to 6 and so on?
Answers:
First, make a categorical series:
weekdays = pd.Series(['Sun', 'Sun', 'Wed', 'Mon', 'Mon']).astype('category')
Then, inspect its “categories”:
weekdays.cat.categories.get_loc('Sun')
The best way of doing this can be to use label encoder of sklearn library.
Something like this:
from sklearn import preprocessing
le = preprocessing.LabelEncoder()
le.fit(["paris", "paris", "tokyo", "amsterdam"])
list(le.classes_)
le.transform(["tokyo", "tokyo", "paris"])
list(le.inverse_transform([2, 2, 1]))
You can create additional dictionary with mapping:
from sklearn import preprocessing
le = preprocessing.LabelEncoder()
le.fit(data['name'])
le_name_mapping = dict(zip(le.classes_, le.transform(le.classes_)))
print(le_name_mapping)
{'Tom': 0, 'Nick': 1, 'Kate': 2}
There are many ways of doing this. You can consider pd.factorize
, sklearn.preprocessing.LabelEncoder
etc. However, in this specific case, you have two options which will suit you best:
Going by your own method, you can add the categories:
pd.Categorical( df.weekday, [
'Sunday', 'Monday', 'Tuesday',
'Wednesday', 'Thursday', 'Friday',
'Saturday'] ).labels
The other option is to map values directly using a dict
df.weekday.map({
'Sunday': 0,
'Monday': 1,
# ... and so on. You get the idea ...
})
A simple & elegant way to do the same.
cat_list = ['Sun', 'Sun', 'Wed', 'Mon', 'Mon']
encoded_data, mapping_index = pd.Series(cat_list).factorize()
and you are done, check below
print(encoded_data)
print(mapping_index)
print(mapping_index.get_loc("Mon"))
If you have numerical and categorical both type of data in dataframe
You can use : here X is my dataframe having categorical and numerical both variables
from sklearn import preprocessing
le = preprocessing.LabelEncoder()
for i in range(0,X.shape[1]):
if X.dtypes[i]=='object':
X[X.columns[i]] = le.fit_transform(X[X.columns[i]])
Or you can try this:
from sklearn.preprocessing import LabelEncoder
le = LabelEncoder()
data = data.apply(le.fit_transform)
Note: This technique is good if you are not interested in converting them back.
train['cat'] = train['cat'].map(list(train['cat'].value_counts().to_frame().reset_index().reset_index().set_index('index').to_dict().values())[0])
Its very simple, they have a built-in function for this.
from sklearn import preprocessing
le = preprocessing.LabelEncoder()
..
# your model steps and when you have results
..
prediction_decoded = le.inverse_transform(prediction_encoded)
print(prediction_decoded)
I am adding my answer even after lot of answers are there to answer this OP’ed question specifically as:
If have you already label encoded your values as:
from sklearn import preprocessing
le = preprocessing.LabelEncoder()
le.fit('column-or-list-of-values')
You can get back the mapping of value to integer codes as:
dict(zip(le.classes_,range(len(le.classes_))))
You can create another column with the indexed values after converting from categorical to numeric. Think of it as primary key column.
#new column,index contains converted values of column weekday
data[‘index’] = pd.Categorical.from_array(data.weekday).labels
#view the corresponding values
df.groupby([‘index’,’weekday’]).first()
I am converting strings to categorical values in my dataset using the following piece of code.
data['weekday'] = pd.Categorical.from_array(data.weekday).labels
For eg,
index weekday
0 Sunday
1 Sunday
2 Wednesday
3 Monday
4 Monday
5 Thursday
6 Tuesday
After encoding the weekday, my dataset appears like this:
index weekday
0 3
1 3
2 6
3 1
4 1
5 4
6 5
Is there any way I can know that Sunday has been mapped to 3, Wednesday to 6 and so on?
First, make a categorical series:
weekdays = pd.Series(['Sun', 'Sun', 'Wed', 'Mon', 'Mon']).astype('category')
Then, inspect its “categories”:
weekdays.cat.categories.get_loc('Sun')
The best way of doing this can be to use label encoder of sklearn library.
Something like this:
from sklearn import preprocessing
le = preprocessing.LabelEncoder()
le.fit(["paris", "paris", "tokyo", "amsterdam"])
list(le.classes_)
le.transform(["tokyo", "tokyo", "paris"])
list(le.inverse_transform([2, 2, 1]))
You can create additional dictionary with mapping:
from sklearn import preprocessing
le = preprocessing.LabelEncoder()
le.fit(data['name'])
le_name_mapping = dict(zip(le.classes_, le.transform(le.classes_)))
print(le_name_mapping)
{'Tom': 0, 'Nick': 1, 'Kate': 2}
There are many ways of doing this. You can consider pd.factorize
, sklearn.preprocessing.LabelEncoder
etc. However, in this specific case, you have two options which will suit you best:
Going by your own method, you can add the categories:
pd.Categorical( df.weekday, [
'Sunday', 'Monday', 'Tuesday',
'Wednesday', 'Thursday', 'Friday',
'Saturday'] ).labels
The other option is to map values directly using a dict
df.weekday.map({
'Sunday': 0,
'Monday': 1,
# ... and so on. You get the idea ...
})
A simple & elegant way to do the same.
cat_list = ['Sun', 'Sun', 'Wed', 'Mon', 'Mon']
encoded_data, mapping_index = pd.Series(cat_list).factorize()
and you are done, check below
print(encoded_data)
print(mapping_index)
print(mapping_index.get_loc("Mon"))
If you have numerical and categorical both type of data in dataframe
You can use : here X is my dataframe having categorical and numerical both variables
from sklearn import preprocessing
le = preprocessing.LabelEncoder()
for i in range(0,X.shape[1]):
if X.dtypes[i]=='object':
X[X.columns[i]] = le.fit_transform(X[X.columns[i]])
Or you can try this:
from sklearn.preprocessing import LabelEncoder
le = LabelEncoder()
data = data.apply(le.fit_transform)
Note: This technique is good if you are not interested in converting them back.
train['cat'] = train['cat'].map(list(train['cat'].value_counts().to_frame().reset_index().reset_index().set_index('index').to_dict().values())[0])
Its very simple, they have a built-in function for this.
from sklearn import preprocessing
le = preprocessing.LabelEncoder()
..
# your model steps and when you have results
..
prediction_decoded = le.inverse_transform(prediction_encoded)
print(prediction_decoded)
I am adding my answer even after lot of answers are there to answer this OP’ed question specifically as:
If have you already label encoded your values as:
from sklearn import preprocessing
le = preprocessing.LabelEncoder()
le.fit('column-or-list-of-values')
You can get back the mapping of value to integer codes as:
dict(zip(le.classes_,range(len(le.classes_))))
You can create another column with the indexed values after converting from categorical to numeric. Think of it as primary key column.
#new column,index contains converted values of column weekday
data[‘index’] = pd.Categorical.from_array(data.weekday).labels
#view the corresponding values
df.groupby([‘index’,’weekday’]).first()