Feature Importance from OneVsRestClassifier
Question:
As the title suggests, how can I obtain the feature importances from a OneVsRestClassifier model?
I tried using model.feature_importances_ but the error message was
"OneVsRestClassifier’ object has no attribute ‘feature_importances_"
Tried searching from the internet but was not able to find any clue.
Answers:
OneVsRestClassifier() basically builds as much binary classifiers as there are classes. Each has its own set of importances (assuming the base classifier supports them), showing the importance of features to distinguish a certain class from all others when generalizing on the train set. Those can be accessed with .estimators_[i].feature_importances_.
Alternatively, you may study other sorts of feature importances, like sklearn.inspection.permutation_importance, which are universally applicable.
Here’s my solution to get important features for a stacked classification with sklearn below. Here the stacked_model is the model built with OneVsRestClassifier :
# to get permutation:
results = permutation_importance(stacked_model, x, y, scoring='accuracy')
# get important features:
important_features = results.importances_mean
# list all features:
for i,v in enumerate(important_features):
print('Feature: %0d, Score: %.5f' % (i,v))
As the title suggests, how can I obtain the feature importances from a OneVsRestClassifier model?
I tried using model.feature_importances_ but the error message was
"OneVsRestClassifier’ object has no attribute ‘feature_importances_"
Tried searching from the internet but was not able to find any clue.
OneVsRestClassifier() basically builds as much binary classifiers as there are classes. Each has its own set of importances (assuming the base classifier supports them), showing the importance of features to distinguish a certain class from all others when generalizing on the train set. Those can be accessed with .estimators_[i].feature_importances_.
Alternatively, you may study other sorts of feature importances, like sklearn.inspection.permutation_importance, which are universally applicable.
Here’s my solution to get important features for a stacked classification with sklearn below. Here the stacked_model is the model built with OneVsRestClassifier :
# to get permutation:
results = permutation_importance(stacked_model, x, y, scoring='accuracy')
# get important features:
important_features = results.importances_mean
# list all features:
for i,v in enumerate(important_features):
print('Feature: %0d, Score: %.5f' % (i,v))