Logistic Regression: How to find top three feature that have highest weights?

Question

I am working on UCI breast cancer dataset and trying to find the top 3 features that have highest weights. I was able to find the weight of all features using logmodel.coef_ but how can I get the feature names? Below is my code, output and dataset (which is imported from scikit).

from sklearn.model_selection import train_test_split
from sklearn.datasets import load_breast_cancer
from sklearn.linear_model import LogisticRegression

cancer = load_breast_cancer()
X_train, X_test, y_train, y_test = train_test_split(
    cancer.data, cancer.target, stratify=cancer.target, random_state=42)

logmodel = LogisticRegression(C=1.0).fit(X_train, y_train)
logmodel.coef_[0]

Above code outputs weights array. Using these weights how can I get the associate feature names?

Output:
    array([  1.90876683e+00,   9.98788148e-02,  -7.65567571e-02,
             1.30875965e-03,  -1.36948317e-01,  -3.86693503e-01,
            -5.71948682e-01,  -2.83323656e-01,  -2.23813863e-01,
            -3.50526844e-02,   3.04455316e-03,   1.25223693e+00,
             9.49523571e-02,  -9.63789785e-02,  -1.32044174e-02,
            -2.43125981e-02,  -5.86034313e-02,  -3.35199227e-02,
            -4.10795998e-02,   1.53205924e-03,   1.24707244e+00,
            -3.19709151e-01,  -9.61881472e-02,  -2.66335879e-02,
            -2.44041661e-01,  -1.24420873e+00,  -1.58319440e+00,
            -5.78354663e-01,  -6.80060645e-01,  -1.30760323e-01])

Thanks. I would really appreciate any help on this.

Asked By: jubins

||

Source

Answer 1

This will do the job:

import numpy as np
coefs=logmodel.coef_[0]
top_three = np.argpartition(coefs, -3)[-3:]
print(cancer.feature_names[top_three])

This prints

['worst radius' 'texture error' 'mean radius']

Note that these features are the top three, but they are not necessarily sorted among themselves. If you want them to be sorted, you can do:

import numpy as np
coefs=logmodel.coef_[0]
top_three = np.argpartition(coefs, -3)[-3:]
top_three_sorted=top_three[np.argsort(coefs[top_three])]
print(cancer.feature_names[top_three_sorted])

Answered By: Miriam Farber

Answer 2

n=len(data.columns)-1

percentage_change=[]

for i in range (n):

    cp=(difference[i]/wei[i])*100

    percentage_change.append(cp)
    
columns=list(data.columns.values)

indices=sorted(range(len(percentage_change)), key=lambda i: percentage_change[i])[-3:]

print("the top 3 features which have higher % change in weights ")

for j in indices:

    print(columns[j])

Answered By: suman

Logistic Regression: How to find top three feature that have highest weights?

Question:

Answers: