Analyzing keras model w.r.t a specific feature
Question:
I have a dataset that consists of different features, like "gender".
The task of the model is to determine if the annual income is above or below 50k. It is the "Adult income dataset" from:
https://www.kaggle.com/datasets/wenruliu/adult-income-dataset
Let say I have a trained network that does the classification.
Now I want to see how often the classifier make false positive respectively false negative predictions w.r.t "gender".
The basic idea is a confusion matrix of some sorts, but not a matrix of class to class but class to feature. The image below illustrates the result I would like to have.
Answers:
The basic idea is as follows:
1)Make a prediction with the Network.
2)Set the predicted values as new column in your Dataset, you now have a new dataset data_new
Your dataset now has two columns, one for the predicted and one for the true values. You can calculate the overall accuracy by boolean comparison (1 and 1 is right prediction and 0 and 1 and 1 and 0 are wrong predictions respectively).
3)Now you can filter the new data for any column you want, so in my case for the specific gender.
4)Now you can calculate the accuracy w.r.t to chosen gender.
I have a dataset that consists of different features, like "gender".
The task of the model is to determine if the annual income is above or below 50k. It is the "Adult income dataset" from:
https://www.kaggle.com/datasets/wenruliu/adult-income-dataset
Let say I have a trained network that does the classification.
Now I want to see how often the classifier make false positive respectively false negative predictions w.r.t "gender".
The basic idea is a confusion matrix of some sorts, but not a matrix of class to class but class to feature. The image below illustrates the result I would like to have.
The basic idea is as follows:
1)Make a prediction with the Network.
2)Set the predicted values as new column in your Dataset, you now have a new dataset data_new
Your dataset now has two columns, one for the predicted and one for the true values. You can calculate the overall accuracy by boolean comparison (1 and 1 is right prediction and 0 and 1 and 1 and 0 are wrong predictions respectively).
3)Now you can filter the new data for any column you want, so in my case for the specific gender.
4)Now you can calculate the accuracy w.r.t to chosen gender.