Is Scikit-Learn's classification report supposed to show samples with 0 support?
Question:
I’m running a text classification task and am using sklearn.metrics.classification_report. I’ve noticed that there are many cases where labels with supports of 0 are also shown, but is this expected behavior?
To give a specific example, let’s say that the task I have at hand has five labels 0, 1, 2, 3, and 4. I wanted to make sure that my model is able to train properly on label 0 and so I created a separate dataset containing only those labels. The classification report after the first epoch looks like this:
precision recall f1-score support
0 1.0000 0.8929 0.9434 9713
1 0.0000 0.0000 0.0000 0
2 0.0000 0.0000 0.0000 0
3 0.0000 0.0000 0.0000 0
accuracy 0.8929 9713
macro avg 0.2500 0.2232 0.2359 9713
weighted avg 1.0000 0.8929 0.9434 9713
As far as I know (and please correct me if I’m wrong), the labels shown on the left column are labels contained in y_true. So not only is the report not supposed to show 1, 2, and 3, but it’s also not showing 4 which it should if the behavior’s supposed to be consistent.
I’ve checked my data and made sure that it only contains 0 so it’s not a data issue, and I’ve also double checked to make sure I’m not swapping y_true and y_pred when calling the function.
Why does this behavior happen? And is there a way to fix it?
Answers:
The labels on the left are the predicted labels. In other words, your model still predicts classes 1, 2 and 3 (but not 4) in some (about 11%) cases where it should ideally predict 0.
Edit: to be precise, classification_report() runs sklearn.utils.multiclass.unique_labels() on both y_true and y_pred (unless you specify a label list explicitly), which is the same as predicted labels in this case.
The rest just wraps precision_recall_fscore_support(). The support is the number of occurrences of each class in y_true.
I’m running a text classification task and am using sklearn.metrics.classification_report. I’ve noticed that there are many cases where labels with supports of 0 are also shown, but is this expected behavior?
To give a specific example, let’s say that the task I have at hand has five labels 0, 1, 2, 3, and 4. I wanted to make sure that my model is able to train properly on label 0 and so I created a separate dataset containing only those labels. The classification report after the first epoch looks like this:
precision recall f1-score support
0 1.0000 0.8929 0.9434 9713
1 0.0000 0.0000 0.0000 0
2 0.0000 0.0000 0.0000 0
3 0.0000 0.0000 0.0000 0
accuracy 0.8929 9713
macro avg 0.2500 0.2232 0.2359 9713
weighted avg 1.0000 0.8929 0.9434 9713
As far as I know (and please correct me if I’m wrong), the labels shown on the left column are labels contained in y_true. So not only is the report not supposed to show 1, 2, and 3, but it’s also not showing 4 which it should if the behavior’s supposed to be consistent.
I’ve checked my data and made sure that it only contains 0 so it’s not a data issue, and I’ve also double checked to make sure I’m not swapping y_true and y_pred when calling the function.
Why does this behavior happen? And is there a way to fix it?
The labels on the left are the predicted labels. In other words, your model still predicts classes 1, 2 and 3 (but not 4) in some (about 11%) cases where it should ideally predict 0.
Edit: to be precise, classification_report() runs sklearn.utils.multiclass.unique_labels() on both y_true and y_pred (unless you specify a label list explicitly), which is the same as predicted labels in this case.
The rest just wraps precision_recall_fscore_support(). The support is the number of occurrences of each class in y_true.