ValueError: pos_label=1 is not a valid label: array(['neg', 'pos'], dtype='<U3')

Question:

I recieve this error while trying to obtain the recall score. 

X_test = test_pos_vec + test_neg_vec
Y_test = ["pos"] * len(test_pos_vec) + ["neg"] * len(test_neg_vec)

recall_average = recall_score(Y_test, y_predict, average="binary")

print(recall_average)

This will give me:

    C:Usersanca_elena.moisaAppDataLocalProgramsPythonPython36libsite-packagessklearnmetricsclassification.py:1030: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison
  if pos_label not in present_labels:
Traceback (most recent call last):
  File "G:/PyCharmProjects/NB/accuracy/script.py", line 812, in <module>
    main()
  File "G:/PyCharmProjects/NB/accuracy/script.py", line 91, in main
    evaluate_model(model, train_pos_vec, train_neg_vec, test_pos_vec, test_neg_vec, False)
  File "G:/PyCharmProjects/NB/accuracy/script.py", line 648, in evaluate_model
    recall_average = recall_score(Y_test, y_predict, average="binary")
  File "C:Usersanca_elena.moisaAppDataLocalProgramsPythonPython36libsite-packagessklearnmetricsclassification.py", line 1359, in recall_score
    sample_weight=sample_weight)
  File "C:Usersanca_elena.moisaAppDataLocalProgramsPythonPython36libsite-packagessklearnmetricsclassification.py", line 1036, in precision_recall_fscore_support
    (pos_label, present_labels))
ValueError: pos_label=1 is not a valid label: array(['neg', 'pos'],
      dtype='<U3')

I tried to transform ‘pos’ in 1 and ‘neg’ in 0 this way:

for i in range(len(Y_test)):
     if 'neg' in Y_test[i]:
         Y_test[i] = 0
     else:
         Y_test[i] = 1

But this is giving me another error:

    C:Usersanca_elena.moisaAppDataLocalProgramsPythonPython36libsite-packagessklearnmetricsclassification.py:181: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison
  score = y_true == y_pred
Traceback (most recent call last):
  File "G:/PyCharmProjects/NB/accuracy/script.py", line 812, in <module>
    main()
  File "G:/PyCharmProjects/NB/accuracy/script.py", line 91, in main
    evaluate_model(model, train_pos_vec, train_neg_vec, test_pos_vec, test_neg_vec, False)
  File "G:/PyCharmProjects/NB/accuracy/script.py", line 648, in evaluate_model
    recall_average = recall_score(Y_test, y_predict, average="binary")
  File "C:Usersanca_elena.moisaAppDataLocalProgramsPythonPython36libsite-packagessklearnmetricsclassification.py", line 1359, in recall_score
    sample_weight=sample_weight)
  File "C:Usersanca_elena.moisaAppDataLocalProgramsPythonPython36libsite-packagessklearnmetricsclassification.py", line 1026, in precision_recall_fscore_support
    present_labels = unique_labels(y_true, y_pred)
  File "C:Usersanca_elena.moisaAppDataLocalProgramsPythonPython36libsite-packagessklearnutilsmulticlass.py", line 103, in unique_labels
    raise ValueError("Mix of label input types (string and number)")
ValueError: Mix of label input types (string and number)

What I am trying to do is to obtain the metrics: accuracy, precision, recall, f_measure. With average='weighted', I obtain the same result: accuracy=recall. I guess this is not correct, so I changed the average='binary', but I have those errors. Any ideas?

Asked By: Mr. Wizard

||
Source

Answers:

recall_average = recall_score(Y_test, y_predict, average="binary", pos_label="neg")

Use "neg" or "pos" as pos_label and this error won’t raise again.

Answered By: Steve

Indicate your positive class with (pos_label=pos)

So use:

Recall=recall_score(Y_test, Y_predict, pos_label='pos')
Answered By: Bowale Samuel

When you face this error it means the values of your target variable are not the expected one for recall_score(), which by default are 1 for positive case and 0 for negative case [This also applies to precision_score()]

From the error you mentioned:

pos_label=1 is not a valid label: array(['neg', 'pos']

It is clear that values for your positive scenarios is pos instead of 1 and for the negative neg instead of 0.

Then you have two options to fix this mismatch:

  • Changing the value default in the recall_score() to consider positive scenarios when pos appears with:
recall_average = recall_score(Y_test, y_predict, average="binary", pos_label='pos')
  • Changing the values of the target variable in your dataset to be 1 or 0
Y_test = Y_test.map({'pos': 1, 'neg': 0}).astype(int)
Answered By: Ignacio Alorre

recall_average = recall_score(Y_test, y_predict, pos_label="no")

pos_label has array of only ["yes","no"]

Answered By: Rohan Devaki