Outputting the prediction scores from MLP keras inferencing

Question

Following the keras tutorial MLP classification here: https://keras.io/examples/nlp/multi_label_classification/.

I am able to successfully train a model and print out the top 3 predicted labels using the code below. I would also like to print out the prediction scores too. I can’t seem to find how to do that in the documentation.

# Create a model for inference.
model_for_inference = keras.Sequential([text_vectorizer, shallow_mlp_model])

# Create a small dataset just for demoing inference.
inference_dataset = make_dataset(test_df.sample(100), is_train=False)
text_batch, label_batch = next(iter(inference_dataset))
predicted_probabilities = model_for_inference.predict(text_batch)

# Perform inference.
for i, text in enumerate(text_batch[:5]):
    label = label_batch[i].numpy()[None, ...]
    print(f"Abstract: {text}")
    predicted_proba = [proba for proba in predicted_probabilities[i]]
    top_3_labels = [
        x
        for _, x in sorted(
            zip(predicted_probabilities[i], lookup.get_vocabulary()),
            key=lambda pair: pair[0],
            reverse=True,
        )
    ][:3]
    print(f"Predicted Label(s): ({', '.join([label for label in top_3_labels])})")
    print(" ")

Asked By: noob_23

||

Source

Answer 1

You can predicted the most likely label (change the [:3] to [0]) and use some sklearn function to get the standar accuracy.

Also you can take all the probabilities for the labels and use the top k accuracy from sklearn

Answered By: itogaston

Answer 2

To access probabilities change this part:

top_3_labels = [
        x
        for _, x in sorted(
            zip(predicted_probabilities[i], lookup.get_vocabulary()),
            key=lambda pair: pair[0],
            reverse=True,
        )
    ][:3]
    print(f"Predicted Label(s): ({', '.join([label for label in top_3_labels])})")
    print(" ")

to this:

top_3_labels = [
        (p, x)
        for p, x in sorted(
            zip(predicted_probabilities[i], lookup.get_vocabulary()),
            key=lambda pair: pair[0],
            reverse=True,
        )
    ][:3]
    print(f"Predicted Label(s): ({', '.join([l[1] for l in top_3_labels])})")
    print(f"Predicted Probabilities(s): ({', '.join([l[0] for l in top_3_labels])})")
    print(" ")

Answered By: Vlad

Outputting the prediction scores from MLP keras inferencing

Question:

Answers: