What values are returned from model.evaluate() in Keras?
Question:
I’ve got multiple outputs from my model from multiple Dense layers. My model has 'accuracy' as the only metric in compilation. I’d like to know the loss and accuracy for each output. This is some part of my code.
scores = model.evaluate(X_test, [y_test_one, y_test_two], verbose=1)
When I printed out the scores, this is the result.
[0.7185557290413819, 0.3189622712272771, 0.39959345855771927, 0.8470299135229717, 0.8016634374641469]
What are these numbers represent?
I’m new to Keras and this might be a trivial question. However, I have read the docs from Keras but I’m still not sure.
Answers:
Quoted from evaluate() method documentation:
Returns
Scalar test loss (if the model has a single output and no metrics) or
list of scalars (if the model has multiple outputs and/or metrics).
The attribute model.metrics_names will give you the display labels
for the scalar outputs.
Therefore, you can use metrics_names property of your model to find out what each of those values corresponds to. For example:
from keras import layers
from keras import models
import numpy as np
input_data = layers.Input(shape=(100,))
out_1 = layers.Dense(1)(input_data)
out_2 = layers.Dense(1)(input_data)
model = models.Model(input_data, [out_1, out_2])
model.compile(loss='mse', optimizer='adam', metrics=['mae'])
print(model.metrics_names)
outputs the following:
['loss', 'dense_1_loss', 'dense_2_loss', 'dense_1_mean_absolute_error', 'dense_2_mean_absolute_error']
which indicates what each of those numbers you see in the output of evaluate method corresponds to.
Further, if you have many layers then those dense_1 and dense_2 names might be a bit ambiguous. To resolve this ambiguity, you can assign names to your layers using name argument of layers (not necessarily on all of them but only on the input and output layers):
# ...
out_1 = layers.Dense(1, name='output_1')(input_data)
out_2 = layers.Dense(1, name='output_2')(input_data)
# ...
print(model.metrics_names)
which outputs a more clear description:
['loss', 'output_1_loss', 'output_2_loss', 'output_1_mean_absolute_error', 'output_2_mean_absolute_error']
We should be clear that the "loss" figure is the sum of ALL the losses calculated for each item in the x_test array. x_test would contain your test data and y_test would contain your labels. The loss figure is the sum of ALL the losses, not just one loss from one item in the x_test array.
I’ve got multiple outputs from my model from multiple Dense layers. My model has 'accuracy' as the only metric in compilation. I’d like to know the loss and accuracy for each output. This is some part of my code.
scores = model.evaluate(X_test, [y_test_one, y_test_two], verbose=1)
When I printed out the scores, this is the result.
[0.7185557290413819, 0.3189622712272771, 0.39959345855771927, 0.8470299135229717, 0.8016634374641469]
What are these numbers represent?
I’m new to Keras and this might be a trivial question. However, I have read the docs from Keras but I’m still not sure.
Quoted from evaluate() method documentation:
Returns
Scalar test loss (if the model has a single output and no metrics) or
list of scalars (if the model has multiple outputs and/or metrics).
The attributemodel.metrics_nameswill give you the display labels
for the scalar outputs.
Therefore, you can use metrics_names property of your model to find out what each of those values corresponds to. For example:
from keras import layers
from keras import models
import numpy as np
input_data = layers.Input(shape=(100,))
out_1 = layers.Dense(1)(input_data)
out_2 = layers.Dense(1)(input_data)
model = models.Model(input_data, [out_1, out_2])
model.compile(loss='mse', optimizer='adam', metrics=['mae'])
print(model.metrics_names)
outputs the following:
['loss', 'dense_1_loss', 'dense_2_loss', 'dense_1_mean_absolute_error', 'dense_2_mean_absolute_error']
which indicates what each of those numbers you see in the output of evaluate method corresponds to.
Further, if you have many layers then those dense_1 and dense_2 names might be a bit ambiguous. To resolve this ambiguity, you can assign names to your layers using name argument of layers (not necessarily on all of them but only on the input and output layers):
# ...
out_1 = layers.Dense(1, name='output_1')(input_data)
out_2 = layers.Dense(1, name='output_2')(input_data)
# ...
print(model.metrics_names)
which outputs a more clear description:
['loss', 'output_1_loss', 'output_2_loss', 'output_1_mean_absolute_error', 'output_2_mean_absolute_error']
We should be clear that the "loss" figure is the sum of ALL the losses calculated for each item in the x_test array. x_test would contain your test data and y_test would contain your labels. The loss figure is the sum of ALL the losses, not just one loss from one item in the x_test array.