How to get the same accuracy value as model.fit

Question

In the following keras model, I added "my_accuracy" and "accuracy" into the model.compile, and then run model.fit:

import tensorflow as tf
import numpy as np

xx = np.random.rand(28,28,1).astype(np.float32)

yy = np.random.rand(28,28,1).astype(np.float32)

zz = np.random.rand(28,28,1).astype(np.float32)

x = tf.keras.layers.Input(shape=(28, 28, 1))
y = tf.keras.layers.Dense(1)(x)

model = tf.keras.Model(x, y)
model.summary()

def my_accuracy(y_true, y_pred):
    return tf.keras.metrics.binary_accuracy(tf.round(y_true), tf.round(y_pred))

model.compile(
    optimizer = tf.keras.optimizers.Adam(),
    loss = tf.keras.losses.MeanSquaredError(),
    metrics = [
        my_accuracy,
        "accuracy",
    ]
)

history = model.fit(xx, yy, epochs = 2, verbose = 1)

Here is the output:

Model: "model_1"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_2 (InputLayer)        [(None, 28, 28, 1)]       0         
                                                                 
 dense_3 (Dense)             (None, 28, 28, 1)         2         
                                                                 
=================================================================
Total params: 2
Trainable params: 2
Non-trainable params: 0
_________________________________________________________________
Epoch 1/2
WARNING:tensorflow:Model was constructed with shape (None, 28, 28, 1) for input KerasTensor(type_spec=TensorSpec(shape=(None, 28, 28, 1), dtype=tf.float32, name='input_2'), name='input_2', description="created by layer 'input_2'"), but it was called on an input with incompatible shape (None, 28, 1, 1).

WARNING:tensorflow:Model was constructed with shape (None, 28, 28, 1) for input KerasTensor(type_spec=TensorSpec(shape=(None, 28, 28, 1), dtype=tf.float32, name='input_2'), name='input_2', description="created by layer 'input_2'"), but it was called on an input with incompatible shape (None, 28, 1, 1).

WARNING:tensorflow:Model was constructed with shape (None, 28, 28, 1) for input KerasTensor(type_spec=TensorSpec(shape=(None, 28, 28, 1), dtype=tf.float32, name='input_2'), name='input_2', description="created by layer 'input_2'"), but it was called on an input with incompatible shape (None, 28, 1, 1).

WARNING:tensorflow:Model was constructed with shape (None, 28, 28, 1) for input KerasTensor(type_spec=TensorSpec(shape=(None, 28, 28, 1), dtype=tf.float32, name='input_2'), name='input_2', description="created by layer 'input_2'"), but it was called on an input with incompatible shape (None, 28, 1, 1).

1/1 [==============================] - 0s 354ms/step - loss: 0.1529 - my_accuracy: 0.4566 - accuracy: 0.0000e+00
Epoch 2/2
1/1 [==============================] - 0s 4ms/step - loss: 0.1527 - my_accuracy: 0.4554 - accuracy: 0.0000e+00

And then, call the my_accuracy directly:

my_pred = model.predict(yy, verbose=0)
my_acc = my_accuracy(yy, my_pred).numpy()

print("my_acc: {}", my_acc)

And the result:

my_acc: {} [[[0. 0. 0. ... 0. 0. 0.]
  [0. 0. 1. ... 0. 0. 1.]
  [0. 0. 1. ... 1. 0. 1.]
  ...
  [1. 0. 1. ... 1. 1. 1.]
  [1. 0. 1. ... 1. 0. 1.]
  [0. 0. 1. ... 0. 1. 0.]]

 [[0. 0. 0. ... 0. 0. 0.]
  [0. 0. 1. ... 0. 0. 1.]
  [0. 0. 1. ... 1. 0. 1.]
  ...
  [1. 0. 1. ... 1. 1. 1.]
  [1. 0. 1. ... 1. 0. 1.]
  [0. 0. 1. ... 0. 1. 0.]]

 [[0. 0. 0. ... 0. 0. 0.]
  [0. 0. 1. ... 0. 0. 1.]
  [0. 0. 1. ... 1. 0. 1.]
  ...
  [1. 0. 1. ... 1. 1. 1.]
  [1. 0. 1. ... 1. 0. 1.]
  [0. 0. 1. ... 0. 1. 0.]]

 ...

 [[0. 0. 0. ... 0. 0. 0.]
  [0. 0. 1. ... 0. 0. 1.]
  [0. 0. 1. ... 1. 0. 1.]
  ...
  [1. 0. 1. ... 1. 1. 1.]
  [1. 0. 1. ... 1. 0. 1.]
  [0. 0. 1. ... 0. 1. 0.]]

 [[0. 0. 0. ... 0. 0. 0.]
  [0. 0. 1. ... 0. 0. 1.]
  [0. 0. 1. ... 1. 0. 1.]
  ...
  [1. 0. 1. ... 1. 1. 1.]
  [1. 0. 1. ... 1. 0. 1.]
  [0. 0. 1. ... 0. 1. 0.]]

 [[0. 0. 0. ... 0. 0. 0.]
  [0. 0. 1. ... 0. 0. 1.]
  [0. 0. 1. ... 1. 0. 1.]
  ...
  [1. 0. 1. ... 1. 1. 1.]
  [1. 0. 1. ... 1. 0. 1.]
  [0. 0. 1. ... 0. 1. 0.]]]

For "my_accuracy":

When calling my_accuracy directly, the result is an array, but the value of model.fit is just a number 0.4566.

How to get the same result as model.fit (0.4566) when calling the method directly?

1/1 [==============================] - 0s 354ms/step - loss: 0.1529 - my_accuracy: 0.4566 - accuracy: 0.0000e+00

For "accuracy":

What is the real method of accuracy? Why is the value equal to 0.0000e+00?

For the warning:

Why this warning says shape (None, 28, 1, 1)? Where is the shape (None, 28, 1, 1)?

WARNING:tensorflow:Model was constructed with shape (None, 28, 28, 1) for input KerasTensor(type_spec=TensorSpec(shape=(None, 28, 28, 1), dtype=tf.float32, name='input_2'), name='input_2', description="created by layer 'input_2'"), but it was called on an input with incompatible shape (None, 28, 1, 1).

Asked By: stackbiz

||

Source

Answer 1

Lot of your problems stem from a wrong input_shape. You have 28,28,1 sized data. But your model’s input shape is (None, 28, 28, 1) i.e. TensorFlow automatically adds a batch dimension in the front. So let’s change your model as follows.

x = tf.keras.layers.Input(shape=(28, 1))
y = tf.keras.layers.Dense(1)(x)

model = tf.keras.Model(x, y)

This fixes the model.

What is the real method of accuracy? Why is the value equal to 0.0000e+00?

You have float values in your labels (i.e. yy). Accuracy is computed with labels being 0 or 1. None of the predictions going to exactly match when they are floating values, thus you get an accuracy of 0. Replace yy with something like np.random.choice([0, 1], size=(28,28,1)) and you’ll get non-zero accuracy, matching the value computed by my_accuracy.

Why this warning says shape (None, 28, 1, 1)? Where is the shape (None, 28, 1, 1)?

This is basically saying the model is not getting correct input shape. It’s because of the incorrect input shape of the model.

Answered By: thushv89

How to get the same accuracy value as model.fit

Question:

Answers: