Why is predict_on_batch is repeating the first output over and over?

Question:

I honestly don’t know how to describe my problem. Basically, it only uses the first values in my tables and repeats it over and over for the rest of the values in the table. Like I said, just look at the output.

Here’s my code:

import tensorflow as tf
import numpy as np

x = np.array([[1, 0, 0],     #x1
              [1, 0.5, 0],   #x2
              [1, 1, 0],   #x3
              [0.5, 1, 0],   #x4
              [0, 1, 0],     #x5
              [0, 1, 0.5],   #x6
              [0, 1, 1],   #x7
              [0, 0.5, 1],   #x8
              [0, 0, 1],     #x9
              [0.5, 0, 1],   #x10
              [1, 0, 1],   #x11
              [1, 0, 1]]) #x12
#Key:
#[red, orange, yellow, green, blue, purple]
y = np.array([[1,0,0,0,0,0],  #y1
              [0,1,0,0,0,0],  #y2
              [0,0,1,0,0,0],  #y3
              [0,0,0,1,0,0],  #y4
              [0,0,0,1,0,0],  #y5
              [0,0,0,1,0,0],  #y6
              [0,0,0,0,1,0],  #y7
              [0,0,0,0,1,0],  #y8
              [0,0,0,0,1,0],  #y9
              [0,0,0,0,0,1],  #y10
              [0,0,0,0,0,1],  #y11
              [1,0,0,0,0,0]]) #y12

# Define the model
model = tf.keras.models.Sequential()
model.add(tf.keras.Input(shape=(3)))
model.add(tf.keras.layers.Dense(10, activation=tf.keras.activations.sigmoid))
model.add(tf.keras.layers.Dense(10, activation=tf.keras.activations.sigmoid))
model.add(tf.keras.layers.Dense(10, activation=tf.keras.activations.sigmoid))
model.add(tf.keras.layers.Dense(10, activation=tf.keras.activations.sigmoid))
model.add(tf.keras.layers.Dense(10, activation=tf.keras.activations.sigmoid))
model.add(tf.keras.layers.Dense(10, activation=tf.keras.activations.sigmoid))
model.add(tf.keras.layers.Dense(10, activation=tf.keras.activations.sigmoid))
model.add(tf.keras.layers.Dense(10, activation=tf.keras.activations.sigmoid))
model.add(tf.keras.layers.Dense(10, activation=tf.keras.activations.sigmoid))
model.add(tf.keras.layers.Dense(10, activation=tf.keras.activations.sigmoid))
model.add(tf.keras.layers.Dense(6, activation=tf.keras.activations.sigmoid))

# Compile the model
model.compile(tf.keras.optimizers.Adam(learning_rate=0.1), "BinaryCrossentropy", metrics=[ 'binary_accuracy'])
model.summary()

history = model.fit(x, y, batch_size=1, epochs=500)

predictions = model.predict_on_batch(x)
print(predictions)

Here is the output:

[[0.16287932 0.07145664 0.08749434 0.26094046 0.25779992 0.16714773]
[0.16287932 0.07145664 0.08749434 0.26094046 0.25779992 0.16714773]
[0.16287932 0.07145664 0.08749434 0.26094046 0.25779992 0.16714773]
[0.16287932 0.07145664 0.08749434 0.26094046 0.25779992 0.16714773]
[0.16287932 0.07145664 0.08749434 0.26094046 0.25779992 0.16714773]
[0.16287932 0.07145664 0.08749434 0.26094046 0.25779992 0.16714773]
[0.16287932 0.07145664 0.08749434 0.26094046 0.25779992 0.16714773]
[0.16287932 0.07145664 0.08749434 0.26094046 0.25779992 0.16714773]
[0.16287932 0.07145664 0.08749434 0.26094046 0.25779992 0.16714773]
[0.16287932 0.07145664 0.08749434 0.26094046 0.25779992 0.16714773]
[0.16287932 0.07145664 0.08749434 0.26094046 0.25779992 0.16714773]
[0.16287932 0.07145664 0.08749434 0.26094046 0.25779992 0.16714773]]

Here is the link to the Replit cover page for more details:
https://replit.com/@EthanKantala/Color-Guesser-Tensorflow?v=1

I have tried adding more neurons and doing some research. I honestly have no idea what to do. Thanks for any help!

Answers:

I’ve spotted two problems with your code.
First of all, the alias of Binary Cross Entropy loss is binary_crossentropy so I got an error running your code on my machine.

Second thing. If you reduce your layers definition to

model.add(tf.keras.Input(shape=(3)))
model.add(tf.keras.layers.Dense(10, activation=tf.keras.activations.sigmoid))
model.add(tf.keras.layers.Dense(6, activation=tf.keras.activations.sigmoid))

Then it should work, at least on my machine it did. But also because you are training for 500 on such a small dataset with such a deep network and then predicting the same dataset, what you will get is actually almost complete memorisation of this dataset.

Answered By: UpmostScarab
Categories: questions Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.