Tensorflow estimator ValueError: logits and labels must have the same shape ((?, 1) vs (?,))

Question

I’m classifying movie reviews as positive or negative using binary crossentropy. So, when I’m trying to wrap my keras model with tensorflow estimator, I get the error:

Tensorflow estimator ValueError: logits and labels must have the same shape ((?, 1) vs (?,))

I’m using sigmoid activation as my last layer, guess I’m missing something trivial here. Any help?

from tensorflow import keras
import tensorflow as tf
print("Tensorflow {} loaded".format(tf.__version__))
import numpy as np

keras.__version__
from keras.datasets import imdb

(train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words=10000)
def vectorize_sequences(sequences, dimension=10000):
    # Create an all-zero matrix of shape (len(sequences), dimension)
    results = np.zeros((len(sequences), dimension))
    for i, sequence in enumerate(sequences):
        results[i, sequence] = 1.  # set specific indices of results[i] to 1s
    return results.astype('float32')

# Our vectorized training data
x_train = vectorize_sequences(train_data)

# Our vectorized test data
x_test = vectorize_sequences(test_data)

# Our vectorized labels
y_train = np.asarray(train_labels).astype('float32')
y_test = np.asarray(test_labels).astype('float32')

x_val = x_train[:10000]
partial_x_train = x_train[10000:]
y_val = y_train[:10000]
partial_y_train = y_train[10000:]

model = keras.models.Sequential()
model.add(keras.layers.Dense(16, activation='relu', input_shape=(10000,), name='reviews'))
model.add(keras.layers.Dense(16, activation='relu'))
model.add(keras.layers.Dense(1, activation='sigmoid'))
model.compile(optimizer='rmsprop',
              loss='binary_crossentropy',
              metrics=['accuracy'])
estimator_model = keras.estimator.model_to_estimator(keras_model=model)

def input_function(features,labels=None,shuffle=False,epochs=None,batch_size=None):
    input_fn = tf.estimator.inputs.numpy_input_fn(
        x={"reviews_input": features},
        y=labels,
        shuffle=shuffle,
        num_epochs=epochs,
        batch_size=batch_size
    )
    return input_fn

estimator_model.train(input_fn=input_function(partial_x_train, partial_y_train, True,20,512))
score = estimator_model.evaluate(input_function(x_val, labels=y_val))
print(score)

Asked By: Bluecrow

||

Source

Answer 1

You should reshape your labels as 2d-tensor (the first dimension will be the batch dimension and the second the scalar label):

# Our vectorized labels
y_train = np.asarray(train_labels).astype('float32').reshape((-1,1))
y_test = np.asarray(test_labels).astype('float32').reshape((-1,1))

Answered By: pfm

Answer 2

Check your network using model.summary()

You eventually need to thin down the network to have the same outputs as your classes.
For example, doing OCR for numbers needs and final output of Dense(10) (for numbers 0 to 9).

For example characterizing dogs vs. cats. The final layer has to have two outputs (0-dog, 1-cat)

Answered By: tj168

Answer 3

We can solve this problem by matching the output with the dimension of the label by adding a Flatten layer after the Dense layer:

model.add(Flatten())

or by adding:

model.add(GlobalAveragePooling2D())

See this GitHub issue for full details

Answered By: jonilyn2730

Answer 4

If you’re doing binary cross-entropy, then your dataset probably has 2 classes and the error is coming because your label vectors (both in testing and training) have the form [0,1,0,1,1,1,0,0,1,…]. To one-hot encode binary labels, the following function can be used: Labels = tf.one_hot(Labels, depth=2)

Answered By: Andrew Garcia

Answer 5

If you’re doing a binary classification, make sure that that your last Dense layer has just (None, 1) in its shape, not None, 2)

tf.keras.layers.Dense(1, activation="sigmoid") # binary activation output

Answered By: Oussama Ouardini

Tensorflow estimator ValueError: logits and labels must have the same shape ((?, 1) vs (?,))

Question:

Answers: