Tensorflow – ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type float)
Question:
Continuation from previous question: Tensorflow – TypeError: 'int' object is not iterable
My training data is a list of lists each comprised of 1000 floats. For example, x_train[0] =
[0.0, 0.0, 0.1, 0.25, 0.5, ...]
Here is my model:
model = Sequential()
model.add(LSTM(128, activation='relu',
input_shape=(1000, 1), return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(128, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(32, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(1, activation='sigmoid'))
opt = tf.keras.optimizers.Adam(lr=1e-3, decay=1e-5)
model.compile(optimizer='rmsprop',
loss='binary_crossentropy',
metrics=['accuracy'])
model.fit(x_train, y_train, epochs=3, validation_data=(x_test, y_test))
Here is the error I’m getting:
Traceback (most recent call last):
File "C:UsersbencuDesktopProjectFilesCodeProgram.py", line 88, in FitModel
model.fit(x_train, y_train, epochs=3, validation_data=(x_test, y_test))
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythonkerasenginetraining.py", line 728, in fit
use_multiprocessing=use_multiprocessing)
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythonkerasenginetraining_v2.py", line 224, in fit
distribution_strategy=strategy)
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythonkerasenginetraining_v2.py", line 547, in _process_training_inputs
use_multiprocessing=use_multiprocessing)
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythonkerasenginetraining_v2.py", line 606, in _process_inputs
use_multiprocessing=use_multiprocessing)
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythonkerasenginedata_adapter.py", line 479, in __init__
batch_size=batch_size, shuffle=shuffle, **kwargs)
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythonkerasenginedata_adapter.py", line 321, in __init__
dataset_ops.DatasetV2.from_tensors(inputs).repeat()
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythondataopsdataset_ops.py", line 414, in from_tensors
return TensorDataset(tensors)
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythondataopsdataset_ops.py", line 2335, in __init__
element = structure.normalize_element(element)
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythondatautilstructure.py", line 111, in normalize_element
ops.convert_to_tensor(t, name="component_%d" % i))
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythonframeworkops.py", line 1184, in convert_to_tensor
return convert_to_tensor_v2(value, dtype, preferred_dtype, name)
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythonframeworkops.py", line 1242, in convert_to_tensor_v2
as_ref=False)
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythonframeworkops.py", line 1296, in internal_convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythonframeworktensor_conversion_registry.py", line 52, in _default_conversion_function
return constant_op.constant(value, dtype, name=name)
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythonframeworkconstant_op.py", line 227, in constant
allow_broadcast=True)
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythonframeworkconstant_op.py", line 235, in _constant_impl
t = convert_to_eager_tensor(value, ctx, dtype)
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythonframeworkconstant_op.py", line 96, in convert_to_eager_tensor
return ops.EagerTensor(value, ctx.device_name, dtype)
ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type float).
I’ve tried googling the error myself, I found something about using the tf.convert_to_tensor function. I tried passing my training and testing lists through this but the function won’t take them.
Answers:
TL;DR Several possible errors, most fixed with x = np.asarray(x).astype('float32').
Others may be faulty data preprocessing; ensure everything is properly formatted (categoricals, nans, strings, etc). Below shows what the model expects:
[print(i.shape, i.dtype) for i in model.inputs]
[print(o.shape, o.dtype) for o in model.outputs]
[print(l.name, l.input_shape, l.dtype) for l in model.layers]
The problem’s rooted in using lists as inputs, as opposed to Numpy arrays; Keras/TF doesn’t support former. A simple conversion is: x_array = np.asarray(x_list).
The next step’s to ensure data is fed in expected format; for LSTM, that’d be a 3D tensor with dimensions (batch_size, timesteps, features) – or equivalently, (num_samples, timesteps, channels). Lastly, as a debug pro-tip, print ALL the shapes for your data. Code accomplishing all of the above, below:
Sequences = np.asarray(Sequences)
Targets = np.asarray(Targets)
show_shapes()
Sequences = np.expand_dims(Sequences, -1)
Targets = np.expand_dims(Targets, -1)
show_shapes()
# OUTPUTS
Expected: (num_samples, timesteps, channels)
Sequences: (200, 1000)
Targets: (200,)
Expected: (num_samples, timesteps, channels)
Sequences: (200, 1000, 1)
Targets: (200, 1)
As a bonus tip, I notice you’re running via main(), so your IDE probably lacks a Jupyter-like cell-based execution; I strongly recommend the Spyder IDE. It’s as simple as adding # In[], and pressing Ctrl + Enter below:
Function used:
def show_shapes(): # can make yours to take inputs; this'll use local variable values
print("Expected: (num_samples, timesteps, channels)")
print("Sequences: {}".format(Sequences.shape))
print("Targets: {}".format(Targets.shape))
Could also happen due to a difference in versions (I had to move back from tensorflow 2.1.0 to 2.0.0.beta1 in order to solve this issue).
After trying everything above with no success, I found that my problem was that one of the columns from my data had boolean values. Converting everything into np.float32 solved the issue!
import numpy as np
X = np.asarray(X).astype(np.float32)
I had many different inputs and target variables and didn’t know which one was causing the problem.
To find out on which variable it breaks you can add a print value in the library package using the path is specified in your stack strace:
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythonframeworkconstant_op.py", line 96, in convert_to_eager_tensor
return ops.EagerTensor(value, ctx.device_name,
Adding a print statement in this part of the code allowed me to see which input was causing the problem:
constant_op.py:
....
dtype = dtype.as_datatype_enum
except AttributeError:
dtype = dtypes.as_dtype(dtype).as_datatype_enum
ctx.ensure_initialized()
print(value) # <--------------------- PUT PRINT HERE
return ops.EagerTensor(value, ctx.device_name, dtype)
After observing which value was problematic conversion from int to astype(np.float32) resolved the problem.
You’d better use this, it is because of the uncompatible version of keras
from keras import backend as K
X_train1 = K.cast_to_floatx(X_train)
y_train1 = K.cast_to_floatx(y_train)
This is a HIGHLY misleading error, as this is basically a general error, which might have NOTHING to do with floats.
For example in my case it was caused by a string column of the pandas dataframe having some np.NaN values in it. Go figure!
Fixed it by replacing them with empty strings:
df.fillna(value='', inplace=True)
Or to be more specific doing this ONLY for the string (eg ‘object’) columns:
cols = df.select_dtypes(include=['object'])
for col in cols.columns.values:
df[col] = df[col].fillna('')
You may want to check data types in input data set or array and than convert it to float32:
train_X[:2, :].view()
#array([[4.6, 3.1, 1.5, 0.2],
# [5.9, 3.0, 5.1, 1.8]], dtype=object)
train_X = train_X.astype(np.float32)
#array([[4.6, 3.1, 1.5, 0.2],
# [5.9, 3. , 5.1, 1.8]], dtype=float32)
This should do the trick:
x_train = np.asarray(x_train).astype(np.float32)
y_train = np.asarray(y_train).astype(np.float32)
Use this if you are using a DataFrame and has multiple columns type:
numeric_list = df.select_dtypes(include=[np.number]).columns
df[numeric_list] = df[numeric_list].astype(np.float32)
Try with it for convert np.float32 to tf.float32 (datatype that read keras and tensorflow):
tf.convert_to_tensor(X_train, dtype=tf.float32)
In my case, it didn’t work to cast to np.float32.
For me, everything ran normally during training (probably because I was using tf.data.Dataset.from_generator as input for fit()), but when I was trying to call predict() on 1 instance (using a np.array), the error shows up.
As a solution, I had to reshape the array x_array.reshape(1, -1) before calling predict and it worked.
I avoided this problem by enforcing floating-point format during data import:
df = pd.read_csv(‘titanic.csv’, dtype=’float’)
Just had the same issue and it ended up being because I was trying to pass an array of array objects, not an array of arrays as expected. Hope this helps someone in the future!
try
X_train =t ensorflow.convert_to_tensor(X_train, dtype=tensorflow.float32)
y_train = tensorflow.convert_to_tensor(y_train, dtype=tensorflow.float32)
X_test = tensorflow.convert_to_tensor(X_test, dtype=tensorflow.float32)
y_test = tensorflow.convert_to_tensor(y_test, dtype=tensorflow.float32)
As answered by most people above, converting data in np.float32 / float32 by various means already told here.
While doing so if you get another error which is "ValueError: setting an array element with a sequence".
In that case, try converting your data into type list and then converting it into type tensor as you were trying earlier.
The selected answer from OverLordGoldDragon provided valuable clues in my situation but I had to spend a few hours trying to rectify my particular situation. So adding some notes here to help any one else in a similar situation.
I am producing the input to my keras.Sequential model.fit() via the TimeseriesGenerator class from keras.preprocessing.sequences module. This class was outputting a 5d sequence with pythons inbuilt int/float types with following structure-
- Level 5 – Batches(tuple of 10 batches since my batch size was 100
samples for a training set containing 1000 items)
- Level 4 – X and Y variables( tuple of 2 n-dimensional numpy arrays, one representing the x variable and the other the y variable for the batch)
- Level 3
3 dimensional nd numpy array of 100 samples each
-for X variable a 100(batch size)x20(sequence length)x10(no of features) array,
-for Y variable the last dimension is of size 1 instead of 10
- Level 2 – each sample is a 2d array of 20(sequence length)x10(no of
features) for x and 20×1 for y variable
- Level 1 – is a scalar of
native python types
When I followed the advise here and tried to convert this data structure to a x = np.asarray(x).astype('float32'), this was failing as the standard output from TimeseriesGenerator has tuples instead of arrays/lists at the first two levels which are immutable and in-homogenous in length (as the last batch has fewer samples than the others since it can only take in the residual number of samples (79 in my case)).
I tried a lot of things to make this casting work on the output of TimeseriesGEnerator, but finally I found the simplest solution was to explicitly cast the inputs to the TimeseriesGenerator constructor using the suggested method (x = np.asarray(x).astype('float32')). The problem was that these numpy nd arrays were being output by Preprocessing pipelines which retain native python int and float types in them. Once I cast these arrays to numpy float32 type, the model.fit() stopped giving the ValueError for failing to convert NumpyArray to Tensor Flow.
Hoping this note helps someone else who is facing this issue.
I encountered the same issue with some Pandas Series, thinking
data = my_series.to_numpy()
would be enough, but it would only provide an Object dtype and forcing to float64 or whatever didn’t work.
The issue was solved by using
data = my_series.to_list()
instead. Then
dataset = tf.data.Dataset.from_tensor_slices((data, labels))
worked as intended.
Continuation from previous question: Tensorflow – TypeError: 'int' object is not iterable
My training data is a list of lists each comprised of 1000 floats. For example, x_train[0] =
[0.0, 0.0, 0.1, 0.25, 0.5, ...]
Here is my model:
model = Sequential()
model.add(LSTM(128, activation='relu',
input_shape=(1000, 1), return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(128, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(32, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(1, activation='sigmoid'))
opt = tf.keras.optimizers.Adam(lr=1e-3, decay=1e-5)
model.compile(optimizer='rmsprop',
loss='binary_crossentropy',
metrics=['accuracy'])
model.fit(x_train, y_train, epochs=3, validation_data=(x_test, y_test))
Here is the error I’m getting:
Traceback (most recent call last):
File "C:UsersbencuDesktopProjectFilesCodeProgram.py", line 88, in FitModel
model.fit(x_train, y_train, epochs=3, validation_data=(x_test, y_test))
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythonkerasenginetraining.py", line 728, in fit
use_multiprocessing=use_multiprocessing)
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythonkerasenginetraining_v2.py", line 224, in fit
distribution_strategy=strategy)
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythonkerasenginetraining_v2.py", line 547, in _process_training_inputs
use_multiprocessing=use_multiprocessing)
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythonkerasenginetraining_v2.py", line 606, in _process_inputs
use_multiprocessing=use_multiprocessing)
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythonkerasenginedata_adapter.py", line 479, in __init__
batch_size=batch_size, shuffle=shuffle, **kwargs)
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythonkerasenginedata_adapter.py", line 321, in __init__
dataset_ops.DatasetV2.from_tensors(inputs).repeat()
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythondataopsdataset_ops.py", line 414, in from_tensors
return TensorDataset(tensors)
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythondataopsdataset_ops.py", line 2335, in __init__
element = structure.normalize_element(element)
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythondatautilstructure.py", line 111, in normalize_element
ops.convert_to_tensor(t, name="component_%d" % i))
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythonframeworkops.py", line 1184, in convert_to_tensor
return convert_to_tensor_v2(value, dtype, preferred_dtype, name)
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythonframeworkops.py", line 1242, in convert_to_tensor_v2
as_ref=False)
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythonframeworkops.py", line 1296, in internal_convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythonframeworktensor_conversion_registry.py", line 52, in _default_conversion_function
return constant_op.constant(value, dtype, name=name)
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythonframeworkconstant_op.py", line 227, in constant
allow_broadcast=True)
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythonframeworkconstant_op.py", line 235, in _constant_impl
t = convert_to_eager_tensor(value, ctx, dtype)
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythonframeworkconstant_op.py", line 96, in convert_to_eager_tensor
return ops.EagerTensor(value, ctx.device_name, dtype)
ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type float).
I’ve tried googling the error myself, I found something about using the tf.convert_to_tensor function. I tried passing my training and testing lists through this but the function won’t take them.
TL;DR Several possible errors, most fixed with x = np.asarray(x).astype('float32').
Others may be faulty data preprocessing; ensure everything is properly formatted (categoricals, nans, strings, etc). Below shows what the model expects:
[print(i.shape, i.dtype) for i in model.inputs]
[print(o.shape, o.dtype) for o in model.outputs]
[print(l.name, l.input_shape, l.dtype) for l in model.layers]
The problem’s rooted in using lists as inputs, as opposed to Numpy arrays; Keras/TF doesn’t support former. A simple conversion is: x_array = np.asarray(x_list).
The next step’s to ensure data is fed in expected format; for LSTM, that’d be a 3D tensor with dimensions (batch_size, timesteps, features) – or equivalently, (num_samples, timesteps, channels). Lastly, as a debug pro-tip, print ALL the shapes for your data. Code accomplishing all of the above, below:
Sequences = np.asarray(Sequences)
Targets = np.asarray(Targets)
show_shapes()
Sequences = np.expand_dims(Sequences, -1)
Targets = np.expand_dims(Targets, -1)
show_shapes()
# OUTPUTS
Expected: (num_samples, timesteps, channels)
Sequences: (200, 1000)
Targets: (200,)
Expected: (num_samples, timesteps, channels)
Sequences: (200, 1000, 1)
Targets: (200, 1)
As a bonus tip, I notice you’re running via main(), so your IDE probably lacks a Jupyter-like cell-based execution; I strongly recommend the Spyder IDE. It’s as simple as adding # In[], and pressing Ctrl + Enter below:
Function used:
def show_shapes(): # can make yours to take inputs; this'll use local variable values
print("Expected: (num_samples, timesteps, channels)")
print("Sequences: {}".format(Sequences.shape))
print("Targets: {}".format(Targets.shape))
Could also happen due to a difference in versions (I had to move back from tensorflow 2.1.0 to 2.0.0.beta1 in order to solve this issue).
After trying everything above with no success, I found that my problem was that one of the columns from my data had boolean values. Converting everything into np.float32 solved the issue!
import numpy as np
X = np.asarray(X).astype(np.float32)
I had many different inputs and target variables and didn’t know which one was causing the problem.
To find out on which variable it breaks you can add a print value in the library package using the path is specified in your stack strace:
File "C:UsersbencuAppDataLocalProgramsPythonPython37libsite-packagestensorflow_corepythonframeworkconstant_op.py", line 96, in convert_to_eager_tensor
return ops.EagerTensor(value, ctx.device_name,
Adding a print statement in this part of the code allowed me to see which input was causing the problem:
constant_op.py:
....
dtype = dtype.as_datatype_enum
except AttributeError:
dtype = dtypes.as_dtype(dtype).as_datatype_enum
ctx.ensure_initialized()
print(value) # <--------------------- PUT PRINT HERE
return ops.EagerTensor(value, ctx.device_name, dtype)
After observing which value was problematic conversion from int to astype(np.float32) resolved the problem.
You’d better use this, it is because of the uncompatible version of keras
from keras import backend as K
X_train1 = K.cast_to_floatx(X_train)
y_train1 = K.cast_to_floatx(y_train)
This is a HIGHLY misleading error, as this is basically a general error, which might have NOTHING to do with floats.
For example in my case it was caused by a string column of the pandas dataframe having some np.NaN values in it. Go figure!
Fixed it by replacing them with empty strings:
df.fillna(value='', inplace=True)
Or to be more specific doing this ONLY for the string (eg ‘object’) columns:
cols = df.select_dtypes(include=['object'])
for col in cols.columns.values:
df[col] = df[col].fillna('')
You may want to check data types in input data set or array and than convert it to float32:
train_X[:2, :].view()
#array([[4.6, 3.1, 1.5, 0.2],
# [5.9, 3.0, 5.1, 1.8]], dtype=object)
train_X = train_X.astype(np.float32)
#array([[4.6, 3.1, 1.5, 0.2],
# [5.9, 3. , 5.1, 1.8]], dtype=float32)
This should do the trick:
x_train = np.asarray(x_train).astype(np.float32)
y_train = np.asarray(y_train).astype(np.float32)
Use this if you are using a DataFrame and has multiple columns type:
numeric_list = df.select_dtypes(include=[np.number]).columns
df[numeric_list] = df[numeric_list].astype(np.float32)
Try with it for convert np.float32 to tf.float32 (datatype that read keras and tensorflow):
tf.convert_to_tensor(X_train, dtype=tf.float32)
In my case, it didn’t work to cast to np.float32.
For me, everything ran normally during training (probably because I was using tf.data.Dataset.from_generator as input for fit()), but when I was trying to call predict() on 1 instance (using a np.array), the error shows up.
As a solution, I had to reshape the array x_array.reshape(1, -1) before calling predict and it worked.
I avoided this problem by enforcing floating-point format during data import:
df = pd.read_csv(‘titanic.csv’, dtype=’float’)
Just had the same issue and it ended up being because I was trying to pass an array of array objects, not an array of arrays as expected. Hope this helps someone in the future!
try
X_train =t ensorflow.convert_to_tensor(X_train, dtype=tensorflow.float32)
y_train = tensorflow.convert_to_tensor(y_train, dtype=tensorflow.float32)
X_test = tensorflow.convert_to_tensor(X_test, dtype=tensorflow.float32)
y_test = tensorflow.convert_to_tensor(y_test, dtype=tensorflow.float32)
As answered by most people above, converting data in np.float32 / float32 by various means already told here.
While doing so if you get another error which is "ValueError: setting an array element with a sequence".
In that case, try converting your data into type list and then converting it into type tensor as you were trying earlier.
The selected answer from OverLordGoldDragon provided valuable clues in my situation but I had to spend a few hours trying to rectify my particular situation. So adding some notes here to help any one else in a similar situation.
I am producing the input to my keras.Sequential model.fit() via the TimeseriesGenerator class from keras.preprocessing.sequences module. This class was outputting a 5d sequence with pythons inbuilt int/float types with following structure-
- Level 5 – Batches(tuple of 10 batches since my batch size was 100
samples for a training set containing 1000 items) - Level 4 – X and Y variables( tuple of 2 n-dimensional numpy arrays, one representing the x variable and the other the y variable for the batch)
- Level 3
3 dimensional nd numpy array of 100 samples each
-for X variable a 100(batch size)x20(sequence length)x10(no of features) array,
-for Y variable the last dimension is of size 1 instead of 10 - Level 2 – each sample is a 2d array of 20(sequence length)x10(no of
features) for x and 20×1 for y variable - Level 1 – is a scalar of
native python types
When I followed the advise here and tried to convert this data structure to a x = np.asarray(x).astype('float32'), this was failing as the standard output from TimeseriesGenerator has tuples instead of arrays/lists at the first two levels which are immutable and in-homogenous in length (as the last batch has fewer samples than the others since it can only take in the residual number of samples (79 in my case)).
I tried a lot of things to make this casting work on the output of TimeseriesGEnerator, but finally I found the simplest solution was to explicitly cast the inputs to the TimeseriesGenerator constructor using the suggested method (x = np.asarray(x).astype('float32')). The problem was that these numpy nd arrays were being output by Preprocessing pipelines which retain native python int and float types in them. Once I cast these arrays to numpy float32 type, the model.fit() stopped giving the ValueError for failing to convert NumpyArray to Tensor Flow.
Hoping this note helps someone else who is facing this issue.
I encountered the same issue with some Pandas Series, thinking
data = my_series.to_numpy()
would be enough, but it would only provide an Object dtype and forcing to float64 or whatever didn’t work.
The issue was solved by using
data = my_series.to_list()
instead. Then
dataset = tf.data.Dataset.from_tensor_slices((data, labels))
worked as intended.