Problem with tensor size in a TF Keras dense network, Flatten layer not working while training
Question:
I’ve been stuck for a while with a Keras model structure issue. I’m using a dataset with grayscale images of size (181,181,1). Image paths are stored in pandas DF along with class names.
Here’s the datagenerator:
trainData = train_generator.flow_from_dataframe(
dataframe=train_df,
x_col='Path',
y_col='CLASS',
image_size=(181,181),
color_mode='grayscale',
class_mode='categorical',
batch_size=64,
interpolation='nearest',
shuffle=True,
seed=42
)
valData = train_generator.flow_from_dataframe(
dataframe=valid_df,
x_col='Path',
y_col='CLASS',
image_size=(181,181),
color_mode='grayscale',
class_mode='categorical',
batch_size=64,
interpolation='nearest',
shuffle=True,
seed=42
)
Here’s the model in question:
model = Sequential([
layers.Rescaling(scale=1./255, offset = -1,input_shape=(181,181,1)),
layers.Flatten(),
Dense(units=90, activation='relu',kernel_initializer='he_normal',bias_initializer=biasInitializer),
Dense(units=45, activation='relu',kernel_initializer='he_normal',bias_initializer=biasInitializer),
Dense(units=20, activation='softmax',kernel_initializer='he_normal',bias_initializer=biasInitializer)
],name='myModel')
Model Summary:
model.summary()
Model: "myModel"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
rescaling_19 (Rescaling) (None, 181, 181, 1) 0
flatten_17 (Flatten) (None, 32761) 0
dense_44 (Dense) (None, 90) 2948580
dense_45 (Dense) (None, 45) 4095
dense_46 (Dense) (None, 20) 920
=================================================================
Total params: 2,953,595
Trainable params: 2,953,595
Non-trainable params: 0
Model compilation:
model.compile(optimizer=k.optimizers.Adam(learning_rate=0.001) , loss='categorical_crossentropy', metrics=['accuracy'])
Model Training:
epochs = 100
# train the model to the dataset
model.fit(x=trainData,
epochs=epochs,
verbose=1,
shuffle=True,
validation_data=valData)
Whenever I use flatten layer in the model, I get this error:
Node: 'myModel/flatten_17/Reshape'
Input to reshape is a tensor with 3145728 values, but the requested shape requires a multiple of 32761
[[{{node myModel/flatten_17/Reshape}}]] [Op:__inference_train_function_731522]
Here’s the full error transcript if that would help:
InvalidArgumentError Traceback (most recent call last)
Input In [79], in <cell line: 3>()
1 epochs = 65
2 # train the model to the dataset
----> 3 model.fit(x=trainData,
4 epochs=epochs,
5 verbose=1,
6 shuffle=True,
7 validation_data=valData)
File /usr/local/lib/python3.9/dist-packages/keras/utils/traceback_utils.py:67, in filter_traceback.<locals>.error_handler(*args, **kwargs)
65 except Exception as e: # pylint: disable=broad-except
66 filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67 raise e.with_traceback(filtered_tb) from None
68 finally:
69 del filtered_tb
File /usr/local/lib/python3.9/dist-packages/tensorflow/python/eager/execute.py:54, in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
52 try:
53 ctx.ensure_initialized()
---> 54 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
55 inputs, attrs, num_outputs)
56 except core._NotOkStatusException as e:
57 if name is not None:
InvalidArgumentError: Graph execution error:
Detected at node 'myModel/flatten_17/Reshape' defined at (most recent call last):
File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.9/dist-packages/ipykernel_launcher.py", line 17, in <module>
app.launch_new_instance()
File "/usr/local/lib/python3.9/dist-packages/traitlets/config/application.py", line 976, in launch_instance
app.start()
File "/usr/local/lib/python3.9/dist-packages/ipykernel/kernelapp.py", line 712, in start
self.io_loop.start()
File "/usr/local/lib/python3.9/dist-packages/tornado/platform/asyncio.py", line 215, in start
self.asyncio_loop.run_forever()
File "/usr/lib/python3.9/asyncio/base_events.py", line 601, in run_forever
self._run_once()
File "/usr/lib/python3.9/asyncio/base_events.py", line 1905, in _run_once
handle._run()
File "/usr/lib/python3.9/asyncio/events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "/usr/local/lib/python3.9/dist-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
await self.process_one()
File "/usr/local/lib/python3.9/dist-packages/ipykernel/kernelbase.py", line 499, in process_one
await dispatch(*args)
File "/usr/local/lib/python3.9/dist-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
await result
File "/usr/local/lib/python3.9/dist-packages/ipykernel/kernelbase.py", line 730, in execute_request
reply_content = await reply_content
File "/usr/local/lib/python3.9/dist-packages/ipykernel/ipkernel.py", line 383, in do_execute
res = shell.run_cell(
File "/usr/local/lib/python3.9/dist-packages/ipykernel/zmqshell.py", line 528, in run_cell
return super().run_cell(*args, **kwargs)
File "/usr/local/lib/python3.9/dist-packages/IPython/core/interactiveshell.py", line 2881, in run_cell
result = self._run_cell(
File "/usr/local/lib/python3.9/dist-packages/IPython/core/interactiveshell.py", line 2936, in _run_cell
return runner(coro)
File "/usr/local/lib/python3.9/dist-packages/IPython/core/async_helpers.py", line 129, in _pseudo_sync_runner
coro.send(None)
File "/usr/local/lib/python3.9/dist-packages/IPython/core/interactiveshell.py", line 3135, in run_cell_async
has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
File "/usr/local/lib/python3.9/dist-packages/IPython/core/interactiveshell.py", line 3338, in run_ast_nodes
if await self.run_code(code, result, async_=asy):
File "/usr/local/lib/python3.9/dist-packages/IPython/core/interactiveshell.py", line 3398, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "/tmp/ipykernel_61/2241442519.py", line 3, in <cell line: 3>
model.fit(x=trainData,
File "/usr/local/lib/python3.9/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
try:
File "/usr/local/lib/python3.9/dist-packages/keras/engine/training.py", line 1409, in fit
`tf.distribute.experimental.ParameterServerStrategy`.
File "/usr/local/lib/python3.9/dist-packages/keras/engine/training.py", line 1051, in train_function
self.loss_tracker.reset_states()
File "/usr/local/lib/python3.9/dist-packages/keras/engine/training.py", line 1040, in step_function
def __init__(self, *args, **kwargs):
File "/usr/local/lib/python3.9/dist-packages/keras/engine/training.py", line 1030, in run_step
def compute_loss(self, x=None, y=None, y_pred=None, sample_weight=None):
File "/usr/local/lib/python3.9/dist-packages/keras/engine/training.py", line 889, in train_step
File "/usr/local/lib/python3.9/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
try:
File "/usr/local/lib/python3.9/dist-packages/keras/engine/training.py", line 490, in __call__
# default.
File "/usr/local/lib/python3.9/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
try:
File "/usr/local/lib/python3.9/dist-packages/keras/engine/base_layer.py", line 1014, in __call__
RuntimeError: if `super().__init__()` was not called in the
File "/usr/local/lib/python3.9/dist-packages/keras/utils/traceback_utils.py", line 92, in error_handler
def error_handler(*args, **kwargs):
File "/usr/local/lib/python3.9/dist-packages/keras/engine/sequential.py", line 374, in call
def build(self, input_shape=None):
File "/usr/local/lib/python3.9/dist-packages/keras/engine/functional.py", line 458, in call
def _trackable_children(self, save_type="checkpoint", **kwargs):
File "/usr/local/lib/python3.9/dist-packages/keras/engine/functional.py", line 596, in _run_internal_graph
# Read final output shapes from layers_to_output_shapes.
File "/usr/local/lib/python3.9/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
try:
File "/usr/local/lib/python3.9/dist-packages/keras/engine/base_layer.py", line 1014, in __call__
RuntimeError: if `super().__init__()` was not called in the
File "/usr/local/lib/python3.9/dist-packages/keras/utils/traceback_utils.py", line 92, in error_handler
def error_handler(*args, **kwargs):
File "/usr/local/lib/python3.9/dist-packages/keras/layers/reshaping/flatten.py", line 98, in call
)
Node: 'myModel/flatten_17/Reshape'
Input to reshape is a tensor with 3145728 values, but the requested shape requires a multiple of 32761
[[{{node myModel/flatten_17/Reshape}}]] [Op:__inference_train_function_731522]
I’m using Python 3.9, tried Tensorflow-gpu 2.9.1 and 2.11.0 .. both having same issue.
Have been trying all sorts of suggestions I could find online, but still no luck. Appreciate any suggestions
Thanks!
Answers:
Well, Turned out the reason for this issue is, I was using the keyword image_size for setting the image size in the flow_from_dataframe functions instead of the correct keyword target_size … so the flow_from_dataframe was using the default value for target_size which is (256,256)
Fixing this and setting this target_size keyword to the correct image size of (181,181) fixed the issue for me
I’ve been stuck for a while with a Keras model structure issue. I’m using a dataset with grayscale images of size (181,181,1). Image paths are stored in pandas DF along with class names.
Here’s the datagenerator:
trainData = train_generator.flow_from_dataframe(
dataframe=train_df,
x_col='Path',
y_col='CLASS',
image_size=(181,181),
color_mode='grayscale',
class_mode='categorical',
batch_size=64,
interpolation='nearest',
shuffle=True,
seed=42
)
valData = train_generator.flow_from_dataframe(
dataframe=valid_df,
x_col='Path',
y_col='CLASS',
image_size=(181,181),
color_mode='grayscale',
class_mode='categorical',
batch_size=64,
interpolation='nearest',
shuffle=True,
seed=42
)
Here’s the model in question:
model = Sequential([
layers.Rescaling(scale=1./255, offset = -1,input_shape=(181,181,1)),
layers.Flatten(),
Dense(units=90, activation='relu',kernel_initializer='he_normal',bias_initializer=biasInitializer),
Dense(units=45, activation='relu',kernel_initializer='he_normal',bias_initializer=biasInitializer),
Dense(units=20, activation='softmax',kernel_initializer='he_normal',bias_initializer=biasInitializer)
],name='myModel')
Model Summary:
model.summary()
Model: "myModel"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
rescaling_19 (Rescaling) (None, 181, 181, 1) 0
flatten_17 (Flatten) (None, 32761) 0
dense_44 (Dense) (None, 90) 2948580
dense_45 (Dense) (None, 45) 4095
dense_46 (Dense) (None, 20) 920
=================================================================
Total params: 2,953,595
Trainable params: 2,953,595
Non-trainable params: 0
Model compilation:
model.compile(optimizer=k.optimizers.Adam(learning_rate=0.001) , loss='categorical_crossentropy', metrics=['accuracy'])
Model Training:
epochs = 100
# train the model to the dataset
model.fit(x=trainData,
epochs=epochs,
verbose=1,
shuffle=True,
validation_data=valData)
Whenever I use flatten layer in the model, I get this error:
Node: 'myModel/flatten_17/Reshape'
Input to reshape is a tensor with 3145728 values, but the requested shape requires a multiple of 32761
[[{{node myModel/flatten_17/Reshape}}]] [Op:__inference_train_function_731522]
Here’s the full error transcript if that would help:
InvalidArgumentError Traceback (most recent call last)
Input In [79], in <cell line: 3>()
1 epochs = 65
2 # train the model to the dataset
----> 3 model.fit(x=trainData,
4 epochs=epochs,
5 verbose=1,
6 shuffle=True,
7 validation_data=valData)
File /usr/local/lib/python3.9/dist-packages/keras/utils/traceback_utils.py:67, in filter_traceback.<locals>.error_handler(*args, **kwargs)
65 except Exception as e: # pylint: disable=broad-except
66 filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67 raise e.with_traceback(filtered_tb) from None
68 finally:
69 del filtered_tb
File /usr/local/lib/python3.9/dist-packages/tensorflow/python/eager/execute.py:54, in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
52 try:
53 ctx.ensure_initialized()
---> 54 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
55 inputs, attrs, num_outputs)
56 except core._NotOkStatusException as e:
57 if name is not None:
InvalidArgumentError: Graph execution error:
Detected at node 'myModel/flatten_17/Reshape' defined at (most recent call last):
File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.9/dist-packages/ipykernel_launcher.py", line 17, in <module>
app.launch_new_instance()
File "/usr/local/lib/python3.9/dist-packages/traitlets/config/application.py", line 976, in launch_instance
app.start()
File "/usr/local/lib/python3.9/dist-packages/ipykernel/kernelapp.py", line 712, in start
self.io_loop.start()
File "/usr/local/lib/python3.9/dist-packages/tornado/platform/asyncio.py", line 215, in start
self.asyncio_loop.run_forever()
File "/usr/lib/python3.9/asyncio/base_events.py", line 601, in run_forever
self._run_once()
File "/usr/lib/python3.9/asyncio/base_events.py", line 1905, in _run_once
handle._run()
File "/usr/lib/python3.9/asyncio/events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "/usr/local/lib/python3.9/dist-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
await self.process_one()
File "/usr/local/lib/python3.9/dist-packages/ipykernel/kernelbase.py", line 499, in process_one
await dispatch(*args)
File "/usr/local/lib/python3.9/dist-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
await result
File "/usr/local/lib/python3.9/dist-packages/ipykernel/kernelbase.py", line 730, in execute_request
reply_content = await reply_content
File "/usr/local/lib/python3.9/dist-packages/ipykernel/ipkernel.py", line 383, in do_execute
res = shell.run_cell(
File "/usr/local/lib/python3.9/dist-packages/ipykernel/zmqshell.py", line 528, in run_cell
return super().run_cell(*args, **kwargs)
File "/usr/local/lib/python3.9/dist-packages/IPython/core/interactiveshell.py", line 2881, in run_cell
result = self._run_cell(
File "/usr/local/lib/python3.9/dist-packages/IPython/core/interactiveshell.py", line 2936, in _run_cell
return runner(coro)
File "/usr/local/lib/python3.9/dist-packages/IPython/core/async_helpers.py", line 129, in _pseudo_sync_runner
coro.send(None)
File "/usr/local/lib/python3.9/dist-packages/IPython/core/interactiveshell.py", line 3135, in run_cell_async
has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
File "/usr/local/lib/python3.9/dist-packages/IPython/core/interactiveshell.py", line 3338, in run_ast_nodes
if await self.run_code(code, result, async_=asy):
File "/usr/local/lib/python3.9/dist-packages/IPython/core/interactiveshell.py", line 3398, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "/tmp/ipykernel_61/2241442519.py", line 3, in <cell line: 3>
model.fit(x=trainData,
File "/usr/local/lib/python3.9/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
try:
File "/usr/local/lib/python3.9/dist-packages/keras/engine/training.py", line 1409, in fit
`tf.distribute.experimental.ParameterServerStrategy`.
File "/usr/local/lib/python3.9/dist-packages/keras/engine/training.py", line 1051, in train_function
self.loss_tracker.reset_states()
File "/usr/local/lib/python3.9/dist-packages/keras/engine/training.py", line 1040, in step_function
def __init__(self, *args, **kwargs):
File "/usr/local/lib/python3.9/dist-packages/keras/engine/training.py", line 1030, in run_step
def compute_loss(self, x=None, y=None, y_pred=None, sample_weight=None):
File "/usr/local/lib/python3.9/dist-packages/keras/engine/training.py", line 889, in train_step
File "/usr/local/lib/python3.9/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
try:
File "/usr/local/lib/python3.9/dist-packages/keras/engine/training.py", line 490, in __call__
# default.
File "/usr/local/lib/python3.9/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
try:
File "/usr/local/lib/python3.9/dist-packages/keras/engine/base_layer.py", line 1014, in __call__
RuntimeError: if `super().__init__()` was not called in the
File "/usr/local/lib/python3.9/dist-packages/keras/utils/traceback_utils.py", line 92, in error_handler
def error_handler(*args, **kwargs):
File "/usr/local/lib/python3.9/dist-packages/keras/engine/sequential.py", line 374, in call
def build(self, input_shape=None):
File "/usr/local/lib/python3.9/dist-packages/keras/engine/functional.py", line 458, in call
def _trackable_children(self, save_type="checkpoint", **kwargs):
File "/usr/local/lib/python3.9/dist-packages/keras/engine/functional.py", line 596, in _run_internal_graph
# Read final output shapes from layers_to_output_shapes.
File "/usr/local/lib/python3.9/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
try:
File "/usr/local/lib/python3.9/dist-packages/keras/engine/base_layer.py", line 1014, in __call__
RuntimeError: if `super().__init__()` was not called in the
File "/usr/local/lib/python3.9/dist-packages/keras/utils/traceback_utils.py", line 92, in error_handler
def error_handler(*args, **kwargs):
File "/usr/local/lib/python3.9/dist-packages/keras/layers/reshaping/flatten.py", line 98, in call
)
Node: 'myModel/flatten_17/Reshape'
Input to reshape is a tensor with 3145728 values, but the requested shape requires a multiple of 32761
[[{{node myModel/flatten_17/Reshape}}]] [Op:__inference_train_function_731522]
I’m using Python 3.9, tried Tensorflow-gpu 2.9.1 and 2.11.0 .. both having same issue.
Have been trying all sorts of suggestions I could find online, but still no luck. Appreciate any suggestions
Thanks!
Well, Turned out the reason for this issue is, I was using the keyword image_size for setting the image size in the flow_from_dataframe functions instead of the correct keyword target_size … so the flow_from_dataframe was using the default value for target_size which is (256,256)
Fixing this and setting this target_size keyword to the correct image size of (181,181) fixed the issue for me