Can't broadcast input array from one shape to another shape when displaying CNN Filter layers
Question:
I am trying to display my CNN filter images however I get a input shape error and I am unsure how to fix it.
My code is the following:
# Let's define a new Model that will take an image as input, and will output
# intermediate representations for all layers in the previous model after
# the first.
successive_outputs = [layer.output for layer in model.layers[1:]]
visualization_model = tf.keras.models.Model(inputs = model.input, outputs = successive_outputs)
img = load_img(img_path, target_size=(360, 640)) # this is a PIL image
x = img_to_array(img) # Numpy array with shape (150, 150, 3)
x = x.reshape((1,) + x.shape) # Numpy array with shape (1, 150, 150, 3)
# Rescale by 1/255
x /= 255.0
# Let's run our image through our network, thus obtaining all
# intermediate representations for this image.
successive_feature_maps = visualization_model.predict(x)
# These are the names of the layers, so can have them as part of our plot
layer_names = [layer.name for layer in model.layers]
# -----------------------------------------------------------------------
# Now let's display our representations
# -----------------------------------------------------------------------
for layer_name, feature_map in zip(layer_names, successive_feature_maps):
print(feature_map.shape)
if len(feature_map.shape) == 4:
#-------------------------------------------
# Just do this for the conv / maxpool layers, not the fully-connected layers
#-------------------------------------------
n_features = feature_map.shape[-1] # number of features in the feature map
size = feature_map.shape[ 1] # feature map shape (1, size, size, n_features)
# We will tile our images in this matrix
display_grid = np.zeros((size, size * n_features))
print(display_grid.shape)
#-------------------------------------------------
# Postprocess the feature to be visually palatable
#-------------------------------------------------
for i in range(n_features):
x = feature_map[0, :, :, i]
x -= x.mean()
x /= x.std ()
x *= 64
x += 128
x = np.clip(x, 0, 255).astype('uint8')
display_grid[:, i * size : (i + 1) * size] = x # Tile each filter into a horizontal grid
#-----------------
# Display the grid
#-----------------
scale = 20. / n_features
plt.figure( figsize=(scale * n_features, scale) )
plt.title ( layer_name )
plt.grid ( False )
plt.imshow( display_grid, aspect='auto', cmap='viridis' )
The layers within my model and it’s shapes:
Model: "model"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 360, 640, 3)] 0
block1_conv1 (Conv2D) (None, 360, 640, 64) 1792
block1_conv2 (Conv2D) (None, 360, 640, 64) 36928
block1_pool (MaxPooling2D) (None, 180, 320, 64) 0
block2_conv1 (Conv2D) (None, 180, 320, 128) 73856
block2_conv2 (Conv2D) (None, 180, 320, 128) 147584
block2_pool (MaxPooling2D) (None, 90, 160, 128) 0
block3_conv1 (Conv2D) (None, 90, 160, 256) 295168
block3_conv2 (Conv2D) (None, 90, 160, 256) 590080
block3_conv3 (Conv2D) (None, 90, 160, 256) 590080
block3_pool (MaxPooling2D) (None, 45, 80, 256) 0
block4_conv1 (Conv2D) (None, 45, 80, 512) 1180160
block4_conv2 (Conv2D) (None, 45, 80, 512) 2359808
block4_conv3 (Conv2D) (None, 45, 80, 512) 2359808
block4_pool (MaxPooling2D) (None, 22, 40, 512) 0
block5_conv1 (Conv2D) (None, 22, 40, 512) 2359808
block5_conv2 (Conv2D) (None, 22, 40, 512) 2359808
block5_conv3 (Conv2D) (None, 22, 40, 512) 2359808
block5_pool (MaxPooling2D) (None, 11, 20, 512) 0
flatten (Flatten) (None, 112640) 0
dense (Dense) (None, 4096) 461377536
batch_normalization (BatchN (None, 4096) 16384
ormalization)
dropout (Dropout) (None, 4096) 0
dense_1 (Dense) (None, 4096) 16781312
batch_normalization_1 (Batc (None, 4096) 16384
hNormalization)
dropout_1 (Dropout) (None, 4096) 0
dense_2 (Dense) (None, 3) 12291
=================================================================
Total params: 492,918,595
Trainable params: 480,547,331
Non-trainable params: 12,371,264
_________________________________________________________________
After running the code, this is the error that I am getting:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[44], line 53
51 x += 128
52 x = np.clip(x, 0, 255).astype('uint8')
---> 53 display_grid[:, i * size : (i + 1) * size] = x # Tile each filter into a horizontal grid
56 #-----------------
57 # Display the grid
58 #-----------------
60 scale = 20. / n_features
ValueError: could not broadcast input array from shape (360,640) into shape (360,360)
Wouldn’t the shape be altered to fit into the appropriate shape? Is there some sort of transformation that I am missing? I have also tried using the preprocess_input function however the same error occurs
Answers:
Your feature maps are not square. So # feature map shape (1, size, size, n_features) is not true. i suggest you to try this changes.
size1 , size2= feature_map.shape[1],feature_map.shape[2]
display_grid = np.zeros((size1, size2 * n_features))
and
display_grid[:, i * size2 : (i + 1) * size2] = x
I am trying to display my CNN filter images however I get a input shape error and I am unsure how to fix it.
My code is the following:
# Let's define a new Model that will take an image as input, and will output
# intermediate representations for all layers in the previous model after
# the first.
successive_outputs = [layer.output for layer in model.layers[1:]]
visualization_model = tf.keras.models.Model(inputs = model.input, outputs = successive_outputs)
img = load_img(img_path, target_size=(360, 640)) # this is a PIL image
x = img_to_array(img) # Numpy array with shape (150, 150, 3)
x = x.reshape((1,) + x.shape) # Numpy array with shape (1, 150, 150, 3)
# Rescale by 1/255
x /= 255.0
# Let's run our image through our network, thus obtaining all
# intermediate representations for this image.
successive_feature_maps = visualization_model.predict(x)
# These are the names of the layers, so can have them as part of our plot
layer_names = [layer.name for layer in model.layers]
# -----------------------------------------------------------------------
# Now let's display our representations
# -----------------------------------------------------------------------
for layer_name, feature_map in zip(layer_names, successive_feature_maps):
print(feature_map.shape)
if len(feature_map.shape) == 4:
#-------------------------------------------
# Just do this for the conv / maxpool layers, not the fully-connected layers
#-------------------------------------------
n_features = feature_map.shape[-1] # number of features in the feature map
size = feature_map.shape[ 1] # feature map shape (1, size, size, n_features)
# We will tile our images in this matrix
display_grid = np.zeros((size, size * n_features))
print(display_grid.shape)
#-------------------------------------------------
# Postprocess the feature to be visually palatable
#-------------------------------------------------
for i in range(n_features):
x = feature_map[0, :, :, i]
x -= x.mean()
x /= x.std ()
x *= 64
x += 128
x = np.clip(x, 0, 255).astype('uint8')
display_grid[:, i * size : (i + 1) * size] = x # Tile each filter into a horizontal grid
#-----------------
# Display the grid
#-----------------
scale = 20. / n_features
plt.figure( figsize=(scale * n_features, scale) )
plt.title ( layer_name )
plt.grid ( False )
plt.imshow( display_grid, aspect='auto', cmap='viridis' )
The layers within my model and it’s shapes:
Model: "model"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 360, 640, 3)] 0
block1_conv1 (Conv2D) (None, 360, 640, 64) 1792
block1_conv2 (Conv2D) (None, 360, 640, 64) 36928
block1_pool (MaxPooling2D) (None, 180, 320, 64) 0
block2_conv1 (Conv2D) (None, 180, 320, 128) 73856
block2_conv2 (Conv2D) (None, 180, 320, 128) 147584
block2_pool (MaxPooling2D) (None, 90, 160, 128) 0
block3_conv1 (Conv2D) (None, 90, 160, 256) 295168
block3_conv2 (Conv2D) (None, 90, 160, 256) 590080
block3_conv3 (Conv2D) (None, 90, 160, 256) 590080
block3_pool (MaxPooling2D) (None, 45, 80, 256) 0
block4_conv1 (Conv2D) (None, 45, 80, 512) 1180160
block4_conv2 (Conv2D) (None, 45, 80, 512) 2359808
block4_conv3 (Conv2D) (None, 45, 80, 512) 2359808
block4_pool (MaxPooling2D) (None, 22, 40, 512) 0
block5_conv1 (Conv2D) (None, 22, 40, 512) 2359808
block5_conv2 (Conv2D) (None, 22, 40, 512) 2359808
block5_conv3 (Conv2D) (None, 22, 40, 512) 2359808
block5_pool (MaxPooling2D) (None, 11, 20, 512) 0
flatten (Flatten) (None, 112640) 0
dense (Dense) (None, 4096) 461377536
batch_normalization (BatchN (None, 4096) 16384
ormalization)
dropout (Dropout) (None, 4096) 0
dense_1 (Dense) (None, 4096) 16781312
batch_normalization_1 (Batc (None, 4096) 16384
hNormalization)
dropout_1 (Dropout) (None, 4096) 0
dense_2 (Dense) (None, 3) 12291
=================================================================
Total params: 492,918,595
Trainable params: 480,547,331
Non-trainable params: 12,371,264
_________________________________________________________________
After running the code, this is the error that I am getting:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[44], line 53
51 x += 128
52 x = np.clip(x, 0, 255).astype('uint8')
---> 53 display_grid[:, i * size : (i + 1) * size] = x # Tile each filter into a horizontal grid
56 #-----------------
57 # Display the grid
58 #-----------------
60 scale = 20. / n_features
ValueError: could not broadcast input array from shape (360,640) into shape (360,360)
Wouldn’t the shape be altered to fit into the appropriate shape? Is there some sort of transformation that I am missing? I have also tried using the preprocess_input function however the same error occurs
Your feature maps are not square. So # feature map shape (1, size, size, n_features) is not true. i suggest you to try this changes.
size1 , size2= feature_map.shape[1],feature_map.shape[2]
display_grid = np.zeros((size1, size2 * n_features))
and
display_grid[:, i * size2 : (i + 1) * size2] = x