How to multiply two input of different dimensions (keras_tensor element) in CNN keras model?

Question

Suppose that after giving input to a 2d-CNN, as the output of ith layer I have an Output Shape: (None, 1, 3, 1). I also have another input with shape (None, 50, 27, 1) and what I want to multiply each element of columns 1-9 with the first element of (None, 1, 3, 1), each element of columns 10-18 with the second element of (None, 1, 3, 1) and each element of columns 19-27 with the third element of (None, 1, 3, 1). To summarize, what I want to do is as follows:

Doing this using lists is not difficult however, I want to do this in my keras model. Therefore, the type of my inputs are ‘keras.engine.keras_tensor.KerasTensor’. Do you have an idea about how to do that?

Asked By: Sumeyye A

||

Source

Answer 1

If your inputs are a and b,

c = tf.keras.layers.Lambda(
    lambda x: tf.concat(
        [a*b for a, b in zip(
            tf.split(x[1], 3, axis=-2), 
            tf.split(x[0], 3, axis=-2)
            )
        ], axis=-2
        )
    )([a,b])

Answered By: thushv89

Answer 2

Another solution which is similar to previous answer but with tf.repeat:

import tensorflow as tf

x = tf.random.normal((2, 50, 27, 1))
y = tf.random.normal((2, 1, 3, 1))

repeated = tf.concat([tf.repeat(y[:, :, i:i+1, :], 9, axis=2) for i in range(3)], axis=2) # shape = [2, 1, 27, 1]
res = x * repeated # shape = [2, 50, 27, 1]

And using Keras model:

x = tf.keras.layers.Input((50, 27, 1))
y = tf.keras.layers.Input((1, 3, 1))

res = tf.keras.layers.Lambda(
    lambda x: x[0] * tf.concat([tf.repeat(x[1][:, :, i:i+1, :], 9, axis=2) for i in range(3)], axis=2) 
)([x, y])

model = tf.keras.models.Model([x, y], res)

Answered By: Vlad

How to multiply two input of different dimensions (keras_tensor element) in CNN keras model?

Question:

Answers: