Getting value error for Concatenate layer

Well, I’m trying to replicate a model from this paper on Pubmed 200k Randomized controlled clinical trials dataset which you can find in this paper.

the dataset with 5 unique tokens for the output is something like this:

well, I’ve made some progress I tried my best to replicate the paper but in the end, I’m getting a value error:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-55-d583261b0496> in <module>()
     23 
     24 # 4. create an output layer
---> 25 dropout_layer = tf.keras.layers.Dropout(0.5,name='dropout_layer')(concat_model)
     26 dense_layer = tf.keras.layers.Dense(200,name='dense_layer')(dropout_layer)
     27 dropout_layer = tf.keras.layers.Dropout(0.5,name='dropout_layer_2')(dense_layer)

1 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/constant_op.py in convert_to_eager_tensor(value, ctx, dtype)
    100       dtype = dtypes.as_dtype(dtype).as_datatype_enum
    101   ctx.ensure_initialized()
--> 102   return ops.EagerTensor(value, ctx.device_name, dtype)
    103 
    104 

ValueError: Exception encountered when calling layer "dropout_layer" (type Dropout).

Attempt to convert a value (<keras.layers.merge.Concatenate object at 0x7f51a0b5f850>) with an unsupported type (<class 'keras.layers.merge.Concatenate'>) to a Tensor.

Call arguments received:
  • inputs=<keras.layers.merge.Concatenate object at 0x7f51a0b5f850>
  • training=False

for the code, I tried my best to comment on as many things as possible.

# 1. word model:
inputs = tf.keras.layers.Input(
    shape=(1,), dtype=tf.string, name="text_input"
)  # takes a list of input
text_vectorization_layer = text_vectorizer(inputs)  # word level vectorizer
text_embedding_layer = embedding(text_vectorization_layer)  # word level embedding
# flatten_layer = tf.keras.layers.Flatten()(text_embedding_layer)
outputs = tf.keras.layers.Dense(300, activation="relu", name="text_output")(
    text_embedding_layer
)  # output of 300 from fig 1. https://arxiv.org/pdf/1612.05251.pdf
token_model = tf.keras.Model(inputs, outputs)  # token model

# 2. char model:
inputs = tf.keras.layers.Input(shape=(1,), dtype=tf.string, name="char_input")
char_vectorization_layer = char_vectorizer(inputs)  # character level vectorization
char_embedding_layer = char_embedding(char_vectorization_layer)  # char level embedding
outputs = tf.keras.layers.Bidirectional(
    tf.keras.layers.LSTM(25), name="bi_directional_lstm"
)(
    char_embedding_layer
)  # bi directional lstm as output
char_model = tf.keras.Model(inputs, outputs)

# 3. concatinating both the model
concat_model = tf.keras.layers.Concatenate([token_model, char_model])

# 4. create output layer
dropout_layer = tf.keras.layers.Dropout(0.5, name="dropout_layer")(concat_model)
dense_layer = tf.keras.layers.Dense(200, name="dense_layer")(dropout_layer)
dropout_layer = tf.keras.layers.Dropout(0.5, name="dropout_layer_2")(dense_layer)
output_layer = tf.keras.layer.Dense(5, activation="softmax", name="final_output_layer")(
    dropout_layer
)

I’m not getting what will be the output from the concatenate layer that it’s causing an error for the dropout layer

fixed code thanks to @Djinn :

# 1. word model:
inputs = tf.keras.layers.Input(
    shape=(1,), dtype=tf.string, name="text_input"
)  # takes a list of input
text_vectorization_layer = text_vectorizer(inputs)  # word level vectorizer
text_embedding_layer = embedding(text_vectorization_layer)  # word level embedding
flatten_layer = tf.keras.layers.Flatten()(text_embedding_layer)
outputs = tf.keras.layers.Dense(300, activation="relu", name="text_output")(
    flatten_layer
)  # output of 300 from fig 1. https://arxiv.org/pdf/1612.05251.pdf
token_model = tf.keras.Model(inputs, outputs)  # token model

# 2. char model:
inputs = tf.keras.layers.Input(shape=(1,), dtype=tf.string, name="char_input")
char_vectorization_layer = char_vectorizer(inputs)  # character level vectorization
char_embedding_layer = char_embedding(char_vectorization_layer)  # char level embedding
outputs = tf.keras.layers.Bidirectional(
    tf.keras.layers.LSTM(25), name="bi_directional_lstm"
)(
    char_embedding_layer
)  # bi directional lstm as output
char_model = tf.keras.Model(inputs, outputs)

# 3. concatinating both the model
concat_model = tf.keras.layers.Concatenate()([token_model.output, char_model.output])

# 4. create output layer
dropout_layer = tf.keras.layers.Dropout(0.5, name="dropout_layer")(concat_model)
dense_layer = tf.keras.layers.Dense(200, name="dense_layer")(dropout_layer)
dropout_layer = tf.keras.layers.Dropout(0.5, name="dropout_layer_2")(dense_layer)
output_layer = tf.keras.layers.Dense(
    5, activation="softmax", name="final_output_layer"
)(dropout_layer)

# 5.final model
model_2 = tf.keras.Model(
    inputs=[token_model.input, char_model.input],
    outputs=output_layer,
    name="token_char_model",
)