How can I put new input in the middle of the layers in Keras?
Question:
This model is what I want to implement. When the teacher model gets a sentence, it will output values that give more weight to important words for classification (e.g When the sentence is "I want to slaughter those people", then the teacher model gives more weight to the word ‘slaughter’ for hate speech classification). Then, student model gets the same input sentence and additionally, the weight values that teacher gave. However, it seems that if I use Sequential module in Keras, then new input value can’t be added in the middle of layers. (It would be simple in Pytorch, but I’m new to Keras..)
Can anyone give me some advice?
I tried to search information in Keras documents, but it feels like ‘Model’ module is not for this situation
Answers:
The Sequential class is designed to only have one input and one output. In order to have multiple inputs, for example the sentence and the weights from the teacher model, you need to use the Model class. You will be able to design the setup you aim to.
see https://www.tensorflow.org/guide/kera/functional#models_with_multiple_inputs_and_outputs
I might be able to provide additional help, if you add some more details.
This model is what I want to implement. When the teacher model gets a sentence, it will output values that give more weight to important words for classification (e.g When the sentence is "I want to slaughter those people", then the teacher model gives more weight to the word ‘slaughter’ for hate speech classification). Then, student model gets the same input sentence and additionally, the weight values that teacher gave. However, it seems that if I use Sequential module in Keras, then new input value can’t be added in the middle of layers. (It would be simple in Pytorch, but I’m new to Keras..)
Can anyone give me some advice?
I tried to search information in Keras documents, but it feels like ‘Model’ module is not for this situation
The Sequential class is designed to only have one input and one output. In order to have multiple inputs, for example the sentence and the weights from the teacher model, you need to use the Model class. You will be able to design the setup you aim to.
see https://www.tensorflow.org/guide/kera/functional#models_with_multiple_inputs_and_outputs
I might be able to provide additional help, if you add some more details.