Calculating BLEU and Rouge score as fast as possible Question: I have around 200 candidate sentences and for each candidate, I want to measure the bleu score by comparing each sentence with thousands of reference sentences. These references are the same for all candidates. Here is how I’m doing it right now: ref_for_all = [reference] …
Tensorflow – Decoder for Machine Translation Question: I am going through Tensorflow’s tutorial on Neural Machine Translation using Attention mechanism. It has the following code for the Decoder : class Decoder(tf.keras.Model): def __init__(self, vocab_size, embedding_dim, dec_units, batch_sz): super(Decoder, self).__init__() self.batch_sz = batch_sz self.dec_units = dec_units self.embedding = tf.keras.layers.Embedding(vocab_size, embedding_dim) self.gru = tf.keras.layers.GRU(self.dec_units, return_sequences=True, return_state=True, recurrent_initializer=’glorot_uniform’) …
Huggingface MarianMT translators lose content, depending on the model Question: Context I am using MarianMT von Huggingface via Python in order to translate text from a source to a target language. Expected behaviour I enter a sequence into the MarianMT model and get this sequence translated back. For this, I use a corresponding language model …
