huggingface

How to efficiently convert a large parallel corpus to a Huggingface dataset to train an EncoderDecoderModel? Question: Typical EncoderDecoderModel that works on a Pre-coded Dataset The code snippet snippet as below is frequently used to train an EncoderDecoderModel from Huggingface’s transformer library from transformers import EncoderDecoderModel from transformers import PreTrainedTokenizerFast multibert = EncoderDecoderModel.from_encoder_decoder_pretrained( "bert-base-multilingual-uncased", "bert-base-multilingual-uncased" …

How to fine-tune gpt-j using Huggingface Trainer Question: I’m attempting to fine-tune gpt-j using the huggingface trainer and failing miserably. I followed the example that references bert, but of course, the gpt-j model isn’t exactly like the bert model. The error indicates that the model isn’t producing a loss, which is great, except that I …

Why there are no logs and which model is saved? Question: I’m using Trainer to train my model. I have the following outputs on screen: Epoch Training Loss Validation Loss Accuracy 0 No log 1.114260 0.342667 1 No log 0.939480 0.545333 2 No log 0.816581 0.660000 3 No log 0.752204 0.710667 4 No log 0.741462 …

How to choose grid search (when working with trainer.hyperparameter_search)? Question: I want to run trainer.hyperparameter_search (with grid search) and I haven’t seen any HP algorithm type parameter. How can I configure trainer.hyperparameter_search to run with grid-search ? Asked By: user3668129 || Source Answers: You can use Optuna for this: def hp_search(trial): return { "learning_rate": trial.suggest_float("learning_rate", …

issue when importing BloomTokenizer from transformers in python Question: I am trying to import BloomTokenizer from transformers from transformers import BloomTokenizer and I receive the following error Traceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: cannot import name ‘BloomTokenizer’ from ‘transformers’ (/root/miniforge3/envs/pytorch/lib/python3.8/site-packages/transformers/__init__.py) my version of transformers: transformers 4.20.1 what could I …