How to add a dense layer on top of SentenceTransformer? Question: In this tutorial (Train and Fine-Tune Sentence Transformers Models) they go through creating a SentenceTransformer by combining a word embedding module with a pooling layer: from sentence_transformers import SentenceTransformer, models ## Step 1: use an existing language model word_embedding_model = models.Transformer(‘distilroberta-base’) ## Step 2: …
How to use adapter transformers with a Huggingface Pipeline Question: I tried to run the model "AdapterHub/bert-base-uncased-pf-conll2003" (Model description here) for token classification in NLP. First I tried to install the adapter transformers pip install -U adapter-transformers The output of the above command was Collecting adapter-transformers [… see edit history for skipped lines …] Installing …
Question about data_collator throwing a key error in Hugging face Question: I am trying to use data_collator function in hugging face using this code: datasets = dataset.train_test_split(test_size=0.1) train_dataset = datasets["train"] val_dataset = datasets["test"] print(type(train_dataset)) def data_collator(data): # Initialize lists to store pixel values and input ids pixel_values_list = [] input_ids_list = [] # Iterate over …
ImportError: Using the Trainer with PyTorch requires accelerate = 0.20.1 Question: Please help me when I tried to use it in my Google Colab for transformers error: ImportError: Using the Trainer with PyTorch requires accelerate=0.20.1: Please run pip install transformers[torch] or pip install accelerate -U` NOTE: If your import is failing due to a missing …
Losing all training gains when switching to another PC Question: I am losing all of my training gains when moving to another PC and I can’t figure out why, it should be saving the model after each chunk and it does so because when I restart it on the same PC, the loss is the …
Where does AutoModelForSeq2SeqLM.from_pretrained() store the model to disk? Question: I attempted the following on a Windows device: from transformers import AutoModelForSeq2SeqLM, AutoTokenizer model = AutoModelForSeq2SeqLM.from_pretrained("google/flan-t5-small") tokenizer = AutoTokenizer.from_pretrained("google/flan-t5-small") It worked, but when I tried to download a larger version of the model: model = AutoModelForSeq2SeqLM.from_pretrained("google/flan-t5-XL") I ran out of disk space: OSError: [Errno 28] No …
How to interpret the model_max_len attribute of the PreTrainedTokenizer object in Huggingface Transformers Question: I’ve been trying to check the maximum length allowed by emilyalsentzer/Bio_ClinicalBERT, and after these lines of code: model_name = "emilyalsentzer/Bio_ClinicalBERT" tokenizer = AutoTokenizer.from_pretrained(model_name) tokenizer I’ve obtained the following: PreTrainedTokenizerFast(name_or_path=’emilyalsentzer/Bio_ClinicalBERT’, vocab_size=28996, model_max_len=1000000000000000019884624838656, is_fast=True, padding_side=’right’, truncation_side=’right’, special_tokens={‘unk_token’: ‘[UNK]’, ‘sep_token’: ‘[SEP]’, ‘pad_token’: ‘[PAD]’, ‘cls_token’: …
How to compute sentence level perplexity from hugging face language models? Question: I have a large collection of documents each consisting of ~ 10 sentences. For each document, I wish to find the sentence that maximises perplexity, or equivalently the loss from a fine-tuned causal LM. I have decided to use Hugging Face and the …
How to install diff version of a package (transformers) without internet in kaggle notebook w/o killing the kernel while keeping variables in memory? Question: I have prepared an inference pipeline for a Kaggle competition and it has to be executed without internet connection. I’m trying to use different versions of transformers but I had some …
Problem tokenizing with HuggingFace's library when fine tuning bloom Question: I have a problem with my tokenizer function. To be honest I am quiet lost, since I do not really understand whats happening inside the transformer library. Here is what I wanted to do: I would like to fine tune the BLOOM model to a …
