nlp

How to add a dense layer on top of SentenceTransformer?

How to add a dense layer on top of SentenceTransformer? Question: In this tutorial (Train and Fine-Tune Sentence Transformers Models) they go through creating a SentenceTransformer by combining a word embedding module with a pooling layer: from sentence_transformers import SentenceTransformer, models ## Step 1: use an existing language model word_embedding_model = models.Transformer(‘distilroberta-base’) ## Step 2: …

Total answers: 1

How to use adapter transformers with a Huggingface Pipeline

How to use adapter transformers with a Huggingface Pipeline Question: I tried to run the model "AdapterHub/bert-base-uncased-pf-conll2003" (Model description here) for token classification in NLP. First I tried to install the adapter transformers pip install -U adapter-transformers The output of the above command was Collecting adapter-transformers [… see edit history for skipped lines …] Installing …

Total answers: 1

Import error in training arguments in Colaboratory

Import error in training arguments in Colaboratory Question: I am using Google Colaboratory for my NLP project. I installed transformers and other libraries, but I got an error. from transformers import Trainer, TrainingArguments batch_size = 64 logging_steps = len(stationary_dataset_encoded["train"]) // batch_size model_name = f"{model_ckpt}-finetuned-stationary-update" training_args = TrainingArguments(output_dir=model_name, num_train_epochs=10, learning_rate=2e-5, per_device_train_batch_size=batch_size, per_device_eval_batch_size=batch_size, weight_decay=0.01, evaluation_strategy="epoch", disable_tqdm=False, logging_steps=logging_steps, …

Total answers: 1

Question about data_collator throwing a key error in Hugging face

Question about data_collator throwing a key error in Hugging face Question: I am trying to use data_collator function in hugging face using this code: datasets = dataset.train_test_split(test_size=0.1) train_dataset = datasets["train"] val_dataset = datasets["test"] print(type(train_dataset)) def data_collator(data): # Initialize lists to store pixel values and input ids pixel_values_list = [] input_ids_list = [] # Iterate over …

Total answers: 1

ImportError: cannot import name 'deprecated' from 'typing_extensions'

ImportError: cannot import name 'deprecated' from 'typing_extensions' Question: I want to download spacy, but the version of typing-extensions is lowered in the terminal: ERROR: pydantic 2.3.0 has requirement typing-extensions>=4.6.1, but you’ll have typing-extensions 4.4.0 which is incompatible. ERROR: pydantic-core 2.6.3 has requirement typing-extensions!=4.7.0,>=4.6.0, but you’ll have typing-extensions 4.4.0 which is incompatible. Installing collected packages: typing-extensions …

Total answers: 1

shape of my dataframe(#rows) and that of final embeddings array doesn't match

shape of my dataframe(#rows) and that of final embeddings array doesn't match Question: I generated the word embeddings for my corpus(2-D List) then tried to generate the Average Word2Vec embeddings for each of the individual word list(that is for each comment which have been converted into a list though split() method) inside my corpus but …

Total answers: 1

How to interpret the model_max_len attribute of the PreTrainedTokenizer object in Huggingface Transformers

How to interpret the model_max_len attribute of the PreTrainedTokenizer object in Huggingface Transformers Question: I’ve been trying to check the maximum length allowed by emilyalsentzer/Bio_ClinicalBERT, and after these lines of code: model_name = "emilyalsentzer/Bio_ClinicalBERT" tokenizer = AutoTokenizer.from_pretrained(model_name) tokenizer I’ve obtained the following: PreTrainedTokenizerFast(name_or_path=’emilyalsentzer/Bio_ClinicalBERT’, vocab_size=28996, model_max_len=1000000000000000019884624838656, is_fast=True, padding_side=’right’, truncation_side=’right’, special_tokens={‘unk_token’: ‘[UNK]’, ‘sep_token’: ‘[SEP]’, ‘pad_token’: ‘[PAD]’, ‘cls_token’: …

Total answers: 1

How to compute sentence level perplexity from hugging face language models?

How to compute sentence level perplexity from hugging face language models? Question: I have a large collection of documents each consisting of ~ 10 sentences. For each document, I wish to find the sentence that maximises perplexity, or equivalently the loss from a fine-tuned causal LM. I have decided to use Hugging Face and the …

Total answers: 1

Problem tokenizing with HuggingFace's library when fine tuning bloom

Problem tokenizing with HuggingFace's library when fine tuning bloom Question: I have a problem with my tokenizer function. To be honest I am quiet lost, since I do not really understand whats happening inside the transformer library. Here is what I wanted to do: I would like to fine tune the BLOOM model to a …

Total answers: 2