Decoding hidden layer embeddings in T5 Question: I’m new to NLP (pardon the very noob question!), and am looking for a way to perform vector operations on sentence embeddings (e.g., randomization in embedding-space in a uniform ball around a given sentence) and then decode them. I’m currently attempting to use the following strategy with T5 …
How to merge predicted values to original pandas test data frame where X_test has been converted using CountVectorizer before splitting Question: I want to merge my predicted results of my test data to my X_test. I was able to merge it with y_test but since my X_test is a corpus I’m not sure how I …
Getting a weird behaviour when using Matcher from Spacy several times Question: I would like to use Matcher from Spacy on a list of span (sents) class Chunker: def __init__(self, nlp, matcher): self.nlp = nlp self.matcher = matcher self.matcher.add("NP", NP_pattern, on_match=self.on_match_callback, greedy="LONGEST") self.matcher.add("VP", VP_pattern, on_match=self.on_match_callback, greedy="LONGEST") self.matcher.add("VVP", VVP_pattern, on_match=self.on_match_callback, greedy="LONGEST") def on_match_callback(self, matcher, doc, i, …
How to extract data from field in json line format and store it in a new file in python as a text Question: I have json file that looks like this: {"reviewerID": "A11N155CW1UV02", "asin": "B000H00VBQ", "reviewerName": "AdrianaM", "helpful": [0, 0], "reviewText": "I had big expectations because I love English TV, in particular Investigative and detective …
Terminal error from download_bios.py: the following arguments are required: wetpaths Question: I’ve been attempting to download a dataset downloaded off of PapersWithCode and when I run the download program I get the following error message: usage: download_bios.py [-h] [-o OUT] [-r RETRIES] [-p N] wetpaths download_bios.py: error: the following arguments are required: wetpaths and am …
SparseTermSimilarityMatrix().inner_product() throws "cannot unpack non-iterable bool object" Question: While working with cosine similarity, I am facing issue calculating the inner product of two vectors. Code: from gensim.similarities import ( WordEmbeddingSimilarityIndex, SparseTermSimilarityMatrix ) w2v_model = api.load("glove-wiki-gigaword-50") similarity_index = WordEmbeddingSimilarityIndex(w2v_model) similarity_matrix = SparseTermSimilarityMatrix(similarity_index, dictionary) score = similarity_matrix.inner_product( X = [ (0, 1), (1, 1), (2, 1), (3, …
How can I check similarity in meaning and not just having same words between two texts with spacy Question: I’m trying to compare two different texts—one coming from a Curriculum Vitae (CV) and the other from a job announcement. After cleaning the texts, I’m trying to compare them to detect if a job announcement is …
split dataframe text column with given strings – NLP Question: I have a dataframe with a text column in this form: column_description "this section includes: animals: cats and dogs and vegetables but doesn’t include: plants and fruits: coco" "this section includes the following: axis: x and y but doesn’t include: z, k and c" "this …
Spacy incorrectly identifying pronouns Question: When I try this code using Spacy, I get the desired result: import spacy nlp = spacy.load("en_core_web_sm") # example 1 test = "All my stuff is at to MyBOQ" doc = nlp(test) for word in doc: if word.pos_ == ‘PRON’: print(word.text) The output shows All and my. However, if I …
Match most similar string pairs from two lists of strings? Question: I have two lists of strings (of equal sizes): l1 = [ "Which of the following products have you used", "Provide a rating to the product", "Will you purchase the product again" ] l2 = [ "Please give a rating to the product" "Will …
