spacy

ImportError: cannot import name 'deprecated' from 'typing_extensions' Question: I want to download spacy, but the version of typing-extensions is lowered in the terminal: ERROR: pydantic 2.3.0 has requirement typing-extensions>=4.6.1, but you’ll have typing-extensions 4.4.0 which is incompatible. ERROR: pydantic-core 2.6.3 has requirement typing-extensions!=4.7.0,>=4.6.0, but you’ll have typing-extensions 4.4.0 which is incompatible. Installing collected packages: typing-extensions …

How to split a document into approximately 500 word chunks in python using spacy? Question: I’m trying to split up some long documents for use with an Open AI back end, so they need to be broken up into chunks of roughly 500 words/3000 characters or fewer. I’m using spacy to try and break these …

Reading files in a directory and saving each dynamically Question: text_open = open("inputfiles/(22).txt", "r") text = text_open.read() doc = nlp(text) db.add(doc) db.to_disk("./outputfiles/22.spacy") I am trying to loop over each of the 500+ documents in the inputfiles folder and output them through db.to_disk. Instead of changing the hard coded numbers every-time, how would I dynamically rename …

How to maximize performance of spaCy on an M1 Mac (currently much slower than Intel) Question: I’ve observed that nlp.pipe is 30-40% slower on my almost brand new M1 Pro Macbook than on my old Macbook Pro from 2017. Most other functions are faster on the M1 by a similar margin, so this is not …

Python NLP processing if statement not in stop words list Question: I’m working with NLP spacy library and I created a function to return a list of token from a text. import spacy def preprocess_text_spacy(text): stop_words = ["a", "the", "is", "are"] nlp = spacy.load(‘en_core_web_sm’) tokens = set() doc = nlp(text) for word in doc: if …

How to normalise keywords extracted with Named Entity Recognition Question: I’m trying to employ NER to extract keywords (tags) from job postings. This can be anything along with React, AWS, Team Building, Marketing. After training a custom model in SpaCy I’m presented with a problem – extracted tags are not unified/normalized across all of the …

Getting a weird behaviour when using Matcher from Spacy several times Question: I would like to use Matcher from Spacy on a list of span (sents) class Chunker: def __init__(self, nlp, matcher): self.nlp = nlp self.matcher = matcher self.matcher.add("NP", NP_pattern, on_match=self.on_match_callback, greedy="LONGEST") self.matcher.add("VP", VP_pattern, on_match=self.on_match_callback, greedy="LONGEST") self.matcher.add("VVP", VVP_pattern, on_match=self.on_match_callback, greedy="LONGEST") def on_match_callback(self, matcher, doc, i, …

How can I check similarity in meaning and not just having same words between two texts with spacy Question: I’m trying to compare two different texts—one coming from a Curriculum Vitae (CV) and the other from a job announcement. After cleaning the texts, I’m trying to compare them to detect if a job announcement is …

Spacy incorrectly identifying pronouns Question: When I try this code using Spacy, I get the desired result: import spacy nlp = spacy.load("en_core_web_sm") # example 1 test = "All my stuff is at to MyBOQ" doc = nlp(test) for word in doc: if word.pos_ == ‘PRON’: print(word.text) The output shows All and my. However, if I …

Why is the spaCy Scorer returning None for the entity scores but the model is extracting entities? Question: I am really confused why the Scorer.score is returning ents_p, ents_r, and ents_f as None for the below example. I am seeing something every similar with my own custom model and want to understand why it is …