Is it more correct to export bigrams from the bigram model or the trigram model in Gensim? Question: After I train a bigram model and a trigram model using Gensim, I can export the bigrams from the bigram model. Alternatively, I can export the bigrams from the trigram model. I find that the bigrams from …
'pseudocorpus' no longer available from 'gensim.models.phrases'? Question: Several months ago, I used "pseudocorpus" to create a fake corpus as part of phrase training using Gensim with the following code: from gensim.models.phrases import pseudocorpus corpus = pseudocorpus(bigram_model.vocab, bigram_model.delimiter, bigram_model.common_terms) bigrams = [] for bigram, score in bigram_model.export_phrases(corpus, bigram_model.delimiter, as_tuples=False): if score >= bigram_model.threshold: bigrams.append(bigram.decode(‘utf-8’)) Now when …
SparseTermSimilarityMatrix().inner_product() throws "cannot unpack non-iterable bool object" Question: While working with cosine similarity, I am facing issue calculating the inner product of two vectors. Code: from gensim.similarities import ( WordEmbeddingSimilarityIndex, SparseTermSimilarityMatrix ) w2v_model = api.load("glove-wiki-gigaword-50") similarity_index = WordEmbeddingSimilarityIndex(w2v_model) similarity_matrix = SparseTermSimilarityMatrix(similarity_index, dictionary) score = similarity_matrix.inner_product( X = [ (0, 1), (1, 1), (2, 1), (3, …
Gensim Word2Vec exhausting iterable Question: I’m getting the following prompt when calling model.train() from gensim word2vec INFO : EPOCH 0: training on 0 raw words (0 effective words) took 0.0s, 0 effective words/s The only solutions I found on my search for an answer point to the itarable vs iterator difference, and at this point, …
Converting word to vector using GloVe Question: I loaded my glove package as follows: import gensim.downloader as api model = api.load("glove-wiki-gigaword-100") and would want to create a function where I pass in a word and the GloVe model, and it will return the corresponding vector, for instance, def convert_word_to_vec(word, model): and when I pass in …
KeyedVectors' object has no attribute 'wv for gensim 4.1.2 Question: i have migrated from gensim 3.8.3 to 4.1.2 and i am using this claim = [token for token in claim_text if token in w2v_model.wv.vocab] reference = [token for token in ref_text if token in w2v_model.wv.vocab] i am not sure how to replace w2v_model.wv.vocab to newer …
LDA Mallet Gensim CalledProcessError Question: Seems like many people are having issues with Mallet. import os from gensim.models.wrappers import LdaMallet os.environ.update({‘MALLET_HOME’:r’C:/Users/myusername/Desktop/Topic_Modelling/mallet-2.0.8′}) mallet_path = r’C:/Users/myusername/Desktop/Topic_Modelling/mallet-2.0.8/bin/mallet’ model = gensim.models.wrappers.LdaMallet(mallet_path, corpus=corpus,num_topics=num_topics, id2word=id2word) Getting the following errors: /bin/sh: C:/Users/myusername/Desktop/Topic_Modelling/mallet-2.0.8/bin/mallet.bat: No such file or directory CalledProcessError: Command ‘C:/Users/myusername/Desktop/Topic_Modelling/mallet-2.0.8/bin/mallet.bat import-file –preserve-case –keep-sequence –remove-stopwords –token-regex "S+" –input /var/folders/ml/lxzrtxwn02vbvq65c80g1b640000gn/T/c52cdc_corpus.txt –output /var/folders/ml/lxzrtxwn02vbvq65c80g1b640000gn/T/c52cdc_corpus.mallet’ returned non-zero …
What would the output of skip-gram model look like? Question: To my understanding, the output of the skip-gram model must be compared with many training labels (depending on the window size) My question is: Does the final output of the skip-gram model look like the description in this picture? Ps. the most similar question I …
How do I subtract and add vectors with gensim KeyedVectors? Question: I need to add and subtract word vectors, for a project in which I use gensim.models.KeyedVectors (from the word2vec-google-news-300 model) Unfortunately, I’ve tried but can’t manage to do it correctly. Let’s look at the poular example queen ~= king – man + woman. When …
Word embedding with gensim and FastText, training on pretrained vectors Question: I am trying to load the pretrained vec file of Facebook fasttext crawl-300d-2M.vec with the next code: from gensim.models.fasttext import load_facebook_model, load_facebook_vectors model_facebook = load_facebook_vectors(‘fasttext/crawl-300d-2M.vec’) But it fails with the next error: NotImplementedError: Supervised fastText models are not supported It is not possible to …
