Can't load saved gensim word2vec model
Question:
I tried saving a word2vec model that I had trained with gensim like so:
from gensim.models import Word2Vec
model = Word2Vec(sentences, parameters)
model.save('modelfile.model')
Now when I try Word2Vec.load('modelfile.model'), I get:
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'
Can post the full traceback if it helps.
Answers:
That’s odd. Are you using the exact same Python environment & gensim version for the load() as the save()? How did you install gensim & numpy?
When I search for that error, I find other discussions that suggest it may be a symptom of having pickled (saved) a numpy array from numpy 1.16. but trying to unpickle (load) it in an earlier numpy. See for example:
https://github.com/numpy/numpy/issues/12825#issuecomment-456561919
It looks like numpy has recently merged a fix – https://github.com/numpy/numpy/issues/12837 – but in the meantime your best bet might be making sure the place where you’re loading is using numpy 1.16.0+.
I would try virtualenv just to avoid any conflicts among packages and environments. Virtualenv is a tool to create isolated Python environments. Here are the quick steps to setup virtualenv and give it a try to load your word2vec model.
sudo pip install virtualenv # install virtualenv
virtualenv test1 # create an env named test1
cd test1 # go ot test1
source bin/activate # activate test1
pip install gensim # install your packages for test1 env
python path-to-your-file.py # run your program
Don’t use the .model while saving the model, just put "model", as it will be saved in a .bin format
I tried saving a word2vec model that I had trained with gensim like so:
from gensim.models import Word2Vec
model = Word2Vec(sentences, parameters)
model.save('modelfile.model')
Now when I try Word2Vec.load('modelfile.model'), I get:
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'
Can post the full traceback if it helps.
That’s odd. Are you using the exact same Python environment & gensim version for the load() as the save()? How did you install gensim & numpy?
When I search for that error, I find other discussions that suggest it may be a symptom of having pickled (saved) a numpy array from numpy 1.16. but trying to unpickle (load) it in an earlier numpy. See for example:
https://github.com/numpy/numpy/issues/12825#issuecomment-456561919
It looks like numpy has recently merged a fix – https://github.com/numpy/numpy/issues/12837 – but in the meantime your best bet might be making sure the place where you’re loading is using numpy 1.16.0+.
I would try virtualenv just to avoid any conflicts among packages and environments. Virtualenv is a tool to create isolated Python environments. Here are the quick steps to setup virtualenv and give it a try to load your word2vec model.
sudo pip install virtualenv # install virtualenv
virtualenv test1 # create an env named test1
cd test1 # go ot test1
source bin/activate # activate test1
pip install gensim # install your packages for test1 env
python path-to-your-file.py # run your program
Don’t use the .model while saving the model, just put "model", as it will be saved in a .bin format