Language detection for short string in a user content generated context Question: I have some question about the detection of short string. I need to detect the language of text sent in a chat, and I am faced with 2 problems: the lenght of the message the errors that may be in it and the …
Reduce fastText memory usage for big models Question: I trained a machine learning sentence classification model that uses, among other features, also the vectors obtained from a pretrained fastText model (like these) which is 7Gb. I use the pretrained fastText Italian model: I am using this word embedding only to get some semantic features to …
ERROR:root:can't pickle fasttext_pybind.fasttext objects Question: I am using gunicorn with multiple workers for my machine learning project. But the problem is when I send a train request only the worker getting the training request gets updated with the latest model after training is done. Here it is worth to mention that, to make the inference …
Word embedding with gensim and FastText, training on pretrained vectors Question: I am trying to load the pretrained vec file of Facebook fasttext crawl-300d-2M.vec with the next code: from gensim.models.fasttext import load_facebook_model, load_facebook_vectors model_facebook = load_facebook_vectors(‘fasttext/crawl-300d-2M.vec’) But it fails with the next error: NotImplementedError: Supervised fastText models are not supported It is not possible to …
Continue training a FastText model Question: I have downloaded a .bin FastText model, and I use it with gensim as follows: model = FastText.load_fasttext_format(“cc.fr.300.bin”) I would like to continue the training of the model to adapt it to my domain. After checking FastText’s Github and the Gensim documentation it seems like it is not currently …
