Joblib.load error: No module named 'scipy.sparse._csr'

Question:

  • Python version: 3.7 (I have to use this version)
  • OS: Linux
  • Cloud Platform: Azure
  • Resource: Azure function with python
  • Goal: Load a model created with skit-learn version 1.0.2 with the following dependencies installed:

numpy: 1.17.3 joblib: 1.1.0 scipy: 1.7.3

I am using joblib to load a skit-learn model that I trained (By the way I created the model locally in my machine with python 3.9). However, I am getting the following error:

Traceback (most recent call last):
File "/home/site/wwwroot/sortierung/__init__.py", line 51, in main
prediction_file_path)
File "/home/site/wwwroot/shared_code/custom_functions_prediction.py", line 255, in predict
result.update(classify_mail(m,s,X, stop_words, model_folder_path))
File "/home/site/wwwroot/shared_code/custom_functions_prediction.py", line 105, in classify_mail
model = load(modelFilePath)
File "/home/site/wwwroot/.python_packages/lib/site-packages/joblib/numpy_pickle.py", line 587, in load
obj = _unpickle(fobj, filename, mmap_mode)
File "/home/site/wwwroot/.python_packages/lib/site-packages/joblib/numpy_pickle.py", line 506, in _unpickle
obj = unpickler.load()
File "/usr/local/lib/python3.7/pickle.py", line 1088, in load
dispatch[key[0]](self)
File "/usr/local/lib/python3.7/pickle.py", line 1385, in load_stack_global
self.append(self.find_class(module, name))
File "/usr/local/lib/python3.7/pickle.py", line 1426, in find_class
__import__(module, level=0)nModuleNotFoundError: No module named 'scipy.sparse._csr'

I checked in the scipy folder installed and I could not find this module. How could I solve this issue?. Tks in advance

Asked By: sergioMoreno

||
Source

Answers:

To resolve this ModuleNotFoundError: No module named 'scipy.sparse._csr' error, try the following way:

This error occurred because you have created a model in Python 3.9 but running it on Python 3.7.

You can try creating a model in Python 3.7 or upgrade the Azure Python function app to a specific version of Python 3.9.

To change the Python version to 3.9, according to documentation:

  • You can update the linuxFxVersion setting in the function app with the az functionapp config set command.
az functionapp config set --name <FUNCTION_APP> 
--resource-group <RESOURCE_GROUP> 
--linux-fx-version "python|3.9"

References: No module named ‘scipy.sparse._csr’ and How to change python version of azure function

Answered By: Ecstasy

I agree with the last answer.

I have used python 3.7.10 and scipy 1.8.1 to dump the dict with csr matrix in local windows environment. After that, load the file in linux environment by python 3.8.10 and scipy 1.7.1.

self.word_multi_model_feature = pickle.load(f_dict_feature)
ModuleNotFoundError: No module named ‘scipy.sparse._csr’

You should change the load python environment to be same as the dump enviroment.

Answered By: Xucheng Huang