Update scikit model so it is compatible with newest version
Question:
I have a question about scikit models and (retro-)compatibility.
I have a model (saved using joblib) created in Python 3.5 from scikit-learn 0.21.2, which I then analyze with the package shap version 0.30. Since I upgraded to Ubuntu 20.04 I have Python 3.8 (and newer versions of both scikit-learn and shap).
Because of the new packages version I cannot load them with Python 3.8, so I make a virtual environment with Py3.5 and the original package versions.
Now my question is: is there a way to re-dump with joblib the models so I can also open them with Python 3.8? I’d like to re-analyze the model with the newest version of the package shap (but of course it has a scikit version requirement that would break the joblib loading).
Alternatively, what other options do I have? (The only thing I do not want is to re-train the model).
Answers:
There are no standard solutions within scikit-learn. If your model is supported, you can try sklearn-json.
Although this does not solve your current issue, you can in the future save your models in formats with fewer compatibility issues – see the Interoperable formats section in scikit-learn’s Model persistence page.
Your options:
- load the old version and then serialize it in a more interoperable format that doesn’t suffer from the version requirements of pickle:
https://scikit-learn.org/stable/model_persistence.html#interoperable-formats
- retrain the model using original data with the new library version
- manually transfer weights and all prameters to new model instance (inspect old object for all paramaters and copy them to a new object with the new library/python version) See: https://stackoverflow.com/a/77657267/21040193
I have a question about scikit models and (retro-)compatibility.
I have a model (saved using joblib) created in Python 3.5 from scikit-learn 0.21.2, which I then analyze with the package shap version 0.30. Since I upgraded to Ubuntu 20.04 I have Python 3.8 (and newer versions of both scikit-learn and shap).
Because of the new packages version I cannot load them with Python 3.8, so I make a virtual environment with Py3.5 and the original package versions.
Now my question is: is there a way to re-dump with joblib the models so I can also open them with Python 3.8? I’d like to re-analyze the model with the newest version of the package shap (but of course it has a scikit version requirement that would break the joblib loading).
Alternatively, what other options do I have? (The only thing I do not want is to re-train the model).
There are no standard solutions within scikit-learn. If your model is supported, you can try sklearn-json.
Although this does not solve your current issue, you can in the future save your models in formats with fewer compatibility issues – see the Interoperable formats section in scikit-learn’s Model persistence page.
Your options:
- load the old version and then serialize it in a more interoperable format that doesn’t suffer from the version requirements of pickle:
https://scikit-learn.org/stable/model_persistence.html#interoperable-formats - retrain the model using original data with the new library version
- manually transfer weights and all prameters to new model instance (inspect old object for all paramaters and copy them to a new object with the new library/python version) See: https://stackoverflow.com/a/77657267/21040193