Using throttler and joblib for heavy jobs and getting "cannot unpack non-iterable function object" Question: I have a lot of heavy jobs which need to send throttled HTTP requests (2 requests/sec) and calculate the results based on the response values. I use throttler to control the request rate, and use joblib to calculate the results …
Using filename to initialize new DataFrame without exec Question: I have a series of files to process and I need DataFrames that will have the same names as the files. The number of files is large so I wanted to make the processing parallel with joblib. Unfortunately joblib is not accepting exec as an element …
Parallelize loop with "non-mapable" function in Python Question: I solve numerically integral equation on the time interval from 0 to Tmax with predefined step dT. I do it in the for loop: list_of_values = [] for i in range(dT,Tmax+dT,dT): func_at_t= my_fancy_solver(func_at_t) list_of_values.append(func_at_t) The function my_fancy_solver has one argument and inside it looks as follows: def …
Pyinstaller created exe file cannot load Decision Tree model using joblib Question: I created an exe file of my large python script using the following command – pyinstaller gui_final.py –onefile –hidden-import=sklearn –hidden-import=ipaddress –hidden-import=PIL –hidden-import=pickle –hidden-import=shutil –hidden-import=joblib The exe file works fine until I load my decision tree model file (dtree.joblib) using JOBLIB. clf = joblib.load("dtree.joblib") …
No module named joblib Python Question: I’m trying to run this : https://github.com/HansiMcKlaus/AudioSpectrumVisualizer And so I followed needed to be done to run it. I pip install all the requirements, ffmpeg and try to run with the default example. % python AudioSpectrumVisualizer.py ‘sample.mp3’ And I have the error message : ImportError: No module named joblib …
Joblib.load error: No module named 'scipy.sparse._csr' Question: Python version: 3.7 (I have to use this version) OS: Linux Cloud Platform: Azure Resource: Azure function with python Goal: Load a model created with skit-learn version 1.0.2 with the following dependencies installed: numpy: 1.17.3 joblib: 1.1.0 scipy: 1.7.3 I am using joblib to load a skit-learn model …
Joblib Parallel doesn't terminate processes Question: I run the code in parallel in the following fashion: grouped_data = Parallel(n_jobs=14)(delayed(function)(group) for group in grouped_data) After the computation is done I can see all the spawned processes are still active and memory consuming in a system monitor: And all these processes are not killed till the main …
Update scikit model so it is compatible with newest version Question: I have a question about scikit models and (retro-)compatibility. I have a model (saved using joblib) created in Python 3.5 from scikit-learn 0.21.2, which I then analyze with the package shap version 0.30. Since I upgraded to Ubuntu 20.04 I have Python 3.8 (and …
How to write/load machine learning model to/from S3 bucket through joblib? Question: I have an ml model which I want to save on an S3 bucket. from lightgbm.sklearn import LGBMClassifier # Initialize model mdl_lightgbm = LGBMClassifier(boosting_type=’rf’, objective=’binary’) # Fit data mdl_lightgbm.fit(X,Y) # Save model to dictionary mdl_dict = {‘mdl_fitted’:mdl_lightgbm} For some reasons, I’m storing the …
TransformedTargetRegressor save and load error Question: I’m defining my custom regressor using the TransformedTargetRegressor, adding it to the pipeline and saving the model in the ‘joblib’ file. However as I’m trying to load the model, I get an error module ‘main‘ has no attribute ‘transform_targets’ where transform_targets is one of the functions defined for the …
