pip freeze > requirements.txt in VScode calls libraries that i didn't import to my project
Question:
When i call pip freeze > requirements.txt in VScode, it also adds libraries that i didn’t import to my project. I have these libraries in my Jupyter notebook:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import scipy.stats as stats
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.model_selection import GridSearchCV,KFold
from sklearn.metrics import r2_score,mean_squared_error,mean_absolute_error
from sklearn.preprocessing import LabelEncoder,OneHotEncoder,OrdinalEncoder,StandardScaler,MinMaxScaler
from sklearn.model_selection import train_test_split,cross_val_score
from sklearn.ensemble import AdaBoostRegressor,GradientBoostingRegressor,RandomForestRegressor
from catboost import CatBoostRegressor
from xgboost import XGBRegressor
import pickle
import warnings
How can i create requirements.txt file with only imported libraries?
image link:
https://i.stack.imgur.com/sD1IX.jpg
Answers:
Usually better approach, than using pip freeze, is to construct the requirements file manually, as freezing requirements will also include requirements of requirements, etc, which can easily change without you knowing. Within your requirements file you should be also specifying version ranges, like
cachetools>=5.0,<6.0
which is nearly impossible to manage with a huge number of sub-requirements flushed from pip freeze.
See how it is done in any big open source python libraries like click, pillow or rich.
Most libraries use semantic versioning which has really straightforward rules about version tags.
EDIT:
There happens to be a really handy tool, called poetry, especially for bigger projects (but might be useful also for smaller ones too If you wish to pay enough attention to configuring it). I’m aware that it doesn’t fully match the topic, but this tool is great when it comes to dependencies, I hope it will help anyone.
When i call pip freeze > requirements.txt in VScode, it also adds libraries that i didn’t import to my project. I have these libraries in my Jupyter notebook:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import scipy.stats as stats
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.model_selection import GridSearchCV,KFold
from sklearn.metrics import r2_score,mean_squared_error,mean_absolute_error
from sklearn.preprocessing import LabelEncoder,OneHotEncoder,OrdinalEncoder,StandardScaler,MinMaxScaler
from sklearn.model_selection import train_test_split,cross_val_score
from sklearn.ensemble import AdaBoostRegressor,GradientBoostingRegressor,RandomForestRegressor
from catboost import CatBoostRegressor
from xgboost import XGBRegressor
import pickle
import warnings
How can i create requirements.txt file with only imported libraries?
image link:
https://i.stack.imgur.com/sD1IX.jpg
Usually better approach, than using pip freeze, is to construct the requirements file manually, as freezing requirements will also include requirements of requirements, etc, which can easily change without you knowing. Within your requirements file you should be also specifying version ranges, like
cachetools>=5.0,<6.0
which is nearly impossible to manage with a huge number of sub-requirements flushed from pip freeze.
See how it is done in any big open source python libraries like click, pillow or rich.
Most libraries use semantic versioning which has really straightforward rules about version tags.
EDIT:
There happens to be a really handy tool, called poetry, especially for bigger projects (but might be useful also for smaller ones too If you wish to pay enough attention to configuring it). I’m aware that it doesn’t fully match the topic, but this tool is great when it comes to dependencies, I hope it will help anyone.