Python vitual environment (venv): Share libraries in usage and dev/test venvs
Question:
I am new in python venv, so sorry for possible stupid question.
I am developing a small library. I’ve created dev virtual environment with all packages which is necessary for the library usage and freeze all versions of requirements to requirements.txt.
I also would like to create requirements_test.txt with all packages needed for development and tests. So the user will install requirements from requirements.txt while the developer from requirements_test.txt with all nessesary libs (e.g. pytest, asv, sphinx).
Now I’ve created dev venv and now I want to create test venv, of course I don’t want to install the same libs twice. Is it possible to share some libs from one venv to another?
Answers:
I think it is recommended and advised to have multiple venvs, and multiple environments, be it on the same machine. so just have another venv. Its okay to have same library being present in both venvs.
Even with virtual environments, there are many libraries that come preinstalled with python and are not necessary in the package that you are developing, when I run pip freeze in a brand new virtual environment it dumps 30 packages, and surely they are not needed for my project.
I recommend you to do the dependency maintenance manually (at least the production ones), this way you won’t include useless libraries and you will keep your dependency file clean.
Is it possible to share some libs from one venv to another?
No. The same library (or application) will be installed once per virtual environment, the installations can not be shared between environments. And it is perfectly fine like this. That is the whole point of virtual environments, that two installations from the same library are isolated from each other, in particular for the case where two different versions of the same library are required for two different projects.
To be completely fair, there are ways to share one installation of the same library between two virtual environments and reasons to do so. One famous example I know of currently is in the newer releases of virtualenv (versions 20+). In short: this tool creates virtual environments and (under specific conditions) is able to reuse (share) the installations of pip, setuptools, and wheel in multiple environments, see the app-data seeder for virtualenv.
Some more discussions on the topic:
You can use virtualenv --system-site-packages to symlink from the base system for sharing between dev and user. Then add the dev specific testing packages.
I am new in python venv, so sorry for possible stupid question.
I am developing a small library. I’ve created dev virtual environment with all packages which is necessary for the library usage and freeze all versions of requirements to requirements.txt.
I also would like to create requirements_test.txt with all packages needed for development and tests. So the user will install requirements from requirements.txt while the developer from requirements_test.txt with all nessesary libs (e.g. pytest, asv, sphinx).
Now I’ve created dev venv and now I want to create test venv, of course I don’t want to install the same libs twice. Is it possible to share some libs from one venv to another?
I think it is recommended and advised to have multiple venvs, and multiple environments, be it on the same machine. so just have another venv. Its okay to have same library being present in both venvs.
Even with virtual environments, there are many libraries that come preinstalled with python and are not necessary in the package that you are developing, when I run pip freeze in a brand new virtual environment it dumps 30 packages, and surely they are not needed for my project.
I recommend you to do the dependency maintenance manually (at least the production ones), this way you won’t include useless libraries and you will keep your dependency file clean.
Is it possible to share some libs from one venv to another?
No. The same library (or application) will be installed once per virtual environment, the installations can not be shared between environments. And it is perfectly fine like this. That is the whole point of virtual environments, that two installations from the same library are isolated from each other, in particular for the case where two different versions of the same library are required for two different projects.
To be completely fair, there are ways to share one installation of the same library between two virtual environments and reasons to do so. One famous example I know of currently is in the newer releases of virtualenv (versions 20+). In short: this tool creates virtual environments and (under specific conditions) is able to reuse (share) the installations of pip, setuptools, and wheel in multiple environments, see the app-data seeder for virtualenv.
Some more discussions on the topic:
You can use virtualenv --system-site-packages to symlink from the base system for sharing between dev and user. Then add the dev specific testing packages.