How can I list all the virtual environments created with venv?
Question:
Someone’s just asked me how to list all the virtual environments created with venv.
I could only think of searching for pyvenv.cfg files to find them. Something like:
from pathlib import Path
venv_list = [str(p.parent) for p in Path.home().rglob('pyvenv.cfg')]
This could potentially include some false positives. Is there a better way to list all the virtual environment created with venv?
NB: The question is about venv specifically, NOT Anaconda, virtualenv, etc.
Answers:
On Linux/macOS this should get most of it
find ~ -d -name "site-packages" 2>/dev/null
Looking for directories under your home that are named "site-packages" which is where venv puts its pip-installed stuff. the /dev/null bit cuts down on the chattiness of things you don’t have permission to look into.
Or you can look at the specifics of a particular expected file. For example, activate has nondestructive as content. Then you need to look for a pattern than matches venv but not anaconda and the rest.
find ~ -type f -name "activate" -exec egrep -l nondestructive /dev/null {} ; 2>/dev/null
macos mdfind
On macos, this is is pretty fast, using mdfind (locate on Linux would probably have similar performance.
mdfind -name activate | egrep /bin/activate$| xargs -o egrep -l nondestructive 2>/dev/null | xargs -L 1 dirname | xargs -L 1 dirname
So we :
- look for all activate files
- egrep to match only bin/activate files (
mdfind matches on things like .../bin/ec2-activate-license)
- look for that
nondestructive and print filename where there is a match.
- the 2
xargs -L 1 dirname allow us to "climb up" from /bin/activate to the virtual env’s root.
Helper function with -v flag to show details.
jvenvfindall(){ # search for Python virtual envs. -v for verbose details
unset verbose
OPTIND=1
while getopts 'v' OPTION; do
case "$OPTION" in
v)
verbose=1
;;
?)
;;
esac
done
shift "$(($OPTIND -1))"
local bup=$PWD
for dn in $(mdfind -name activate | egrep /bin/activate$| xargs -o egrep -l nondestructive 2>/dev/null | xargs -L 1 dirname | xargs -L 1 dirname)
do
if [[ -z "$verbose" ]]; then
printf "$dnn"
else
printf "nnvenv info for $dn:n"
cd $dn
echo space usage, $(du -d 0 -h)
#requires the jq and jc utilities... to extract create and modification times
echo create, mod dttm: $(stat . | jc --stat | jq '.[]|{birth_time, change_time}')
tree -d -L 1 lib
fi
done
cd $bup
}
output:
...
venv info for /Users/me/kds2/issues2/067.pip-stripper/010.fixed.p1.check_venv/venvtest:
space usage, 12M .
create, mod dttm: { "birth_time": "Apr 16 13:04:43 2019", "change_time": "Sep 30 00:00:39 2019" }
lib
└── python3.6
...
Hmmm, disk usage is not that bad, but something similar for node_modules might save some real space.
The standard library venv does not keep track of any of the created virtual environments. Therefore, the only possibility to list all of them is to search for your hard drive for folders that meet certain criterion.
The PEP 405 gives quite good listing about what should be included in a folder so that it is a virtual environment. Also this blog post explains part of the virtual environment internals quite well. The definition of a virtual environment is
A Python virtual environment in its simplest form would consist
of nothing more than a copy or symlink of the Python binary
accompanied by a pyvenv.cfg file and a site-packages directory. (PEP 405)
In summary, you will have to search your hard drive for folders that:
Linux / macOS
- Has
pyvenv.cfg with home key*
- Has
bin/python3 or bin/python
- Has
lib/<python-version>/site-packages/, where <python-version> is for example python3.3.
- Optional: If created with
venv, has also bin/activate (source). A folder is considered virtual environment even if this would be lacking. (PEP 405)
Windows
- Has
pyvenv.cfg with home key*
- Has
Script/python.exe
- Has
lib/<python-version>/site-packages/, where <python-version> is for example python3.3.
- Optional: If created with
venv, has also Scripts/activate.bat and Scripts/Activate.ps1 (source). A folder is considered virtual environment even if these would be lacking. (PEP 405)
* pyvenv.cfg
The pyvenv.cfg can actually be in the same folder or one subfolder above the python executable. The pyvenv.cfg that belongs to a virtual environment must have home = <home> row, where <home> is the directory containing the Python executable used to create the virtual environment. (PEP 405).
Someone’s just asked me how to list all the virtual environments created with venv.
I could only think of searching for pyvenv.cfg files to find them. Something like:
from pathlib import Path
venv_list = [str(p.parent) for p in Path.home().rglob('pyvenv.cfg')]
This could potentially include some false positives. Is there a better way to list all the virtual environment created with venv?
NB: The question is about venv specifically, NOT Anaconda, virtualenv, etc.
On Linux/macOS this should get most of it
find ~ -d -name "site-packages" 2>/dev/null
Looking for directories under your home that are named "site-packages" which is where venv puts its pip-installed stuff. the /dev/null bit cuts down on the chattiness of things you don’t have permission to look into.
Or you can look at the specifics of a particular expected file. For example, activate has nondestructive as content. Then you need to look for a pattern than matches venv but not anaconda and the rest.
find ~ -type f -name "activate" -exec egrep -l nondestructive /dev/null {} ; 2>/dev/null
macos mdfind
On macos, this is is pretty fast, using mdfind (locate on Linux would probably have similar performance.
mdfind -name activate | egrep /bin/activate$| xargs -o egrep -l nondestructive 2>/dev/null | xargs -L 1 dirname | xargs -L 1 dirname
So we :
- look for all activate files
- egrep to match only bin/activate files (
mdfindmatches on things like.../bin/ec2-activate-license) - look for that
nondestructiveand print filename where there is a match. - the 2
xargs -L 1 dirnameallow us to "climb up" from/bin/activateto the virtual env’s root.
Helper function with -v flag to show details.
jvenvfindall(){ # search for Python virtual envs. -v for verbose details
unset verbose
OPTIND=1
while getopts 'v' OPTION; do
case "$OPTION" in
v)
verbose=1
;;
?)
;;
esac
done
shift "$(($OPTIND -1))"
local bup=$PWD
for dn in $(mdfind -name activate | egrep /bin/activate$| xargs -o egrep -l nondestructive 2>/dev/null | xargs -L 1 dirname | xargs -L 1 dirname)
do
if [[ -z "$verbose" ]]; then
printf "$dnn"
else
printf "nnvenv info for $dn:n"
cd $dn
echo space usage, $(du -d 0 -h)
#requires the jq and jc utilities... to extract create and modification times
echo create, mod dttm: $(stat . | jc --stat | jq '.[]|{birth_time, change_time}')
tree -d -L 1 lib
fi
done
cd $bup
}
output:
...
venv info for /Users/me/kds2/issues2/067.pip-stripper/010.fixed.p1.check_venv/venvtest:
space usage, 12M .
create, mod dttm: { "birth_time": "Apr 16 13:04:43 2019", "change_time": "Sep 30 00:00:39 2019" }
lib
└── python3.6
...
Hmmm, disk usage is not that bad, but something similar for node_modules might save some real space.
The standard library venv does not keep track of any of the created virtual environments. Therefore, the only possibility to list all of them is to search for your hard drive for folders that meet certain criterion.
The PEP 405 gives quite good listing about what should be included in a folder so that it is a virtual environment. Also this blog post explains part of the virtual environment internals quite well. The definition of a virtual environment is
A Python virtual environment in its simplest form would consist
of nothing more than a copy or symlink of the Python binary
accompanied by a pyvenv.cfg file and a site-packages directory. (PEP 405)
In summary, you will have to search your hard drive for folders that:
Linux / macOS
- Has
pyvenv.cfgwithhomekey* - Has
bin/python3orbin/python - Has
lib/<python-version>/site-packages/, where<python-version>is for examplepython3.3. - Optional: If created with
venv, has alsobin/activate(source). A folder is considered virtual environment even if this would be lacking. (PEP 405)
Windows
- Has
pyvenv.cfgwithhomekey* - Has
Script/python.exe - Has
lib/<python-version>/site-packages/, where<python-version>is for examplepython3.3. - Optional: If created with
venv, has alsoScripts/activate.batandScripts/Activate.ps1(source). A folder is considered virtual environment even if these would be lacking. (PEP 405)
* pyvenv.cfg
The pyvenv.cfg can actually be in the same folder or one subfolder above the python executable. The pyvenv.cfg that belongs to a virtual environment must have home = <home> row, where <home> is the directory containing the Python executable used to create the virtual environment. (PEP 405).