How do I get a list of locally installed Python modules?
Question:
How do I get a list of Python modules installed on my computer?
Answers:
From the shell
ls site-packages
If that’s not helpful, you can do this.
import sys
import os
for p in sys.path:
print os.listdir( p )
And see what that produces.
help('modules')
in a Python shell/prompt.
Now, these methods I tried myself, and I got exactly what was advertised: All the modules.
Alas, really you don’t care much about the stdlib, you know what you get with a python install.
Really, I want the stuff that I installed.
What actually, surprisingly, worked just fine was:
pip freeze
Which returned:
Fabric==0.9.3
apache-libcloud==0.4.0
bzr==2.3b4
distribute==0.6.14
docutils==0.7
greenlet==0.3.1
ipython==0.10.1
iterpipes==0.4
libxml2-python==2.6.21
I say “surprisingly” because the package install tool is the exact place one would expect to find this functionality, although not under the name ‘freeze’ but python packaging is so weird, that I am flabbergasted that this tool makes sense. Pip 0.8.2, Python 2.7.
-
In ipython
you can type “import
Tab“.
-
In the standard Python interpreter, you can type “help('modules')
“.
-
At the command-line, you can use pydoc
modules
.
-
In a script, call pkgutil.iter_modules()
.
Very simple searching using pkgutil.iter_modules
from pkgutil import iter_modules
a=iter_modules()
while True:
try: x=a.next()
except: break
if 'searchstr' in x[1]: print x[1]
I just use this to see currently used modules:
import sys as s
s.modules.keys()
which shows all modules running on your python.
For all built-in modules use:
s.modules
Which is a dict containing all modules and import objects.
I ran into a custom installed python 2.7 on OS X. It required X11 to list modules installed (both using help and pydoc).
To be able to list all modules without installing X11 I ran pydoc as http-server, i.e.:
pydoc -p 12345
Then it’s possible to direct Safari to http://localhost:12345/
to see all modules.
Aside from using pip freeze
I have been installing yolk in my virtual environments.
Since pip version 1.3, you’ve got access to:
pip list
Which seems to be syntactic sugar for “pip freeze”. It will list all of the modules particular to your installation or virtualenv, along with their version numbers. Unfortunately it does not display the current version number of any module, nor does it wash your dishes or shine your shoes.
In normal shell just use
pydoc modules
In case you have an anaconda python distribution installed, you could also use
$conda list
in addition to solutions described above.
If we need to list the installed packages in the Python shell, we can use the help
command as follows
>>> help('modules package')
- to get all available modules, run
sys.modules
- to get all installed modules (read: installed by
pip
), you may look at pip.get_installed_distributions()
For the second purpose, example code:
import pip
for package in pip.get_installed_distributions():
name = package.project_name # SQLAlchemy, Django, Flask-OAuthlib
key = package.key # sqlalchemy, django, flask-oauthlib
module_name = package._get_metadata("top_level.txt") # sqlalchemy, django, flask_oauthlib
location = package.location # virtualenv lib directory etc.
version = package.version # version number
Solution
Do not use with pip > 10.0!
My 50 cents for getting a pip freeze
-like list from a Python script:
import pip
installed_packages = pip.get_installed_distributions()
installed_packages_list = sorted(["%s==%s" % (i.key, i.version)
for i in installed_packages])
print(installed_packages_list)
As a (too long) one liner:
sorted(["%s==%s" % (i.key, i.version) for i in pip.get_installed_distributions()])
Giving:
['behave==1.2.4', 'enum34==1.0', 'flask==0.10.1', 'itsdangerous==0.24',
'jinja2==2.7.2', 'jsonschema==2.3.0', 'markupsafe==0.23', 'nose==1.3.3',
'parse-type==0.3.4', 'parse==1.6.4', 'prettytable==0.7.2', 'requests==2.3.0',
'six==1.6.1', 'vioozer-metadata==0.1', 'vioozer-users-server==0.1',
'werkzeug==0.9.4']
Scope
This solution applies to the system scope or to a virtual environment scope, and covers packages installed by setuptools
, pip
and (god forbid) easy_install
.
My use case
I added the result of this call to my flask server, so when I call it with http://example.com/exampleServer/environment
I get the list of packages installed on the server’s virtualenv. It makes debugging a whole lot easier.
Caveats
I have noticed a strange behaviour of this technique – when the Python interpreter is invoked in the same directory as a setup.py
file, it does not list the package installed by setup.py
.
Steps to reproduce:
Create a virtual environment
$ cd /tmp
$ virtualenv test_env
New python executable in test_env/bin/python
Installing setuptools, pip...done.
$ source test_env/bin/activate
(test_env) $
Clone a git repo with setup.py
(test_env) $ git clone https://github.com/behave/behave.git
Cloning into 'behave'...
remote: Reusing existing pack: 4350, done.
remote: Total 4350 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (4350/4350), 1.85 MiB | 418.00 KiB/s, done.
Resolving deltas: 100% (2388/2388), done.
Checking connectivity... done.
We have behave’s setup.py
in /tmp/behave
:
(test_env) $ ls /tmp/behave/setup.py
/tmp/behave/setup.py
Install the python package from the git repo
(test_env) $ cd /tmp/behave && pip install .
running install
...
Installed /private/tmp/test_env/lib/python2.7/site-packages/enum34-1.0-py2.7.egg
Finished processing dependencies for behave==1.2.5a1
If we run the aforementioned solution from /tmp
>>> import pip
>>> sorted(["%s==%s" % (i.key, i.version) for i in pip.get_installed_distributions()])
['behave==1.2.5a1', 'enum34==1.0', 'parse-type==0.3.4', 'parse==1.6.4', 'six==1.6.1']
>>> import os
>>> os.getcwd()
'/private/tmp'
If we run the aforementioned solution from /tmp/behave
>>> import pip
>>> sorted(["%s==%s" % (i.key, i.version) for i in pip.get_installed_distributions()])
['enum34==1.0', 'parse-type==0.3.4', 'parse==1.6.4', 'six==1.6.1']
>>> import os
>>> os.getcwd()
'/private/tmp/behave'
behave==1.2.5a1
is missing from the second example, because the working directory contains behave
‘s setup.py
file.
I could not find any reference to this issue in the documentation. Perhaps I shall open a bug for it.
on windows, Enter this in cmd
c:pythonlibs>python -m pip freeze
pip freeze does it all finding packages however one can simply write the following command to list all paths where python packages are.
>>> import site; site.getsitepackages()
['/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages']
This solution is primary based on modules importlib
and pkgutil
and work with CPython 3.4 and CPython 3.5, but has no support for the CPython 2.
Explanation
sys.builtin_module_names
– names all built-in modules (look my answer here)
pkgutil.iter_modules()
– returns an information about all available modules
importlib.util.find_spec()
– returns an information about importing module, if exists
BuiltinImporter
– an importer for built-in modules (docs)
SourceFileLoader
– an importer for a standard Python module (by default has extension *.py) (docs)
ExtensionFileLoader
– an importer for modules as shared library (written on the C or C++)
Full code
import sys
import os
import shutil
import pkgutil
import importlib
import collections
if sys.version_info.major == 2:
raise NotImplementedError('CPython 2 is not supported yet')
def main():
# name this file (module)
this_module_name = os.path.basename(__file__).rsplit('.')[0]
# dict for loaders with their modules
loaders = collections.OrderedDict()
# names`s of build-in modules
for module_name in sys.builtin_module_names:
# find an information about a module by name
module = importlib.util.find_spec(module_name)
# add a key about a loader in the dict, if not exists yet
if module.loader not in loaders:
loaders[module.loader] = []
# add a name and a location about imported module in the dict
loaders[module.loader].append((module.name, module.origin))
# all available non-build-in modules
for module_name in pkgutil.iter_modules():
# ignore this module
if this_module_name == module_name[1]:
continue
# find an information about a module by name
module = importlib.util.find_spec(module_name[1])
# add a key about a loader in the dict, if not exists yet
loader = type(module.loader)
if loader not in loaders:
loaders[loader] = []
# add a name and a location about imported module in the dict
loaders[loader].append((module.name, module.origin))
# pretty print
line = '-' * shutil.get_terminal_size().columns
for loader, modules in loaders.items():
print('{0}n{1}: {2}n{0}'.format(line, len(modules), loader))
for module in modules:
print('{0:30} | {1}'.format(module[0], module[1]))
if __name__ == '__main__':
main()
Usage
For the CPython3.5 (truncated)
$ python3.5 python_modules_info.py
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
30: <class '_frozen_importlib.BuiltinImporter'>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
_ast | built-in
_codecs | built-in
_collections | built-in
_functools | built-in
_imp | None
_io | built-in
_locale | built-in
_operator | built-in
_signal | built-in
_sre | built-in
_stat | built-in
_string | built-in
_symtable | built-in
_thread | built-in
(****************************truncated*******************************)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
227: <class '_frozen_importlib_external.SourceFileLoader'>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__future__ | /usr/local/lib/python3.5/__future__.py
_bootlocale | /usr/local/lib/python3.5/_bootlocale.py
_collections_abc | /usr/local/lib/python3.5/_collections_abc.py
_compat_pickle | /usr/local/lib/python3.5/_compat_pickle.py
_compression | /usr/local/lib/python3.5/_compression.py
_dummy_thread | /usr/local/lib/python3.5/_dummy_thread.py
_markupbase | /usr/local/lib/python3.5/_markupbase.py
_osx_support | /usr/local/lib/python3.5/_osx_support.py
_pydecimal | /usr/local/lib/python3.5/_pydecimal.py
_pyio | /usr/local/lib/python3.5/_pyio.py
_sitebuiltins | /usr/local/lib/python3.5/_sitebuiltins.py
(****************************truncated*******************************)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
64: <class '_frozen_importlib_external.ExtensionFileLoader'>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
_bisect | /usr/local/lib/python3.5/lib-dynload/_bisect.cpython-35m-x86_64-linux-gnu.so
_bz2 | /usr/local/lib/python3.5/lib-dynload/_bz2.cpython-35m-x86_64-linux-gnu.so
_codecs_cn | /usr/local/lib/python3.5/lib-dynload/_codecs_cn.cpython-35m-x86_64-linux-gnu.so
_codecs_hk | /usr/local/lib/python3.5/lib-dynload/_codecs_hk.cpython-35m-x86_64-linux-gnu.so
_codecs_iso2022 | /usr/local/lib/python3.5/lib-dynload/_codecs_iso2022.cpython-35m-x86_64-linux-gnu.so
(****************************truncated*******************************)
For the CPython3.4 (truncated)
$ python3.4 python_modules_info.py
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
54: <class '_frozen_importlib.BuiltinImporter'>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
_ast | built-in
_bisect | built-in
_codecs | built-in
_collections | built-in
_datetime | built-in
_elementtree | built-in
_functools | built-in
_heapq | built-in
_imp | None
_io | built-in
_locale | built-in
_md5 | built-in
_operator | built-in
_pickle | built-in
_posixsubprocess | built-in
_random | built-in
(****************************truncated*******************************)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
246: <class '_frozen_importlib.SourceFileLoader'>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__future__ | /usr/lib/python3.4/__future__.py
_bootlocale | /usr/lib/python3.4/_bootlocale.py
_collections_abc | /usr/lib/python3.4/_collections_abc.py
_compat_pickle | /usr/lib/python3.4/_compat_pickle.py
_dummy_thread | /usr/lib/python3.4/_dummy_thread.py
_markupbase | /usr/lib/python3.4/_markupbase.py
_osx_support | /usr/lib/python3.4/_osx_support.py
_pyio | /usr/lib/python3.4/_pyio.py
(****************************truncated*******************************)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
44: <class '_frozen_importlib.ExtensionFileLoader'>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
_bz2 | /usr/lib/python3.4/lib-dynload/_bz2.cpython-34m-x86_64-linux-gnu.so
_codecs_cn | /usr/lib/python3.4/lib-dynload/_codecs_cn.cpython-34m-x86_64-linux-gnu.so
_codecs_hk | /usr/lib/python3.4/lib-dynload/_codecs_hk.cpython-34m-x86_64-linux-gnu.so
_codecs_iso2022 | /usr/lib/python3.4/lib-dynload/_codecs_iso2022.cpython-34m-x86_64-linux-gnu.so
_codecs_jp | /usr/lib/python3.4/lib-dynload/_codecs_jp.cpython-34m-x86_64-linux-gnu.so
_codecs_kr | /usr/lib/python3.4/lib-dynload/_codecs_kr.cpython-34m-x86_64-linux-gnu.so
_codecs_tw | /usr/lib/python3.4/lib-dynload/_codecs_tw.cpython-34m-x86_64-linux-gnu.so
_crypt | /usr/lib/python3.4/lib-dynload/_crypt.cpython-34m-x86_64-linux-gnu.so
(****************************truncated*******************************)
As of pip 10, the accepted answer will no longer work. The development team has removed access to the get_installed_distributions
routine. There is an alternate function in the setuptools
for doing the same thing. Here is an alternate version that works with pip 10:
import pkg_resources
installed_packages = pkg_resources.working_set
installed_packages_list = sorted(["%s==%s" % (i.key, i.version)
for i in installed_packages])
print(installed_packages_list)
Please let me know if it will or won’t work in previous versions of pip, too.
I normally use pip list
to get a list of packages (with version).
This works in a virtual environment too, of course. To show what’s installed in only the virtual environment (not global packages), use pip list --local
.
Here’s documentation showing all the available pip list
options, with several good examples.
There are many ideas, initially I am pondering on these two:
pip
cons: not always installed
help(‘modules’)
cons: output to console; with broken modules (see ubuntu…) can segfault
I need an easy approach, using basic libraries and compatible with old python 2.x
And I see the light: listmodules.py
Hidden in the documentation source directory in 2.5 is a small script that lists all available modules for a Python installation.
Pros:
uses only imp, sys, os, re, time
designed to run on Python 1.5.2 and newer
the source code is really compact, so you can easy tinkering with it, for example to pass an exception list of buggy modules (don’t try to import them)
Warning: Adam Matan discourages this use in pip > 10.0. Also, read @sinoroc’s comment below
This was inspired by Adam Matan’s answer (the accepted one):
import tabulate
try:
from pip import get_installed_distributions
except:
from pip._internal.utils.misc import get_installed_distributions
tabpackages = []
for _, package in sorted([('%s %s' % (i.location, i.key), i) for i in get_installed_distributions()]):
tabpackages.append([package.location, package.key, package.version])
print(tabulate.tabulate(tabpackages))
which then prints out a table in the form of
19:33 [email protected] [iot-wifi-2] ~/python$ python installed_packages.py
------------------------------------------- -------------- ------
/home/pi/.local/lib/python2.7/site-packages enum-compat 0.0.2
/home/pi/.local/lib/python2.7/site-packages enum34 1.1.6
/home/pi/.local/lib/python2.7/site-packages pexpect 4.2.1
/home/pi/.local/lib/python2.7/site-packages ptyprocess 0.5.2
/home/pi/.local/lib/python2.7/site-packages pygatt 3.2.0
/home/pi/.local/lib/python2.7/site-packages pyserial 3.4
/usr/local/lib/python2.7/dist-packages bluepy 1.1.1
/usr/local/lib/python2.7/dist-packages click 6.7
/usr/local/lib/python2.7/dist-packages click-datetime 0.2
/usr/local/lib/python2.7/dist-packages construct 2.8.21
/usr/local/lib/python2.7/dist-packages pyaudio 0.2.11
/usr/local/lib/python2.7/dist-packages tabulate 0.8.2
------------------------------------------- -------------- ------
which lets you then easily discern which packages you installed with and without sudo
.
A note aside: I’ve noticed that when I install a packet once via sudo
and once without, one takes precedence so that the other one isn’t being listed (only one location is shown). I believe that only the one in the local directory is then listed. This could be improved.
There are many way to skin a cat.
-
The most simple way is to use the pydoc
function directly from the shell with:
pydoc modules
-
But for more information use the tool called pip-date that also tell you the installation dates.
pip install pip-date
I needed to find the specific version of packages available by default in AWS Lambda. I did so with a mashup of ideas from this page. I’m sharing it for posterity.
import pkgutil
__version__ = '0.1.1'
def get_ver(name):
try:
return str(__import__(name).__version__)
except:
return None
def lambda_handler(event, context):
return {
'statusCode': 200,
'body': [{
'path': m.module_finder.path,
'name': m.name,
'version': get_ver(m.name),
} for m in list(pkgutil.iter_modules())
#if m.module_finder.path == "/var/runtime" # Uncomment this if you only care about a certain path
],
}
What I discovered is that the provided boto3 library was way out of date and it wasn’t my fault that my code was failing. I just needed to add boto3 and botocore to my project. But without this I would have been banging my head thinking my code was bad.
{
"statusCode": 200,
"body": [
{
"path": "/var/task",
"name": "lambda_function",
"version": "0.1.1"
},
{
"path": "/var/runtime",
"name": "bootstrap",
"version": null
},
{
"path": "/var/runtime",
"name": "boto3",
"version": "1.9.42"
},
{
"path": "/var/runtime",
"name": "botocore",
"version": "1.12.42"
},
{
"path": "/var/runtime",
"name": "dateutil",
"version": "2.7.5"
},
{
"path": "/var/runtime",
"name": "docutils",
"version": "0.14"
},
{
"path": "/var/runtime",
"name": "jmespath",
"version": "0.9.3"
},
{
"path": "/var/runtime",
"name": "lambda_runtime_client",
"version": null
},
{
"path": "/var/runtime",
"name": "lambda_runtime_exception",
"version": null
},
{
"path": "/var/runtime",
"name": "lambda_runtime_marshaller",
"version": null
},
{
"path": "/var/runtime",
"name": "s3transfer",
"version": "0.1.13"
},
{
"path": "/var/runtime",
"name": "six",
"version": "1.11.0"
},
{
"path": "/var/runtime",
"name": "test_bootstrap",
"version": null
},
{
"path": "/var/runtime",
"name": "test_lambda_runtime_client",
"version": null
},
{
"path": "/var/runtime",
"name": "test_lambda_runtime_marshaller",
"version": null
},
{
"path": "/var/runtime",
"name": "urllib3",
"version": "1.24.1"
},
{
"path": "/var/lang/lib/python3.7",
"name": "__future__",
"version": null
},
...
What I discovered was also different from what they officially publish. At the time of writing this:
- Operating system – Amazon Linux
- AMI – amzn-ami-hvm-2017.03.1.20170812-x86_64-gp2
- Linux kernel – 4.14.77-70.59.amzn1.x86_64
- AWS SDK for JavaScript – 2.290.0
- SDK for Python (Boto 3) – 3-1.7.74 botocore-1.10.74
Try these
pip list
or
pip freeze
For anyone wondering how to call pip list
from a Python program you can use the following:
import pip
pip.main(['list]) # this will print all the packages
This will help
In terminal or IPython, type:
help('modules')
then
In [1]: import #import press-TAB
Display all 631 possibilities? (y or n)
ANSI audiodev markupbase
AptUrl audioop markupsafe
ArgImagePlugin avahi marshal
BaseHTTPServer axi math
Bastion base64 md5
BdfFontFile bdb mhlib
BmpImagePlugin binascii mimetools
BufrStubImagePlugin binhex mimetypes
CDDB bisect mimify
CDROM bonobo mmap
CGIHTTPServer brlapi mmkeys
Canvas bsddb modulefinder
CommandNotFound butterfly multifile
ConfigParser bz2 multiprocessing
ContainerIO cPickle musicbrainz2
Cookie cProfile mutagen
Crypto cStringIO mutex
CurImagePlugin cairo mx
DLFCN calendar netrc
DcxImagePlugin cdrom new
Dialog cgi nis
DiscID cgitb nntplib
DistUpgrade checkbox ntpath
Installation
pip install pkgutil
Code
import pkgutil
for i in pkgutil.iter_modules(None): # returns a tuple (path, package_name, ispkg_flag)
print(i[1]) #or you can append it to a list
Sample Output:
multiprocessing
netrc
nntplib
ntpath
nturl2path
numbers
opcode
pickle
pickletools
pipes
pkgutil
Here is a python code solution that will return a list of modules installed. One can easily modify the code to include version numbers.
import subprocess
import sys
from pprint import pprint
installed_packages = reqs = subprocess.check_output([sys.executable, '-m', 'pip', 'freeze']).decode('utf-8')
installed_packages = installed_packages.split('rn')
installed_packages = [pkg.split('==')[0] for pkg in installed_packages if pkg != '']
pprint(installed_packages)
Works Regardless of Pip Version
Run the following in your python editor or IPython:
import pkg_resources
installed_packages = {d.project_name: d.version for d in pkg_resources.working_set}
print(installed_packages)
Read other answers and pulled together this combo, which is quickest and easiest inside Python.
Find the specific Packages
Conveniently you can then get items from your dict easily, i.e.
installed_packages['pandas']
>> ‘1.16.4’
Using Pip List Well
!pip list
will run inside your jupyter notebook if working there, simplifying the ‘quick check’
Combine with other utilities like grep(if you have installed)
pip list | grep pandas
will get you your current pandas version for example
If none of the above seem to help, in my environment was broken from a system upgrade and I could not upgrade pip. While it won’t give you an accurate list you can get an idea of which libraries were installed simply by looking inside your env>lib>python(version here)>site-packages> . Here you will get a good indication of modules installed.
pip install pip-chill
pip-chill
pip3 freeze
run this command globally – will show you all the global installations
run this command on the virtual environment – will show you all the local installations
I’m comparing five methods to retrieve installed "modules", all of which I’ve seen in this thread
iter_modules
help("modules")
builtin_module_names
pip list
working_set
Includes distributions
❌
❌
❌
✔️
✔️
Includes modules (No built-in)
✔️
✔️
❌
❌
❌
Includes built-in modules
❌
✔️
✔️
❌
❌
Includes frozen
✔️
✔️
❌
❌
❌
Includes venv
✔️
✔️
❌
✔️
✔️
Includes global
✔️
✔️
❌
✔️
✔️
Includes editable installs
✔️
✔️
❌
✔️
✔️
Includes PyCharm helpers
✔️
❌
❌
❌
❌
Lowers capital letters
❌
❌
❌
❌
✔️
Time taken (665 modules total)
53.7 msec
1.03 sec
577 nsec
284 msec
36.2 usec
Summary
pip list
and working_set
are for distributions, not modules.
iter_modules
and help("modules")
are very similar, the biggest difference is that iter_modules
doesn’t include built-in.
pip list
and working_set
are very similar, only difference is that working_set
lowers all capital letters.
- Built-in modules are only included by
help("modules")
and builtin_module_names
.
Related caveats
- Distributions, packages, and modules often have identical names making it easy to mistake one for the other.
importlib.util.find_spec
is for modules and is case-sensitive.
sys.modules
only lists imported modules.
Distributions
I’m saying distribution instead of package because I think it will reduce misunderstandings. A distribution/package can have multiple packages/modules inside it.
An installed distribution is not always importable by the same name. For example pip install Pillow
is imported with import PIL
. Sometimes a distribution even makes multiple modules importable.
Methods (Each column in order)
iter_modules
import pkgutil
{module.name for module in pkgutil.iter_modules()}
help("modules") (Only prints in terminal)
help("modules")
builtin_module_names
import sys
set(sys.builtin_module_names)
pip list (Only prints in terminal)
pip list
in terminal
working_set
import pkg_resources
{pkg.key for pkg in pkg_resources.working_set}
Conclusion
- For terminal I recommend
help("modules")
or python -c "help('modules')"
.
- For programmatically I recommend
iter_modules
+ builtin_module_names
.
- This answer in this thread
- It is however very convoluted with information, see my minimal example below.
import sys
import pkgutil
def get_installed_modules_names():
iter_modules = {module.name for module in pkgutil.iter_modules()}
builtin = sys.builtin_module_names
return set.union(iter_modules, builtin)
How do I get a list of Python modules installed on my computer?
From the shell
ls site-packages
If that’s not helpful, you can do this.
import sys
import os
for p in sys.path:
print os.listdir( p )
And see what that produces.
help('modules')
in a Python shell/prompt.
Now, these methods I tried myself, and I got exactly what was advertised: All the modules.
Alas, really you don’t care much about the stdlib, you know what you get with a python install.
Really, I want the stuff that I installed.
What actually, surprisingly, worked just fine was:
pip freeze
Which returned:
Fabric==0.9.3
apache-libcloud==0.4.0
bzr==2.3b4
distribute==0.6.14
docutils==0.7
greenlet==0.3.1
ipython==0.10.1
iterpipes==0.4
libxml2-python==2.6.21
I say “surprisingly” because the package install tool is the exact place one would expect to find this functionality, although not under the name ‘freeze’ but python packaging is so weird, that I am flabbergasted that this tool makes sense. Pip 0.8.2, Python 2.7.
-
In
ipython
you can type “import
Tab“. -
In the standard Python interpreter, you can type “
help('modules')
“. -
At the command-line, you can use
pydoc
modules
. -
In a script, call
pkgutil.iter_modules()
.
Very simple searching using pkgutil.iter_modules
from pkgutil import iter_modules
a=iter_modules()
while True:
try: x=a.next()
except: break
if 'searchstr' in x[1]: print x[1]
I just use this to see currently used modules:
import sys as s
s.modules.keys()
which shows all modules running on your python.
For all built-in modules use:
s.modules
Which is a dict containing all modules and import objects.
I ran into a custom installed python 2.7 on OS X. It required X11 to list modules installed (both using help and pydoc).
To be able to list all modules without installing X11 I ran pydoc as http-server, i.e.:
pydoc -p 12345
Then it’s possible to direct Safari to http://localhost:12345/
to see all modules.
Aside from using pip freeze
I have been installing yolk in my virtual environments.
Since pip version 1.3, you’ve got access to:
pip list
Which seems to be syntactic sugar for “pip freeze”. It will list all of the modules particular to your installation or virtualenv, along with their version numbers. Unfortunately it does not display the current version number of any module, nor does it wash your dishes or shine your shoes.
In normal shell just use
pydoc modules
In case you have an anaconda python distribution installed, you could also use
$conda list
in addition to solutions described above.
If we need to list the installed packages in the Python shell, we can use the help
command as follows
>>> help('modules package')
- to get all available modules, run
sys.modules
- to get all installed modules (read: installed by
pip
), you may look atpip.get_installed_distributions()
For the second purpose, example code:
import pip
for package in pip.get_installed_distributions():
name = package.project_name # SQLAlchemy, Django, Flask-OAuthlib
key = package.key # sqlalchemy, django, flask-oauthlib
module_name = package._get_metadata("top_level.txt") # sqlalchemy, django, flask_oauthlib
location = package.location # virtualenv lib directory etc.
version = package.version # version number
Solution
Do not use with pip > 10.0!
My 50 cents for getting a pip freeze
-like list from a Python script:
import pip
installed_packages = pip.get_installed_distributions()
installed_packages_list = sorted(["%s==%s" % (i.key, i.version)
for i in installed_packages])
print(installed_packages_list)
As a (too long) one liner:
sorted(["%s==%s" % (i.key, i.version) for i in pip.get_installed_distributions()])
Giving:
['behave==1.2.4', 'enum34==1.0', 'flask==0.10.1', 'itsdangerous==0.24',
'jinja2==2.7.2', 'jsonschema==2.3.0', 'markupsafe==0.23', 'nose==1.3.3',
'parse-type==0.3.4', 'parse==1.6.4', 'prettytable==0.7.2', 'requests==2.3.0',
'six==1.6.1', 'vioozer-metadata==0.1', 'vioozer-users-server==0.1',
'werkzeug==0.9.4']
Scope
This solution applies to the system scope or to a virtual environment scope, and covers packages installed by setuptools
, pip
and (god forbid) easy_install
.
My use case
I added the result of this call to my flask server, so when I call it with http://example.com/exampleServer/environment
I get the list of packages installed on the server’s virtualenv. It makes debugging a whole lot easier.
Caveats
I have noticed a strange behaviour of this technique – when the Python interpreter is invoked in the same directory as a setup.py
file, it does not list the package installed by setup.py
.
Steps to reproduce:
Create a virtual environment
$ cd /tmp
$ virtualenv test_env
New python executable in test_env/bin/python
Installing setuptools, pip...done.
$ source test_env/bin/activate
(test_env) $
Clone a git repo with setup.py
(test_env) $ git clone https://github.com/behave/behave.git
Cloning into 'behave'...
remote: Reusing existing pack: 4350, done.
remote: Total 4350 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (4350/4350), 1.85 MiB | 418.00 KiB/s, done.
Resolving deltas: 100% (2388/2388), done.
Checking connectivity... done.
We have behave’s setup.py
in /tmp/behave
:
(test_env) $ ls /tmp/behave/setup.py
/tmp/behave/setup.py
Install the python package from the git repo
(test_env) $ cd /tmp/behave && pip install .
running install
...
Installed /private/tmp/test_env/lib/python2.7/site-packages/enum34-1.0-py2.7.egg
Finished processing dependencies for behave==1.2.5a1
If we run the aforementioned solution from /tmp
>>> import pip
>>> sorted(["%s==%s" % (i.key, i.version) for i in pip.get_installed_distributions()])
['behave==1.2.5a1', 'enum34==1.0', 'parse-type==0.3.4', 'parse==1.6.4', 'six==1.6.1']
>>> import os
>>> os.getcwd()
'/private/tmp'
If we run the aforementioned solution from /tmp/behave
>>> import pip
>>> sorted(["%s==%s" % (i.key, i.version) for i in pip.get_installed_distributions()])
['enum34==1.0', 'parse-type==0.3.4', 'parse==1.6.4', 'six==1.6.1']
>>> import os
>>> os.getcwd()
'/private/tmp/behave'
behave==1.2.5a1
is missing from the second example, because the working directory contains behave
‘s setup.py
file.
I could not find any reference to this issue in the documentation. Perhaps I shall open a bug for it.
on windows, Enter this in cmd
c:pythonlibs>python -m pip freeze
pip freeze does it all finding packages however one can simply write the following command to list all paths where python packages are.
>>> import site; site.getsitepackages()
['/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages']
This solution is primary based on modules importlib
and pkgutil
and work with CPython 3.4 and CPython 3.5, but has no support for the CPython 2.
Explanation
sys.builtin_module_names
– names all built-in modules (look my answer here)pkgutil.iter_modules()
– returns an information about all available modulesimportlib.util.find_spec()
– returns an information about importing module, if existsBuiltinImporter
– an importer for built-in modules (docs)SourceFileLoader
– an importer for a standard Python module (by default has extension *.py) (docs)ExtensionFileLoader
– an importer for modules as shared library (written on the C or C++)
Full code
import sys
import os
import shutil
import pkgutil
import importlib
import collections
if sys.version_info.major == 2:
raise NotImplementedError('CPython 2 is not supported yet')
def main():
# name this file (module)
this_module_name = os.path.basename(__file__).rsplit('.')[0]
# dict for loaders with their modules
loaders = collections.OrderedDict()
# names`s of build-in modules
for module_name in sys.builtin_module_names:
# find an information about a module by name
module = importlib.util.find_spec(module_name)
# add a key about a loader in the dict, if not exists yet
if module.loader not in loaders:
loaders[module.loader] = []
# add a name and a location about imported module in the dict
loaders[module.loader].append((module.name, module.origin))
# all available non-build-in modules
for module_name in pkgutil.iter_modules():
# ignore this module
if this_module_name == module_name[1]:
continue
# find an information about a module by name
module = importlib.util.find_spec(module_name[1])
# add a key about a loader in the dict, if not exists yet
loader = type(module.loader)
if loader not in loaders:
loaders[loader] = []
# add a name and a location about imported module in the dict
loaders[loader].append((module.name, module.origin))
# pretty print
line = '-' * shutil.get_terminal_size().columns
for loader, modules in loaders.items():
print('{0}n{1}: {2}n{0}'.format(line, len(modules), loader))
for module in modules:
print('{0:30} | {1}'.format(module[0], module[1]))
if __name__ == '__main__':
main()
Usage
For the CPython3.5 (truncated)
$ python3.5 python_modules_info.py
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
30: <class '_frozen_importlib.BuiltinImporter'>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
_ast | built-in
_codecs | built-in
_collections | built-in
_functools | built-in
_imp | None
_io | built-in
_locale | built-in
_operator | built-in
_signal | built-in
_sre | built-in
_stat | built-in
_string | built-in
_symtable | built-in
_thread | built-in
(****************************truncated*******************************)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
227: <class '_frozen_importlib_external.SourceFileLoader'>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__future__ | /usr/local/lib/python3.5/__future__.py
_bootlocale | /usr/local/lib/python3.5/_bootlocale.py
_collections_abc | /usr/local/lib/python3.5/_collections_abc.py
_compat_pickle | /usr/local/lib/python3.5/_compat_pickle.py
_compression | /usr/local/lib/python3.5/_compression.py
_dummy_thread | /usr/local/lib/python3.5/_dummy_thread.py
_markupbase | /usr/local/lib/python3.5/_markupbase.py
_osx_support | /usr/local/lib/python3.5/_osx_support.py
_pydecimal | /usr/local/lib/python3.5/_pydecimal.py
_pyio | /usr/local/lib/python3.5/_pyio.py
_sitebuiltins | /usr/local/lib/python3.5/_sitebuiltins.py
(****************************truncated*******************************)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
64: <class '_frozen_importlib_external.ExtensionFileLoader'>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
_bisect | /usr/local/lib/python3.5/lib-dynload/_bisect.cpython-35m-x86_64-linux-gnu.so
_bz2 | /usr/local/lib/python3.5/lib-dynload/_bz2.cpython-35m-x86_64-linux-gnu.so
_codecs_cn | /usr/local/lib/python3.5/lib-dynload/_codecs_cn.cpython-35m-x86_64-linux-gnu.so
_codecs_hk | /usr/local/lib/python3.5/lib-dynload/_codecs_hk.cpython-35m-x86_64-linux-gnu.so
_codecs_iso2022 | /usr/local/lib/python3.5/lib-dynload/_codecs_iso2022.cpython-35m-x86_64-linux-gnu.so
(****************************truncated*******************************)
For the CPython3.4 (truncated)
$ python3.4 python_modules_info.py
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
54: <class '_frozen_importlib.BuiltinImporter'>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
_ast | built-in
_bisect | built-in
_codecs | built-in
_collections | built-in
_datetime | built-in
_elementtree | built-in
_functools | built-in
_heapq | built-in
_imp | None
_io | built-in
_locale | built-in
_md5 | built-in
_operator | built-in
_pickle | built-in
_posixsubprocess | built-in
_random | built-in
(****************************truncated*******************************)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
246: <class '_frozen_importlib.SourceFileLoader'>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__future__ | /usr/lib/python3.4/__future__.py
_bootlocale | /usr/lib/python3.4/_bootlocale.py
_collections_abc | /usr/lib/python3.4/_collections_abc.py
_compat_pickle | /usr/lib/python3.4/_compat_pickle.py
_dummy_thread | /usr/lib/python3.4/_dummy_thread.py
_markupbase | /usr/lib/python3.4/_markupbase.py
_osx_support | /usr/lib/python3.4/_osx_support.py
_pyio | /usr/lib/python3.4/_pyio.py
(****************************truncated*******************************)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
44: <class '_frozen_importlib.ExtensionFileLoader'>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
_bz2 | /usr/lib/python3.4/lib-dynload/_bz2.cpython-34m-x86_64-linux-gnu.so
_codecs_cn | /usr/lib/python3.4/lib-dynload/_codecs_cn.cpython-34m-x86_64-linux-gnu.so
_codecs_hk | /usr/lib/python3.4/lib-dynload/_codecs_hk.cpython-34m-x86_64-linux-gnu.so
_codecs_iso2022 | /usr/lib/python3.4/lib-dynload/_codecs_iso2022.cpython-34m-x86_64-linux-gnu.so
_codecs_jp | /usr/lib/python3.4/lib-dynload/_codecs_jp.cpython-34m-x86_64-linux-gnu.so
_codecs_kr | /usr/lib/python3.4/lib-dynload/_codecs_kr.cpython-34m-x86_64-linux-gnu.so
_codecs_tw | /usr/lib/python3.4/lib-dynload/_codecs_tw.cpython-34m-x86_64-linux-gnu.so
_crypt | /usr/lib/python3.4/lib-dynload/_crypt.cpython-34m-x86_64-linux-gnu.so
(****************************truncated*******************************)
As of pip 10, the accepted answer will no longer work. The development team has removed access to the get_installed_distributions
routine. There is an alternate function in the setuptools
for doing the same thing. Here is an alternate version that works with pip 10:
import pkg_resources
installed_packages = pkg_resources.working_set
installed_packages_list = sorted(["%s==%s" % (i.key, i.version)
for i in installed_packages])
print(installed_packages_list)
Please let me know if it will or won’t work in previous versions of pip, too.
I normally use pip list
to get a list of packages (with version).
This works in a virtual environment too, of course. To show what’s installed in only the virtual environment (not global packages), use pip list --local
.
Here’s documentation showing all the available pip list
options, with several good examples.
There are many ideas, initially I am pondering on these two:
pip
cons: not always installed
help(‘modules’)
cons: output to console; with broken modules (see ubuntu…) can segfault
I need an easy approach, using basic libraries and compatible with old python 2.x
And I see the light: listmodules.py
Hidden in the documentation source directory in 2.5 is a small script that lists all available modules for a Python installation.
Pros:
uses only imp, sys, os, re, time
designed to run on Python 1.5.2 and newer
the source code is really compact, so you can easy tinkering with it, for example to pass an exception list of buggy modules (don’t try to import them)
Warning: Adam Matan discourages this use in pip > 10.0. Also, read @sinoroc’s comment below
This was inspired by Adam Matan’s answer (the accepted one):
import tabulate
try:
from pip import get_installed_distributions
except:
from pip._internal.utils.misc import get_installed_distributions
tabpackages = []
for _, package in sorted([('%s %s' % (i.location, i.key), i) for i in get_installed_distributions()]):
tabpackages.append([package.location, package.key, package.version])
print(tabulate.tabulate(tabpackages))
which then prints out a table in the form of
19:33 [email protected] [iot-wifi-2] ~/python$ python installed_packages.py
------------------------------------------- -------------- ------
/home/pi/.local/lib/python2.7/site-packages enum-compat 0.0.2
/home/pi/.local/lib/python2.7/site-packages enum34 1.1.6
/home/pi/.local/lib/python2.7/site-packages pexpect 4.2.1
/home/pi/.local/lib/python2.7/site-packages ptyprocess 0.5.2
/home/pi/.local/lib/python2.7/site-packages pygatt 3.2.0
/home/pi/.local/lib/python2.7/site-packages pyserial 3.4
/usr/local/lib/python2.7/dist-packages bluepy 1.1.1
/usr/local/lib/python2.7/dist-packages click 6.7
/usr/local/lib/python2.7/dist-packages click-datetime 0.2
/usr/local/lib/python2.7/dist-packages construct 2.8.21
/usr/local/lib/python2.7/dist-packages pyaudio 0.2.11
/usr/local/lib/python2.7/dist-packages tabulate 0.8.2
------------------------------------------- -------------- ------
which lets you then easily discern which packages you installed with and without sudo
.
A note aside: I’ve noticed that when I install a packet once via sudo
and once without, one takes precedence so that the other one isn’t being listed (only one location is shown). I believe that only the one in the local directory is then listed. This could be improved.
There are many way to skin a cat.
-
The most simple way is to use the
pydoc
function directly from the shell with:
pydoc modules
-
But for more information use the tool called pip-date that also tell you the installation dates.
pip install pip-date
I needed to find the specific version of packages available by default in AWS Lambda. I did so with a mashup of ideas from this page. I’m sharing it for posterity.
import pkgutil
__version__ = '0.1.1'
def get_ver(name):
try:
return str(__import__(name).__version__)
except:
return None
def lambda_handler(event, context):
return {
'statusCode': 200,
'body': [{
'path': m.module_finder.path,
'name': m.name,
'version': get_ver(m.name),
} for m in list(pkgutil.iter_modules())
#if m.module_finder.path == "/var/runtime" # Uncomment this if you only care about a certain path
],
}
What I discovered is that the provided boto3 library was way out of date and it wasn’t my fault that my code was failing. I just needed to add boto3 and botocore to my project. But without this I would have been banging my head thinking my code was bad.
{
"statusCode": 200,
"body": [
{
"path": "/var/task",
"name": "lambda_function",
"version": "0.1.1"
},
{
"path": "/var/runtime",
"name": "bootstrap",
"version": null
},
{
"path": "/var/runtime",
"name": "boto3",
"version": "1.9.42"
},
{
"path": "/var/runtime",
"name": "botocore",
"version": "1.12.42"
},
{
"path": "/var/runtime",
"name": "dateutil",
"version": "2.7.5"
},
{
"path": "/var/runtime",
"name": "docutils",
"version": "0.14"
},
{
"path": "/var/runtime",
"name": "jmespath",
"version": "0.9.3"
},
{
"path": "/var/runtime",
"name": "lambda_runtime_client",
"version": null
},
{
"path": "/var/runtime",
"name": "lambda_runtime_exception",
"version": null
},
{
"path": "/var/runtime",
"name": "lambda_runtime_marshaller",
"version": null
},
{
"path": "/var/runtime",
"name": "s3transfer",
"version": "0.1.13"
},
{
"path": "/var/runtime",
"name": "six",
"version": "1.11.0"
},
{
"path": "/var/runtime",
"name": "test_bootstrap",
"version": null
},
{
"path": "/var/runtime",
"name": "test_lambda_runtime_client",
"version": null
},
{
"path": "/var/runtime",
"name": "test_lambda_runtime_marshaller",
"version": null
},
{
"path": "/var/runtime",
"name": "urllib3",
"version": "1.24.1"
},
{
"path": "/var/lang/lib/python3.7",
"name": "__future__",
"version": null
},
...
What I discovered was also different from what they officially publish. At the time of writing this:
- Operating system – Amazon Linux
- AMI – amzn-ami-hvm-2017.03.1.20170812-x86_64-gp2
- Linux kernel – 4.14.77-70.59.amzn1.x86_64
- AWS SDK for JavaScript – 2.290.0
- SDK for Python (Boto 3) – 3-1.7.74 botocore-1.10.74
Try these
pip list
or
pip freeze
For anyone wondering how to call pip list
from a Python program you can use the following:
import pip
pip.main(['list]) # this will print all the packages
This will help
In terminal or IPython, type:
help('modules')
then
In [1]: import #import press-TAB
Display all 631 possibilities? (y or n)
ANSI audiodev markupbase
AptUrl audioop markupsafe
ArgImagePlugin avahi marshal
BaseHTTPServer axi math
Bastion base64 md5
BdfFontFile bdb mhlib
BmpImagePlugin binascii mimetools
BufrStubImagePlugin binhex mimetypes
CDDB bisect mimify
CDROM bonobo mmap
CGIHTTPServer brlapi mmkeys
Canvas bsddb modulefinder
CommandNotFound butterfly multifile
ConfigParser bz2 multiprocessing
ContainerIO cPickle musicbrainz2
Cookie cProfile mutagen
Crypto cStringIO mutex
CurImagePlugin cairo mx
DLFCN calendar netrc
DcxImagePlugin cdrom new
Dialog cgi nis
DiscID cgitb nntplib
DistUpgrade checkbox ntpath
Installation
pip install pkgutil
Code
import pkgutil
for i in pkgutil.iter_modules(None): # returns a tuple (path, package_name, ispkg_flag)
print(i[1]) #or you can append it to a list
Sample Output:
multiprocessing
netrc
nntplib
ntpath
nturl2path
numbers
opcode
pickle
pickletools
pipes
pkgutil
Here is a python code solution that will return a list of modules installed. One can easily modify the code to include version numbers.
import subprocess
import sys
from pprint import pprint
installed_packages = reqs = subprocess.check_output([sys.executable, '-m', 'pip', 'freeze']).decode('utf-8')
installed_packages = installed_packages.split('rn')
installed_packages = [pkg.split('==')[0] for pkg in installed_packages if pkg != '']
pprint(installed_packages)
Works Regardless of Pip Version
Run the following in your python editor or IPython:
import pkg_resources
installed_packages = {d.project_name: d.version for d in pkg_resources.working_set}
print(installed_packages)
Read other answers and pulled together this combo, which is quickest and easiest inside Python.
Find the specific Packages
Conveniently you can then get items from your dict easily, i.e.
installed_packages['pandas']
>> ‘1.16.4’
Using Pip List Well
!pip list
will run inside your jupyter notebook if working there, simplifying the ‘quick check’
Combine with other utilities like grep(if you have installed)
pip list | grep pandas
will get you your current pandas version for example
If none of the above seem to help, in my environment was broken from a system upgrade and I could not upgrade pip. While it won’t give you an accurate list you can get an idea of which libraries were installed simply by looking inside your env>lib>python(version here)>site-packages> . Here you will get a good indication of modules installed.
pip install pip-chill
pip-chill
pip3 freeze
run this command globally – will show you all the global installations
run this command on the virtual environment – will show you all the local installations
I’m comparing five methods to retrieve installed "modules", all of which I’ve seen in this thread
iter_modules | help("modules") | builtin_module_names | pip list | working_set | |
---|---|---|---|---|---|
Includes distributions | ❌ | ❌ | ❌ | ✔️ | ✔️ |
Includes modules (No built-in) | ✔️ | ✔️ | ❌ | ❌ | ❌ |
Includes built-in modules | ❌ | ✔️ | ✔️ | ❌ | ❌ |
Includes frozen | ✔️ | ✔️ | ❌ | ❌ | ❌ |
Includes venv | ✔️ | ✔️ | ❌ | ✔️ | ✔️ |
Includes global | ✔️ | ✔️ | ❌ | ✔️ | ✔️ |
Includes editable installs | ✔️ | ✔️ | ❌ | ✔️ | ✔️ |
Includes PyCharm helpers | ✔️ | ❌ | ❌ | ❌ | ❌ |
Lowers capital letters | ❌ | ❌ | ❌ | ❌ | ✔️ |
Time taken (665 modules total) | 53.7 msec | 1.03 sec | 577 nsec | 284 msec | 36.2 usec |
Summary
pip list
andworking_set
are for distributions, not modules.iter_modules
andhelp("modules")
are very similar, the biggest difference is thatiter_modules
doesn’t include built-in.pip list
andworking_set
are very similar, only difference is thatworking_set
lowers all capital letters.- Built-in modules are only included by
help("modules")
andbuiltin_module_names
.
Related caveats
- Distributions, packages, and modules often have identical names making it easy to mistake one for the other.
importlib.util.find_spec
is for modules and is case-sensitive.sys.modules
only lists imported modules.
Distributions
I’m saying distribution instead of package because I think it will reduce misunderstandings. A distribution/package can have multiple packages/modules inside it.
An installed distribution is not always importable by the same name. For example pip install Pillow
is imported with import PIL
. Sometimes a distribution even makes multiple modules importable.
Methods (Each column in order)
iter_modules
import pkgutil
{module.name for module in pkgutil.iter_modules()}
help("modules") (Only prints in terminal)
help("modules")
builtin_module_names
import sys
set(sys.builtin_module_names)
pip list (Only prints in terminal)
pip list
in terminal
working_set
import pkg_resources
{pkg.key for pkg in pkg_resources.working_set}
Conclusion
- For terminal I recommend
help("modules")
orpython -c "help('modules')"
. - For programmatically I recommend
iter_modules
+builtin_module_names
.- This answer in this thread
- It is however very convoluted with information, see my minimal example below.
- This answer in this thread
import sys
import pkgutil
def get_installed_modules_names():
iter_modules = {module.name for module in pkgutil.iter_modules()}
builtin = sys.builtin_module_names
return set.union(iter_modules, builtin)