Package for listing version of packages used in a Jupyter notebook
Question:
I seem to remember there is a package that printed the versions and relevant information about Python packages used in a Jupyter notebook so the results in it were reproducible. But I cannot remember the name of the package. Can any of you point me in the right direction?
Thanks in advance!
Answers:
This gets all the installed packages
import pip #needed to use the pip functions
for i in pip.get_installed_distributions(local_only=True):
print(i)
To get the list of packages from current notebook
import types
def imports():
for name, val in globals().items():
if isinstance(val, types.ModuleType):
yield val.__name__
list(imports())
I’ve cobbled this answer by combining the two solutions already provided. I ultimately wanted to generate a requirements.txt type file, for easy use with the awesome Binder website. Obviously, I don’t want to pip freeze
my whole system but I also don’t want to create separate virtual environments for every notebook (which is ultimately where my problem stems from).
This outputs a nicely formatted requirements.txt type string and handles some of the intricacies involved when you use import from
rather than just import
.
Get locally imported modules from current notebook
import pkg_resources
import types
def get_imports():
for name, val in globals().items():
if isinstance(val, types.ModuleType):
# Split ensures you get root package,
# not just imported function
name = val.__name__.split(".")[0]
elif isinstance(val, type):
name = val.__module__.split(".")[0]
# Some packages are weird and have different
# imported names vs. system/pip names. Unfortunately,
# there is no systematic way to get pip names from
# a package's imported name. You'll have to add
# exceptions to this list manually!
poorly_named_packages = {
"PIL": "Pillow",
"sklearn": "scikit-learn"
}
if name in poorly_named_packages.keys():
name = poorly_named_packages[name]
yield name
imports = list(set(get_imports()))
# The only way I found to get the version of the root package
# from only the name of the package is to cross-check the names
# of installed packages vs. imported packages
requirements = []
for m in pkg_resources.working_set:
if m.project_name in imports and m.project_name!="pip":
requirements.append((m.project_name, m.version))
for r in requirements:
print("{}=={}".format(*r))
Sample output:
scipy==0.19.0
requests==2.18.1
Pillow==5.0.0
numpy==1.13.0
matplotlib==2.0.2
EDITED 2018-04-21: pip version 10 stopped supporting the .get_installed_distributions()
method. Using pkg_resources.working_set
instead.
I think the pip
based approaches are superior in terms of functionality, but it may be possible the OP was trying to recall the name of the version_information
extension for Jupyter: https://pypi.org/project/version_information/
I made some improvements to @Alex P. Miller’s answer so that
(sorry I don’t have enough reps to “comment” directly on his answer)
- Automatically works with module names where case sensitivity was causing problems
- Also lists modules without version numbers as “unknown” to make it clear it couldn’t find a match.
- also lists built in modules if it can detect it.
# show versions of packages
# adopted from https://stackoverflow.com/questions/40428931/package-for-listing-version-of-packages-used-in-a-jupyter-notebook
def get_imports():
for name, val in globals().items():
if isinstance(val, types.ModuleType):
# Split ensures you get root package,
# not just imported function
name = val.__name__.split(".")[0]
elif isinstance(val, type):
name = val.__module__.split(".")[0]
# Some packages are weird and have different
# imported names vs. system/pip names. Unfortunately,
# there is no systematic way to get pip names from
# a package's imported name. You'll have to add
# exceptions to this list manually!
poorly_named_packages = {
"sklearn": "scikit-learn"
}
if name in poorly_named_packages.keys():
name = poorly_named_packages[name]
yield name.lower()
imports = list(set(get_imports()))
# The only way I found to get the version of the root package
# from only the name of the package is to cross-check the names
# of installed packages vs. imported packages
modules = []
for m in sys.builtin_module_names:
if m.lower() in imports and m !='builtins':
modules.append((m,'Python BuiltIn'))
imports.remove(m.lower())
for m in pkg_resources.working_set:
if m.project_name.lower() in imports and m.project_name!="pip":
modules.append((m.project_name, m.version))
imports.remove(m.project_name.lower())
for m in sys.modules:
if m.lower() in imports and m !='builtins':
modules.append((m,'unknown'))
# print('System=='+platform.system()+' '+platform.release()+'; Version=='+platform.version())
for r in modules:
print("{}=={}".format(*r))
Another solution (based on Vivek’s answer):
import types
def imports():
for name, val in globals().items():
if isinstance(val, types.ModuleType):
yield val.__name__
excludes = ['builtins', 'types', 'sys']
imported_modules = [module for module in imports() if module not in excludes]
clean_modules = []
for module in imported_modules:
sep = '.' # to handle 'matplotlib.pyplot' cases
rest = module.split(sep, 1)[0]
clean_modules.append(rest)
changed_imported_modules = list(set(clean_modules)) # drop duplicates
pip_modules = !pip freeze # you could also use `!conda list` with anaconda
for module in pip_modules:
name, version = module.split('==')
if name in changed_imported_modules:
print(name + 't' + version)
Sample output:
astropy 3.2.1
matplotlib 3.1.0
numpy 1.16.4
pandas 0.25.0
Adapted from gafortiby’s answer: a shorter version to list only explicit list of packages. I found this suitable to memorize versions of the most important packages used in a jupyter notebook (for other readers or future use):
import pkg_resources
# list packages to be checked
root_packages = [
'geoviews', 'geopandas', 'pandas', 'numpy',
'matplotlib', 'shapely', 'cartopy', 'holoviews',
'mapclassify', 'fiona', 'bokeh']
# print versions, but check if package is imported first
for m in pkg_resources.working_set:
if m.project_name.lower() in root_packages:
print(f"{m.project_name}=={m.version}")
Output:
Shapely==1.7.0
pandas==1.0.1
numpy==1.18.1
matplotlib==3.1.3
mapclassify==2.2.0
holoviews==1.12.7
geoviews==1.6.6
geopandas==0.6.3
Fiona==1.8.13
Cartopy==0.17.0
bokeh==1.4.0
Enhanced version with nicer display:
import pkg_resources
from IPython.display import display
import pandas as pd
root_packages = [
'geoviews', 'geopandas', 'pandas', 'numpy', 'cloudpickle',
'matplotlib', 'shapely', 'cartopy', 'holoviews',
'mapclassify', 'fiona', 'bokeh', 'pyproj', 'ipython',
'jupyterlab']
root_packages.sort(reverse=True)
root_packages_list = []
for m in pkg_resources.working_set:
if m.project_name.lower() in root_packages:
root_packages_list.append([m.project_name, m.version])
display(pd.DataFrame(
root_packages_list,
columns=["package", "version"]
).set_index("package").transpose())
One-liner:
# In[1]:
import pandas as pd
import numpy as np
import tensorflow as tf
print('n'.join(f'{m.__name__}=={m.__version__}' for m in globals().values() if getattr(m, '__version__', None)))
Output:
pandas==1.1.1
numpy==1.19.1
tensorflow==2.2.0
Since these answers are a bit dated, the simpler solutions didn’t work for me and it took some stumbling to find an easy, working solution elsewhere online:
from sinfo import sinfo
sinfo()
I had some problems just doing writing in an empty cell
pip list
But once I ran it in a whole new file, I had no problems at all, and got all the libraries installed in the notebook!
I seem to remember there is a package that printed the versions and relevant information about Python packages used in a Jupyter notebook so the results in it were reproducible. But I cannot remember the name of the package. Can any of you point me in the right direction?
Thanks in advance!
This gets all the installed packages
import pip #needed to use the pip functions
for i in pip.get_installed_distributions(local_only=True):
print(i)
To get the list of packages from current notebook
import types
def imports():
for name, val in globals().items():
if isinstance(val, types.ModuleType):
yield val.__name__
list(imports())
I’ve cobbled this answer by combining the two solutions already provided. I ultimately wanted to generate a requirements.txt type file, for easy use with the awesome Binder website. Obviously, I don’t want to pip freeze
my whole system but I also don’t want to create separate virtual environments for every notebook (which is ultimately where my problem stems from).
This outputs a nicely formatted requirements.txt type string and handles some of the intricacies involved when you use import from
rather than just import
.
Get locally imported modules from current notebook
import pkg_resources
import types
def get_imports():
for name, val in globals().items():
if isinstance(val, types.ModuleType):
# Split ensures you get root package,
# not just imported function
name = val.__name__.split(".")[0]
elif isinstance(val, type):
name = val.__module__.split(".")[0]
# Some packages are weird and have different
# imported names vs. system/pip names. Unfortunately,
# there is no systematic way to get pip names from
# a package's imported name. You'll have to add
# exceptions to this list manually!
poorly_named_packages = {
"PIL": "Pillow",
"sklearn": "scikit-learn"
}
if name in poorly_named_packages.keys():
name = poorly_named_packages[name]
yield name
imports = list(set(get_imports()))
# The only way I found to get the version of the root package
# from only the name of the package is to cross-check the names
# of installed packages vs. imported packages
requirements = []
for m in pkg_resources.working_set:
if m.project_name in imports and m.project_name!="pip":
requirements.append((m.project_name, m.version))
for r in requirements:
print("{}=={}".format(*r))
Sample output:
scipy==0.19.0
requests==2.18.1
Pillow==5.0.0
numpy==1.13.0
matplotlib==2.0.2
EDITED 2018-04-21: pip version 10 stopped supporting the .get_installed_distributions()
method. Using pkg_resources.working_set
instead.
I think the pip
based approaches are superior in terms of functionality, but it may be possible the OP was trying to recall the name of the version_information
extension for Jupyter: https://pypi.org/project/version_information/
I made some improvements to @Alex P. Miller’s answer so that
(sorry I don’t have enough reps to “comment” directly on his answer)
- Automatically works with module names where case sensitivity was causing problems
- Also lists modules without version numbers as “unknown” to make it clear it couldn’t find a match.
- also lists built in modules if it can detect it.
# show versions of packages
# adopted from https://stackoverflow.com/questions/40428931/package-for-listing-version-of-packages-used-in-a-jupyter-notebook
def get_imports():
for name, val in globals().items():
if isinstance(val, types.ModuleType):
# Split ensures you get root package,
# not just imported function
name = val.__name__.split(".")[0]
elif isinstance(val, type):
name = val.__module__.split(".")[0]
# Some packages are weird and have different
# imported names vs. system/pip names. Unfortunately,
# there is no systematic way to get pip names from
# a package's imported name. You'll have to add
# exceptions to this list manually!
poorly_named_packages = {
"sklearn": "scikit-learn"
}
if name in poorly_named_packages.keys():
name = poorly_named_packages[name]
yield name.lower()
imports = list(set(get_imports()))
# The only way I found to get the version of the root package
# from only the name of the package is to cross-check the names
# of installed packages vs. imported packages
modules = []
for m in sys.builtin_module_names:
if m.lower() in imports and m !='builtins':
modules.append((m,'Python BuiltIn'))
imports.remove(m.lower())
for m in pkg_resources.working_set:
if m.project_name.lower() in imports and m.project_name!="pip":
modules.append((m.project_name, m.version))
imports.remove(m.project_name.lower())
for m in sys.modules:
if m.lower() in imports and m !='builtins':
modules.append((m,'unknown'))
# print('System=='+platform.system()+' '+platform.release()+'; Version=='+platform.version())
for r in modules:
print("{}=={}".format(*r))
Another solution (based on Vivek’s answer):
import types
def imports():
for name, val in globals().items():
if isinstance(val, types.ModuleType):
yield val.__name__
excludes = ['builtins', 'types', 'sys']
imported_modules = [module for module in imports() if module not in excludes]
clean_modules = []
for module in imported_modules:
sep = '.' # to handle 'matplotlib.pyplot' cases
rest = module.split(sep, 1)[0]
clean_modules.append(rest)
changed_imported_modules = list(set(clean_modules)) # drop duplicates
pip_modules = !pip freeze # you could also use `!conda list` with anaconda
for module in pip_modules:
name, version = module.split('==')
if name in changed_imported_modules:
print(name + 't' + version)
Sample output:
astropy 3.2.1
matplotlib 3.1.0
numpy 1.16.4
pandas 0.25.0
Adapted from gafortiby’s answer: a shorter version to list only explicit list of packages. I found this suitable to memorize versions of the most important packages used in a jupyter notebook (for other readers or future use):
import pkg_resources
# list packages to be checked
root_packages = [
'geoviews', 'geopandas', 'pandas', 'numpy',
'matplotlib', 'shapely', 'cartopy', 'holoviews',
'mapclassify', 'fiona', 'bokeh']
# print versions, but check if package is imported first
for m in pkg_resources.working_set:
if m.project_name.lower() in root_packages:
print(f"{m.project_name}=={m.version}")
Output:
Shapely==1.7.0
pandas==1.0.1
numpy==1.18.1
matplotlib==3.1.3
mapclassify==2.2.0
holoviews==1.12.7
geoviews==1.6.6
geopandas==0.6.3
Fiona==1.8.13
Cartopy==0.17.0
bokeh==1.4.0
Enhanced version with nicer display:
import pkg_resources
from IPython.display import display
import pandas as pd
root_packages = [
'geoviews', 'geopandas', 'pandas', 'numpy', 'cloudpickle',
'matplotlib', 'shapely', 'cartopy', 'holoviews',
'mapclassify', 'fiona', 'bokeh', 'pyproj', 'ipython',
'jupyterlab']
root_packages.sort(reverse=True)
root_packages_list = []
for m in pkg_resources.working_set:
if m.project_name.lower() in root_packages:
root_packages_list.append([m.project_name, m.version])
display(pd.DataFrame(
root_packages_list,
columns=["package", "version"]
).set_index("package").transpose())
One-liner:
# In[1]:
import pandas as pd
import numpy as np
import tensorflow as tf
print('n'.join(f'{m.__name__}=={m.__version__}' for m in globals().values() if getattr(m, '__version__', None)))
Output:
pandas==1.1.1
numpy==1.19.1
tensorflow==2.2.0
Since these answers are a bit dated, the simpler solutions didn’t work for me and it took some stumbling to find an easy, working solution elsewhere online:
from sinfo import sinfo
sinfo()
I had some problems just doing writing in an empty cell
pip list
But once I ran it in a whole new file, I had no problems at all, and got all the libraries installed in the notebook!