Package for listing version of packages used in a Jupyter notebook

Question:

I seem to remember there is a package that printed the versions and relevant information about Python packages used in a Jupyter notebook so the results in it were reproducible. But I cannot remember the name of the package. Can any of you point me in the right direction?

Thanks in advance!

Asked By: msx

||

Answers:

This gets all the installed packages

import pip #needed to use the pip functions
for i in pip.get_installed_distributions(local_only=True):
    print(i)

To get the list of packages from current notebook

import types
def imports():
    for name, val in globals().items():
        if isinstance(val, types.ModuleType):
            yield val.__name__
list(imports())
Answered By: Vivek Srinivasan

I’ve cobbled this answer by combining the two solutions already provided. I ultimately wanted to generate a requirements.txt type file, for easy use with the awesome Binder website. Obviously, I don’t want to pip freeze my whole system but I also don’t want to create separate virtual environments for every notebook (which is ultimately where my problem stems from).

This outputs a nicely formatted requirements.txt type string and handles some of the intricacies involved when you use import from rather than just import.

Get locally imported modules from current notebook

import pkg_resources
import types
def get_imports():
    for name, val in globals().items():
        if isinstance(val, types.ModuleType):
            # Split ensures you get root package, 
            # not just imported function
            name = val.__name__.split(".")[0]

        elif isinstance(val, type):
            name = val.__module__.split(".")[0]
            
        # Some packages are weird and have different
        # imported names vs. system/pip names. Unfortunately,
        # there is no systematic way to get pip names from
        # a package's imported name. You'll have to add
        # exceptions to this list manually!
        poorly_named_packages = {
            "PIL": "Pillow",
            "sklearn": "scikit-learn"
        }
        if name in poorly_named_packages.keys():
            name = poorly_named_packages[name]
            
        yield name
imports = list(set(get_imports()))

# The only way I found to get the version of the root package
# from only the name of the package is to cross-check the names 
# of installed packages vs. imported packages
requirements = []
for m in pkg_resources.working_set:
    if m.project_name in imports and m.project_name!="pip":
        requirements.append((m.project_name, m.version))

for r in requirements:
    print("{}=={}".format(*r))

Sample output:

scipy==0.19.0
requests==2.18.1
Pillow==5.0.0
numpy==1.13.0
matplotlib==2.0.2

EDITED 2018-04-21: pip version 10 stopped supporting the .get_installed_distributions() method. Using pkg_resources.working_set instead.

Answered By: Alex P. Miller

I think the pip based approaches are superior in terms of functionality, but it may be possible the OP was trying to recall the name of the version_information extension for Jupyter: https://pypi.org/project/version_information/

Answered By: chepyle

I made some improvements to @Alex P. Miller’s answer so that
(sorry I don’t have enough reps to “comment” directly on his answer)

  1. Automatically works with module names where case sensitivity was causing problems
  2. Also lists modules without version numbers as “unknown” to make it clear it couldn’t find a match.
  3. also lists built in modules if it can detect it.
# show versions of packages
# adopted from https://stackoverflow.com/questions/40428931/package-for-listing-version-of-packages-used-in-a-jupyter-notebook

    def get_imports():
        for name, val in globals().items():
            if isinstance(val, types.ModuleType):
                # Split ensures you get root package, 
                # not just imported function
                name = val.__name__.split(".")[0]
            elif isinstance(val, type):
                name = val.__module__.split(".")[0]
            # Some packages are weird and have different
            # imported names vs. system/pip names. Unfortunately,
            # there is no systematic way to get pip names from
            # a package's imported name. You'll have to add
            # exceptions to this list manually!
            poorly_named_packages = {
                "sklearn": "scikit-learn"
            }
            if name in poorly_named_packages.keys():
                name = poorly_named_packages[name]
            yield name.lower()
    imports = list(set(get_imports()))

    # The only way I found to get the version of the root package
    # from only the name of the package is to cross-check the names 
    # of installed packages vs. imported packages
    modules = []
    for m in sys.builtin_module_names:
        if m.lower() in imports and m !='builtins':
            modules.append((m,'Python BuiltIn'))
            imports.remove(m.lower())

    for m in pkg_resources.working_set:
        if m.project_name.lower() in imports and m.project_name!="pip":
            modules.append((m.project_name, m.version))
            imports.remove(m.project_name.lower())

    for m in sys.modules:
        if m.lower() in imports and m !='builtins':
            modules.append((m,'unknown'))

    # print('System=='+platform.system()+' '+platform.release()+'; Version=='+platform.version())
    for r in modules:
        print("{}=={}".format(*r))
Answered By: gafortiby

Another solution (based on Vivek’s answer):

import types

def imports():
    for name, val in globals().items():
        if isinstance(val, types.ModuleType):
            yield val.__name__

excludes = ['builtins', 'types', 'sys']

imported_modules = [module for module in imports() if module not in excludes]

clean_modules = []

for module in imported_modules:

    sep = '.'  # to handle 'matplotlib.pyplot' cases
    rest = module.split(sep, 1)[0]
    clean_modules.append(rest)

changed_imported_modules = list(set(clean_modules))  # drop duplicates

pip_modules = !pip freeze  # you could also use `!conda list` with anaconda

for module in pip_modules:
    name, version = module.split('==')
    if name in changed_imported_modules:
        print(name + 't' + version)

Sample output:

astropy 3.2.1
matplotlib  3.1.0
numpy   1.16.4
pandas  0.25.0
Answered By: Michael Belvedersky

Adapted from gafortiby’s answer: a shorter version to list only explicit list of packages. I found this suitable to memorize versions of the most important packages used in a jupyter notebook (for other readers or future use):

import pkg_resources
# list packages to be checked
root_packages = [
    'geoviews', 'geopandas', 'pandas', 'numpy', 
    'matplotlib', 'shapely', 'cartopy', 'holoviews',
    'mapclassify', 'fiona', 'bokeh']
# print versions, but check if package is imported first
for m in pkg_resources.working_set:
    if m.project_name.lower() in root_packages:
        print(f"{m.project_name}=={m.version}")

Output:

Shapely==1.7.0
pandas==1.0.1
numpy==1.18.1
matplotlib==3.1.3
mapclassify==2.2.0
holoviews==1.12.7
geoviews==1.6.6
geopandas==0.6.3
Fiona==1.8.13
Cartopy==0.17.0
bokeh==1.4.0

Enhanced version with nicer display:

import pkg_resources
from IPython.display import display
import pandas as pd

root_packages = [
    'geoviews', 'geopandas', 'pandas', 'numpy', 'cloudpickle',
    'matplotlib', 'shapely', 'cartopy', 'holoviews',
    'mapclassify', 'fiona', 'bokeh', 'pyproj', 'ipython',
    'jupyterlab']
root_packages.sort(reverse=True)
root_packages_list = []

for m in pkg_resources.working_set:
    if m.project_name.lower() in root_packages:
        root_packages_list.append([m.project_name, m.version])

display(pd.DataFrame(
            root_packages_list,
            columns=["package", "version"]
        ).set_index("package").transpose())

Output:
Example output package versions

Answered By: Alex

One-liner:

# In[1]:
import pandas as pd
import numpy as np
import tensorflow as tf

print('n'.join(f'{m.__name__}=={m.__version__}' for m in globals().values() if getattr(m, '__version__', None)))

Output:

pandas==1.1.1
numpy==1.19.1
tensorflow==2.2.0
Answered By: Tamal Govinda das

Since these answers are a bit dated, the simpler solutions didn’t work for me and it took some stumbling to find an easy, working solution elsewhere online:

from sinfo import sinfo
sinfo()

Answered By: scideas

I had some problems just doing writing in an empty cell
pip list
But once I ran it in a whole new file, I had no problems at all, and got all the libraries installed in the notebook!

Answered By: Adan Mori
Categories: questions Tags: ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.