How to extract dependencies from a PyPi package without downloading it?

Question

I want to get the dependency of a PyPi package remotely without needing to download it completely.

I seem to understand (reading the pip code) that pip when resolving dependencies seems to read the egg once the package has been downloaded.

Is there any other way?

Asked By: Olivier Girardot

||

Source

Answer 1

Sadly, pip doesn’t have this function. The metadata available for packages on PyPI does not include information about the dependencies.

Normally, you can find the detailed dependency from the README file from project website.

pip search can give some information about the package. It can tell you what is it based on.

$ pip search flask
Flask     - A microframework based on Werkzeug, Jinja2 and good intentions

Answered By: jinghli

Answer 2

I’ve just needed to find a way to do this and this is what I came up with (stolen from pip).

def dist_metadata(setup_py):
    '''Get the dist object for setup.py file'''

    with open(setup_py) as f:
        d = f.read()

    try:
        # we have to do this with current globals else
        # imports will fail. secure? not really. A
        # problem? not really if your setup.py sources are 
        # trusted
        exec d in globals(), globals()
    except SystemExit:
        pass

    return distutils.core._setup_distribution

https://stackoverflow.com/a/12505166/3332282 answers why the exec incantation is subtle and hard to get right.

Answered By: Brian Stengaard – Podio

Answer 3

As jinghli notes, there isn’t currently a reliable way to get the dependency of an arbitrary PyPi package remotely without needing to download it completely. And in fact the dependencies sometimes depend on your environment, so an approach like Brian’s of executing setup.py code is needed in the general case.

The way the Python ecosystem handles dependencies started evolving in the 1990’s before the problem was well understood. PEP 508 — Dependency specification for Python Software Packages sets us on course to improve the situtation, and an “aspirational” draft approach in PEP 426 — Metadata for Python Software Packages 2.0 may improve it more in the future, in conjunction with the reimplementation of PyPI as Warehouse.

The current situation is described well in the document Python Dependency Resolution.

PyPI does provide a json interface to download metadata for each package. The info.requires_dist object contains a list of names of required packages with optional version restrictions etc. It is often missing, but it is one place to start.

E.g. Django (json) indicates:

{ "info": { ... "requires_dist": [ "bcrypt; extra == 'bcrypt'", "argon2-cffi (>=16.1.0); extra == 'argon2'", "pytz" ], ... }

Answered By: nealmcb

Answer 4

Use pipdeptree to view dependencies of installed PyPI packages.

Install:

pip install pipdeptree

Then run:

pipdeptree

You’ll see something like that:

Warning!!! Possible conflicting dependencies found:
* Mako==0.9.1 -> MarkupSafe [required: >=0.9.2, installed: 0.18]
  Jinja2==2.7.2 -> MarkupSafe [installed: 0.18]
------------------------------------------------------------------------
Lookupy==0.1
wsgiref==0.1.2
argparse==1.2.1
psycopg2==2.5.2
Flask-Script==0.6.6
  - Flask [installed: 0.10.1]
    - Werkzeug [required: >=0.7, installed: 0.9.4]
    - Jinja2 [required: >=2.4, installed: 2.7.2]
      - MarkupSafe [installed: 0.18]
    - itsdangerous [required: >=0.21, installed: 0.23]
alembic==0.6.2
  - SQLAlchemy [required: >=0.7.3, installed: 0.9.1]
  - Mako [installed: 0.9.1]
    - MarkupSafe [required: >=0.9.2, installed: 0.18]
ipython==2.0.0
slugify==0.0.1
redis==2.9.1

Answered By: Rockallite

Answer 5

You can fetch PyPI package metadata in JSON format using a URL like: https://pypi.org/pypi/<package name>/json.

Example: https://pypi.org/pypi/tensorflow/json

From a comment on the OP that was the right answer for me.

Answered By: alextes

How to extract dependencies from a PyPi package without downloading it?

Question:

Answers: