How to find "import name" of any package in Python?
Question:
I wonder if is there any reliable and consistant way to get a Python package’s “import name” / namespace. For example;
Package; django-haystack
Import name; haystack
or
Package; ipython
Import name; IPython
So far I know, PyPi doesn’t store that information that I’ve checked with PyPiXmlRpc.
I also tried to automate to download the package, extract it and dig the .egg-info but some packages doesn’t have that folder at all.
Any help will be appreciated and will be used for a good-manner gadget 🙂
Answers:
In principal, everything you need to get that information is in the setup.py that is supposed to be in every such package. That information would roughly be the union of the packages, py_modules, ext_package and ext_modules of the Distribution object. In fact, here’s a little script that mocks out distutils.core.setup just for the purpose of getting that information.
import distutils.core
distutils.core._setup_stop_after = "config"
_real_setup = distutils.core.setup
def _fake_setup(*args, **kwargs):
global dist
dist = _real_setup(*args, **kwargs)
distutils.core.setup = _fake_setup
import sys
setup_file = sys.argv[1]
sys.argv[:] = sys.argv[1:]
import os.path
os.chdir(os.path.dirname(setup_file))
execfile(os.path.basename(setup_file))
cat = lambda *seq: sum((i for i in seq if i is not None), [])
pkgs = set(package.split('.')[0] for package
in cat(dist.packages,
dist.py_modules,
[m.name for m in cat(dist.ext_modules)],
[m.name for m in cat(dist.ext_package)]))
print "n".join(pkgs)
For many packages, this will work like a charm, but for a counterexample, see numpy, It breaks because numpy provides its own distutils, and I can see no obvious way around it.
Note that what you call a package here is not a package but a distribution. A distribution can contain zero or more modules or packages. That means there is no one-to-one mapping of distributions to packages.
I’m not sure there is a way to detect what modules and packages will be installed by a distribution, other than actually installing it and introspect filesystem changes for newly added packages, modules and pth files.
Wheels
I know this is an old question, but wheel packages have since been invented! Since a wheel is simply a zip file that gets extracted into the lib/site-packages directory, an examination of the contents of the wheel archive can give you the top level imports.
>>> import zipfile
>>> zf = zipfile.ZipFile('setuptools-35.0.2-py2.py3-none-any.whl')
>>> top_level = set([x.split('/')[0] for x in zf.namelist()])
>>> # filter out the .dist-info directory
>>> top_level = [x for x in top_level if not x.endswith('.dist-info')]
>>> top_level
['setuptools', 'pkg_resources', 'easy_install.py']
So setuptools actually gives you three top level imports!
pip download
pip now has a download command, so you can simply run pip download setuptools (or whatever package you like) and then examine it.
Reverse look up
Unfortunately I haven’t found an easy way to go the other direction yet. That is, given the import name, what is the package name. This can be a problem if you are looking at some example code or maybe if you use Anaconda that comes with a bunch of packages pre-installed and you want to know the actual package name.
After searching lots of information, finally found the only thing which worked for me as expected. Example for python-dotenv package which has dotenv import name:
$ cat $(python -c "import pkg_resources; print(pkg_resources.get_distribution('python-dotenv').egg_info)")/top_level.txt
dotenv
(taken from this answer)
My project johnnydep has this feature:
>>> from johnnydep import JohnnyDist
>>> dist = JohnnyDist("django-haystack")
>>> dist.import_names
['haystack']
Do note that a distribution may provide multiple import names:
>>> JohnnyDist("setuptools").import_names
['pkg_resources', 'setuptools']
or none at all:
>>> JohnnyDist("bs4").import_names
[]
I wonder if is there any reliable and consistant way to get a Python package’s “import name” / namespace. For example;
Package; django-haystack
Import name; haystack
or
Package; ipython
Import name; IPython
So far I know, PyPi doesn’t store that information that I’ve checked with PyPiXmlRpc.
I also tried to automate to download the package, extract it and dig the .egg-info but some packages doesn’t have that folder at all.
Any help will be appreciated and will be used for a good-manner gadget 🙂
In principal, everything you need to get that information is in the setup.py that is supposed to be in every such package. That information would roughly be the union of the packages, py_modules, ext_package and ext_modules of the Distribution object. In fact, here’s a little script that mocks out distutils.core.setup just for the purpose of getting that information.
import distutils.core
distutils.core._setup_stop_after = "config"
_real_setup = distutils.core.setup
def _fake_setup(*args, **kwargs):
global dist
dist = _real_setup(*args, **kwargs)
distutils.core.setup = _fake_setup
import sys
setup_file = sys.argv[1]
sys.argv[:] = sys.argv[1:]
import os.path
os.chdir(os.path.dirname(setup_file))
execfile(os.path.basename(setup_file))
cat = lambda *seq: sum((i for i in seq if i is not None), [])
pkgs = set(package.split('.')[0] for package
in cat(dist.packages,
dist.py_modules,
[m.name for m in cat(dist.ext_modules)],
[m.name for m in cat(dist.ext_package)]))
print "n".join(pkgs)
For many packages, this will work like a charm, but for a counterexample, see numpy, It breaks because numpy provides its own distutils, and I can see no obvious way around it.
Note that what you call a package here is not a package but a distribution. A distribution can contain zero or more modules or packages. That means there is no one-to-one mapping of distributions to packages.
I’m not sure there is a way to detect what modules and packages will be installed by a distribution, other than actually installing it and introspect filesystem changes for newly added packages, modules and pth files.
Wheels
I know this is an old question, but wheel packages have since been invented! Since a wheel is simply a zip file that gets extracted into the lib/site-packages directory, an examination of the contents of the wheel archive can give you the top level imports.
>>> import zipfile
>>> zf = zipfile.ZipFile('setuptools-35.0.2-py2.py3-none-any.whl')
>>> top_level = set([x.split('/')[0] for x in zf.namelist()])
>>> # filter out the .dist-info directory
>>> top_level = [x for x in top_level if not x.endswith('.dist-info')]
>>> top_level
['setuptools', 'pkg_resources', 'easy_install.py']
So setuptools actually gives you three top level imports!
pip download
pip now has a download command, so you can simply run pip download setuptools (or whatever package you like) and then examine it.
Reverse look up
Unfortunately I haven’t found an easy way to go the other direction yet. That is, given the import name, what is the package name. This can be a problem if you are looking at some example code or maybe if you use Anaconda that comes with a bunch of packages pre-installed and you want to know the actual package name.
After searching lots of information, finally found the only thing which worked for me as expected. Example for python-dotenv package which has dotenv import name:
$ cat $(python -c "import pkg_resources; print(pkg_resources.get_distribution('python-dotenv').egg_info)")/top_level.txt
dotenv
(taken from this answer)
My project johnnydep has this feature:
>>> from johnnydep import JohnnyDist
>>> dist = JohnnyDist("django-haystack")
>>> dist.import_names
['haystack']
Do note that a distribution may provide multiple import names:
>>> JohnnyDist("setuptools").import_names
['pkg_resources', 'setuptools']
or none at all:
>>> JohnnyDist("bs4").import_names
[]