Given the name of a Python package, what is the name of the module to import?
Question:
Does somebody know the logic behind Python modules names vs the name of the actual package used in easy_install?
A few (amongst others) example that seem a bit unlogical to me:
- We do
easy_install mysql-python, but the import is in fact import MySQLdb
- We do
easy_install python-memcached, but the import is in fact import memcache (without the trailing d)
I didn’t find a consistent way of finding the equivalence. For some modules, it took me a lot of browsing to find it. What am I doing wrong?
Answers:
Regrettably, there’s no method to the madness. The name in the package index is independent of the module name you import. Disastrously some packages share module names. If you install both, your application will break with even odds. (Ruby has this problem too)
Packaging in Python is generally dire. The root cause is that the language ships without a package manager. Ruby and Nodejs ship with full-featured package managers Gem and Npm, and have nurtured sharing communities centred around GitHub. Npm makes publishing packages as easy as installing them. Nodejs arrived 2009 and already has 14k packages. The venerable Python package index lists 24k. Ruby Gems lists 44k packages.
Fortunately, there is one decent package manager for Python, called Pip. Pip is inspired by Ruby’s Gem, but lacks some vital features (eg. listing packages, and upgrading en masse). Ironically, Pip itself is complicated to install. Installation on the popular 64-bit Windows demands building and installing two packages from source. This is a big ask for anyone new to programming.
Python’s devs are ignorant of all this frustration because they are seasoned programmers comfortable building from source, and they use Linux distributions with packaged Python modules.
Until Python ships with a package manager, thousands of developers will needlessly waste time reinventing the wheel.
Python 3 solves many problems with packaging. There aren’t any packages for Python 3.
Unlike the accepted answer suggests, it is actually possible, although indeed quite cumbersome.
The module name(s) can be found top_level.txt file in the metadata directory of a package installation.
To access it with python (replace python-memcached with your package name):
>>> import pkg_resources as pkg # included in setuptools package
>>> metadata_dir = pkg.get_distribution('python-memcached').egg_info
>>> open('%s/%s' % (metadata_dir, 'top_level.txt')).read().rstrip()
'memcache'
Or with an equivalent bash “one liner”:
cat $(python -c "import pkg_resources;
print(pkg_resources.get_distribution('python-memcached').egg_info)")/top_level.txt
Keep in mind that some packages install multiple modules, so the method I’m presenting can return multiple module names.
Does somebody know the logic behind Python modules names vs the name of the actual package used in easy_install?
A few (amongst others) example that seem a bit unlogical to me:
- We do
easy_install mysql-python, but the import is in factimport MySQLdb - We do
easy_install python-memcached, but the import is in factimport memcache(without the trailing d)
I didn’t find a consistent way of finding the equivalence. For some modules, it took me a lot of browsing to find it. What am I doing wrong?
Regrettably, there’s no method to the madness. The name in the package index is independent of the module name you import. Disastrously some packages share module names. If you install both, your application will break with even odds. (Ruby has this problem too)
Packaging in Python is generally dire. The root cause is that the language ships without a package manager. Ruby and Nodejs ship with full-featured package managers Gem and Npm, and have nurtured sharing communities centred around GitHub. Npm makes publishing packages as easy as installing them. Nodejs arrived 2009 and already has 14k packages. The venerable Python package index lists 24k. Ruby Gems lists 44k packages.
Fortunately, there is one decent package manager for Python, called Pip. Pip is inspired by Ruby’s Gem, but lacks some vital features (eg. listing packages, and upgrading en masse). Ironically, Pip itself is complicated to install. Installation on the popular 64-bit Windows demands building and installing two packages from source. This is a big ask for anyone new to programming.
Python’s devs are ignorant of all this frustration because they are seasoned programmers comfortable building from source, and they use Linux distributions with packaged Python modules.
Until Python ships with a package manager, thousands of developers will needlessly waste time reinventing the wheel.
Python 3 solves many problems with packaging. There aren’t any packages for Python 3.
Unlike the accepted answer suggests, it is actually possible, although indeed quite cumbersome.
The module name(s) can be found top_level.txt file in the metadata directory of a package installation.
To access it with python (replace python-memcached with your package name):
>>> import pkg_resources as pkg # included in setuptools package
>>> metadata_dir = pkg.get_distribution('python-memcached').egg_info
>>> open('%s/%s' % (metadata_dir, 'top_level.txt')).read().rstrip()
'memcache'
Or with an equivalent bash “one liner”:
cat $(python -c "import pkg_resources;
print(pkg_resources.get_distribution('python-memcached').egg_info)")/top_level.txt
Keep in mind that some packages install multiple modules, so the method I’m presenting can return multiple module names.