Best practices: how do you list required dependencies in your setup.py?

Question

This is how I do it currently:

import os
from setuptools import setup, find_packages
here = os.path.abspath(os.path.dirname(__file__))

requires = [
    'pyramid',
    'pyramid_debugtoolbar',
    'waitress',
    'requests',
    'mock',
    'gunicorn',
    'mongoengine',
    ]

setup(name='repoapi',
      version='0.0',
      description='repoapi',
      packages=find_packages(),
      include_package_data=True,
      zip_safe=False,
      install_requires=requires,
      tests_require=requires,
      test_suite="repoapi",
      entry_points="""
      [paste.app_factory]
      main = repoapi:main
      """,
      )

Is this an okay way? I have some troubles. For example, for pyramid, I cannot use the system-wide nosetests plugin to run tests. I need to install pyramid in the global python site-packages!

But I don’t want that. So I must install nose in the virtualenv of this project. But I don’t want it to be a dependency. I don’t feel like it should belong to requires. It isn’t. Yet, I also don’t want to install by hand all the time. Yeah I know I have a lot of I don’t want to do this and that…

But how would you solve that? I don’t want to tamper the global python site-packages, but I want to install nose as part of the virtualenv.

Also, pip install requirement files. It’s slightly more accurate because I don’t need to specify the version manually and I don’t need to be afraid of updating setup.py manually. Just throw pip freeze > file.txt and done.

However, pip can return garbage because we throw garbage packages into virtualenv.

So many blades. What’s the best practice? How do you deal with these issues?

Maybe I missed it, but https://github.com/django/django/blob/master/setup.py, how did Django do it?

Asked By: CppLearner

||

Source

Answer 1

You can split up your requirements into “install” dependencies and “test” dependencies like this:

import os
from setuptools import setup, find_packages
here = os.path.abspath(os.path.dirname(__file__))

install_requires = [
    'pyramid',
    'pyramid_debugtoolbar',
    'waitress',
    'requests',
    'gunicorn',
    'mongoengine',
    ]

tests_require = [
    'mock',
    'nose',
    ]

setup(name='repoapi',
      ...
      install_requires=install_requires,
      tests_require=tests_require,
      test_suite="nose.collector",
      ...
      )

This way, when someone installs the package, only the “install” dependencies are installed. So, if someone only wants to use the package (and they aren’t interested in running the tests), then they don’t have to install the test dependencies.

When you do want to run the tests, you can use this:

$ python setup.py test

Per the docs:

Note that these required projects will not be installed on the system where the tests are run, but only downloaded to the project’s setup directory if they’re not already installed locally.

Once the “test” dependencies are in place, then it will run the “test_suite” command. Since you mentioned nose as your preferred test runner, I showed how you use “nose.collector” to configure that.

Incidentally, the Django setup.py is not the cleanest example for understanding the basics of setuptools. I think the Sentry setup.py is a better example to learn from.

Answered By: Tom Offermann

Answer 2

If you are using requirement files, then an alternative would be to read their content instead of duplicating it:

import pathlib
from setuptools import setup, find_packages

HERE = pathlib.Path(__file__).parent
INSTALL_REQUIRES = (HERE / "requirements.txt").read_text().splitlines()
TESTS_REQUIRE = (HERE / "test-requirements.txt").read_text().splitlines()[1:]

setup(...,
      install_requires=INSTALL_REQUIRES,
      tests_require=TESTS_REQUIRE,
      ...
      )

I find this approach much more stable and less prone to errors, since requirements change sometimes and one often forgets to update two places.

Note: notice the TESTS_REQUIRE start from the second line, this is due to the fact that the first line of test-requirements.txt is often -r requirements.txt. Feel free to change it if your case is different.

Answered By: Peter K

Best practices: how do you list required dependencies in your setup.py?

Question:

Answers: