Is requirements.txt still needed when using pyproject.toml?
Question:
Since mid 2022 it is now possible to get rid of setup.py, setup.cfg in favor of pyproject.toml. Editable installs work with recent versions of setuptools and pip and even the official packaging tutorial switched away from setup.py to pyproject.toml.
However, documentation regarding requirements.txt seems to be have been also removed, and I wonder where to put the pinned requirements now?
As a refresher: It used to be common practice to put the dependencies (without version pinning) in setup.py avoiding issues when this package gets installed with other packages needing the same dependencies but with conflicting version requirements. For packaging libraries a setup.py was usually sufficient.
For deployments (i.e. non libraries) you usually also provided a requirements.txt with version-pinned dependencies. So you don’t accidentally get the latest and greatest but the exact versions of dependencies that that package has been tested with.
So my question is, did anything change? Do you still put the pinned requirements in the requirements.txt when used together with pyproject.toml? Or is there an extra section
for that in pyproject.toml? Is there some documentation on that somewhere?
Answers:
This is the pip documentation for pyproject.toml
…This file contains build system requirements and information, which are used by pip to build the package.
So this is not the correct place. Looking at the side bar we can see there is an entry for Requirements File Format which is the "old" requirements.txt file
I suggest switching to poetry, it’s way better than a standard pip for dependency management. And because it uses pyproject.toml your dependencies and configs are in one place so it’s easier to manage everything
Quoting myself from here
My current assumption is: […] you put your (mostly unpinned) dependencies to pyproject.toml instead of setup.py, so you library can be installed as a dependency of something else without causing much troubles because of issues resolving version constraints.
On top of that, for "deployable applications" (for lack of a better term), you still want to maintain a separate requirements.txt with exact version pinning.
Which has been confirmed by a Python Packaging Authority (PyPA) member and clarification of PyPA’s recommendations should be updated accordingly at some point.
Since mid 2022 it is now possible to get rid of setup.py, setup.cfg in favor of pyproject.toml. Editable installs work with recent versions of setuptools and pip and even the official packaging tutorial switched away from setup.py to pyproject.toml.
However, documentation regarding requirements.txt seems to be have been also removed, and I wonder where to put the pinned requirements now?
As a refresher: It used to be common practice to put the dependencies (without version pinning) in setup.py avoiding issues when this package gets installed with other packages needing the same dependencies but with conflicting version requirements. For packaging libraries a setup.py was usually sufficient.
For deployments (i.e. non libraries) you usually also provided a requirements.txt with version-pinned dependencies. So you don’t accidentally get the latest and greatest but the exact versions of dependencies that that package has been tested with.
So my question is, did anything change? Do you still put the pinned requirements in the requirements.txt when used together with pyproject.toml? Or is there an extra section
for that in pyproject.toml? Is there some documentation on that somewhere?
This is the pip documentation for pyproject.toml
…This file contains build system requirements and information, which are used by pip to build the package.
So this is not the correct place. Looking at the side bar we can see there is an entry for Requirements File Format which is the "old" requirements.txt file
I suggest switching to poetry, it’s way better than a standard pip for dependency management. And because it uses pyproject.toml your dependencies and configs are in one place so it’s easier to manage everything
Quoting myself from here
My current assumption is: […] you put your (mostly unpinned) dependencies to pyproject.toml instead of setup.py, so you library can be installed as a dependency of something else without causing much troubles because of issues resolving version constraints.
On top of that, for "deployable applications" (for lack of a better term), you still want to maintain a separate requirements.txt with exact version pinning.
Which has been confirmed by a Python Packaging Authority (PyPA) member and clarification of PyPA’s recommendations should be updated accordingly at some point.