How to reference a requirements.txt in the pyproject.toml of a setuptools project?
Question:
I’m trying to migrate a setuptools-based project from the legacy setup.py towards modern pyproject.toml configuration.
At the same time I want to keep well established workflows based on pip-compile, i.e., a requirements.in that gets compiled to a requirements.txt (for end-user / non-library projects of course). This has important benefits as a result of the full transparency:
- 100% reproducible installs due to pinning the full transitive closure of dependencies.
- better understanding of dependency conflicts in the transitive closure of dependencies.
For this reason I don’t want to maintain the dependencies directly inside the pyproject.toml via a dependencies = [] list, but rather externally in the pip-compiled managed requirements.txt.
This makes me wonder: Is there a way to reference a requirements.txt file in the pyproject.toml configuration, without having to fallback to a setup.py script?
Answers:
Use dynamic metadata:
[project]
dynamic = ["dependencies"]
[tool.setuptools.dynamic]
dependencies = {file = ["requirements.txt"]}
pip-tools can read the dependencies from pyproject.toml, just like it would from a requirements.in.
That’s not the same as referencing the compiled dependencies in requirements.txt, but it is one step closer: you don’t need requirements.in and you can distribute requirements.txt.
Like @sinoroc, I think it may be risky to "deliver" requirements with only pinned versions. It’s sufficient that one similarly set up project is set up along with your project, and you’re likely to get clashes.
Of course, you may be able to exclude that, or perhaps even want to exploit that nothing else is used along with your project. But that’s a very specific use case.
I’m trying to migrate a setuptools-based project from the legacy setup.py towards modern pyproject.toml configuration.
At the same time I want to keep well established workflows based on pip-compile, i.e., a requirements.in that gets compiled to a requirements.txt (for end-user / non-library projects of course). This has important benefits as a result of the full transparency:
- 100% reproducible installs due to pinning the full transitive closure of dependencies.
- better understanding of dependency conflicts in the transitive closure of dependencies.
For this reason I don’t want to maintain the dependencies directly inside the pyproject.toml via a dependencies = [] list, but rather externally in the pip-compiled managed requirements.txt.
This makes me wonder: Is there a way to reference a requirements.txt file in the pyproject.toml configuration, without having to fallback to a setup.py script?
Use dynamic metadata:
[project]
dynamic = ["dependencies"]
[tool.setuptools.dynamic]
dependencies = {file = ["requirements.txt"]}
pip-tools can read the dependencies from pyproject.toml, just like it would from a requirements.in.
That’s not the same as referencing the compiled dependencies in requirements.txt, but it is one step closer: you don’t need requirements.in and you can distribute requirements.txt.
Like @sinoroc, I think it may be risky to "deliver" requirements with only pinned versions. It’s sufficient that one similarly set up project is set up along with your project, and you’re likely to get clashes.
Of course, you may be able to exclude that, or perhaps even want to exploit that nothing else is used along with your project. But that’s a very specific use case.