Questions on pyproject.toml vs setup.py
Question:
Reading up on pyproject.toml, python -m pip install, poetry, flit, etc – I have several questions regarding replacing setup.py with pyproject.toml.
My biggest question was – how does a toml file replace a setup.py. Meaning, a toml file can’t do everything a py file can. Reading into it, poetry and flit completely replace setup.py with pyproject.toml. While pip uses the pyproject.toml to specify the build tools, but then still uses the setup.py for everything else.
A good example is, pip currently doesn’t have a way to do entry points for console script directly in a toml file, but poetry and flit do.
- https://flit.readthedocs.io/en/latest/pyproject_toml.html#scripts-section
- https://python-poetry.org/docs/pyproject/#scripts
My main question right now is;
The point of pyproject.toml is to provide build system requirement. It is a metadata file. So wouldn’t the ideal solution to be to use this file only to specify the build system requirements and still leverage the setup.py for everything else.
I am confused because I feel like we’re losing a lot to over come a fairly simple problem. By entirely doing way with the setup.py and replacing it with pyproject.toml, we lose a lot of helpful things we can do in a setup.py. We can’t use a __version.py__, and we lose the ability to automatically create a universal wheel and sdist and upload our packages to PyPi using Twine. which we can currently do in the setup.py file.
I’m just having a had time wrapping my head around why we would want to completely replace the setup.py with a metadata only file. It seems like using them together is the best of both worlds. We solve the chicken and the egg build system issue, and we get to retain a lot of useful things the setup.py can do.
Wouldn’t we need a setup.py to install in Dev mode anyway? Or maybe that is just a pip problem?
Answers:
Currently I am investigating this feature too. I found this experimental feature explanation of setuptools which should just refer to the pyproject.toml without any need of setup.py in the end.
Regarding dynamic behavior of setup.py, I figured out that you can set a dynamic behavior for fields under the [project] metadata
dynamic = ["version"]
[tool.setuptools.dynamic]
version = {attr = "my_package.__version__"}
whereat the corresponding version in this example is set in, e.g. my_package.__init__.py
__version__ = "0.1.0"
__all__ = ["__version__"]
In the end, I guess that setuptools will cover the missing setup.py execution and places the necessary egg-links for the development mode.
Pip doesn’t use setup.py for everything else.
Setuptools itself discourages using setup.py and transitions to setup.cfg (or pyproject.toml).
License, summary, name, author-email, version… all work with pip without a setup.py, and a wheel is created by pip too.
See for yourself what pip can all do with a pyproject.toml file:
https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/
You can specify entry points with the entry-points key as shown at the link by @casabre:
https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html
Toml is more than a simple requirements.txt. You can set complex directives like i.e:
tool.setuptools.packages = {find = {where = ["src"], exclude=["tests*"]}}
And __version__.py becomes a simple entry in your pyproject.toml file, even dynamic as @casabre stated:
[project]
version = "0.1.0"
Upload to PyPi with twine should alsow work as per this link:
https://setuptools.pypa.io/en/latest/userguide/quickstart.html#uploading-your-package-to-pypi
To install in dev mode you can configure someting like this in your pyproject.toml:
[project.optional-dependencies]
dev = ["flake8", "pytest"]
And then pip install .[dev] in your project folder.
Reading up on pyproject.toml, python -m pip install, poetry, flit, etc – I have several questions regarding replacing setup.py with pyproject.toml.
My biggest question was – how does a toml file replace a setup.py. Meaning, a toml file can’t do everything a py file can. Reading into it, poetry and flit completely replace setup.py with pyproject.toml. While pip uses the pyproject.toml to specify the build tools, but then still uses the setup.py for everything else.
A good example is, pip currently doesn’t have a way to do entry points for console script directly in a toml file, but poetry and flit do.
- https://flit.readthedocs.io/en/latest/pyproject_toml.html#scripts-section
- https://python-poetry.org/docs/pyproject/#scripts
My main question right now is;
The point of pyproject.toml is to provide build system requirement. It is a metadata file. So wouldn’t the ideal solution to be to use this file only to specify the build system requirements and still leverage the setup.py for everything else.
I am confused because I feel like we’re losing a lot to over come a fairly simple problem. By entirely doing way with the setup.py and replacing it with pyproject.toml, we lose a lot of helpful things we can do in a setup.py. We can’t use a __version.py__, and we lose the ability to automatically create a universal wheel and sdist and upload our packages to PyPi using Twine. which we can currently do in the setup.py file.
I’m just having a had time wrapping my head around why we would want to completely replace the setup.py with a metadata only file. It seems like using them together is the best of both worlds. We solve the chicken and the egg build system issue, and we get to retain a lot of useful things the setup.py can do.
Wouldn’t we need a setup.py to install in Dev mode anyway? Or maybe that is just a pip problem?
Currently I am investigating this feature too. I found this experimental feature explanation of setuptools which should just refer to the pyproject.toml without any need of setup.py in the end.
Regarding dynamic behavior of setup.py, I figured out that you can set a dynamic behavior for fields under the [project] metadata
dynamic = ["version"]
[tool.setuptools.dynamic]
version = {attr = "my_package.__version__"}
whereat the corresponding version in this example is set in, e.g. my_package.__init__.py
__version__ = "0.1.0"
__all__ = ["__version__"]
In the end, I guess that setuptools will cover the missing setup.py execution and places the necessary egg-links for the development mode.
Pip doesn’t use setup.py for everything else.
Setuptools itself discourages using setup.py and transitions to setup.cfg (or pyproject.toml).
License, summary, name, author-email, version… all work with pip without a setup.py, and a wheel is created by pip too.
See for yourself what pip can all do with a pyproject.toml file:
https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/
You can specify entry points with the entry-points key as shown at the link by @casabre:
https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html
Toml is more than a simple requirements.txt. You can set complex directives like i.e:
tool.setuptools.packages = {find = {where = ["src"], exclude=["tests*"]}}
And __version__.py becomes a simple entry in your pyproject.toml file, even dynamic as @casabre stated:
[project]
version = "0.1.0"
Upload to PyPi with twine should alsow work as per this link:
https://setuptools.pypa.io/en/latest/userguide/quickstart.html#uploading-your-package-to-pypi
To install in dev mode you can configure someting like this in your pyproject.toml:
[project.optional-dependencies]
dev = ["flake8", "pytest"]
And then pip install .[dev] in your project folder.