Which Python version to be used for building PyPi package?
Question:
I’m building PyPi package, which is compatible with Python 3.8 and higher.
That means that the minimum version of Python is 3.8, so I build the package in Python 3.8 environment.
My question is should I build package separately for Python 3.8 and 3.9?
I think the thing that actually builds package is the module, build, so the version of Python doesn’t matter. Isn’t it?
Answers:
The version does not matter, you can technically build a package using Python 3.9 which can also work on Python 3.8. The quickest and easiest way to ensure version compatibility would be to build it on the minimum version that you intend to support, so I would suggest building once using 3.8.
If you want to test that the package will work on different Python versions, you can use tox to run your automated tests on multiple versions in an easy way (automated tests are tests that you execute through pytest or unittest).
Here is how Python does its versioning and what the versions signify. Assuming that you are building the package to have it work on version 3.8 and all subsequent 3.x versions, then if it builds and works for 3.8, it will almost certainly work for 3.9, 3.10, 3.11 etc…
Although major versions such as 2.x and 3.x (and 4.x if it ever gets released) will have enough incompatible changes between them that it is not safe to assume that they are compatible.
It depends on what you gonna publish. If you build and publish an sdist (source distribution, .tar.gz) — Python version doesn’t matter. If you build and publish a source wheel — Python version doesn’t matter. If you build and publish a binary wheel with compiled extension(s) written in C — Python version matters very much and you must build wheels for every major Python version separately.
Python will load a wheel based on API, ABI, OS, Architecture tags that it supports. For pure Python wheels you largely get fairly universal compatibility.
For wheels with C extensions however, the ABI version must match the interpreter. CPython 3.10 will not load a wheel built for Python 3.9.
The only C extensions that are forward-compatible are ones built with the Stable API. These extensions declare what Python version they’re built against and can only use a limited subset of the API. In exchange they declare an abi3 ABI and can be loaded by newer Python versions. Note that pybind11 does NOT support the stable API.
To see the full list of tag combinations for your Python run:
$ python -m pip debug --verbose
...
Compatible tags: 450
...
cp310-cp310-macosx_10_7_universal2
cp310-cp310-macosx_10_6_universal2
...
cp310-abi3-macosx_10_7_universal2
cp310-abi3-macosx_10_6_universal2
...
cp310-none-macosx_10_5_universal2
cp310-none-macosx_10_4_universal2
cp39-abi3-macosx_13_0_arm64
cp39-abi3-macosx_13_0_universal2
...
So my Python 3.10 will load wheels built for Python 3.10, stable API wheels built all the way back to 3.2, and pure-python wheels. That’s it.
The practical way to deal with this is to use a tool like cibuildwheel in your CI pipeline (like GitHub Actions) to build all the combinations.
I’m building PyPi package, which is compatible with Python 3.8 and higher.
That means that the minimum version of Python is 3.8, so I build the package in Python 3.8 environment.
My question is should I build package separately for Python 3.8 and 3.9?
I think the thing that actually builds package is the module, build, so the version of Python doesn’t matter. Isn’t it?
The version does not matter, you can technically build a package using Python 3.9 which can also work on Python 3.8. The quickest and easiest way to ensure version compatibility would be to build it on the minimum version that you intend to support, so I would suggest building once using 3.8.
If you want to test that the package will work on different Python versions, you can use tox to run your automated tests on multiple versions in an easy way (automated tests are tests that you execute through pytest or unittest).
Here is how Python does its versioning and what the versions signify. Assuming that you are building the package to have it work on version 3.8 and all subsequent 3.x versions, then if it builds and works for 3.8, it will almost certainly work for 3.9, 3.10, 3.11 etc…
Although major versions such as 2.x and 3.x (and 4.x if it ever gets released) will have enough incompatible changes between them that it is not safe to assume that they are compatible.
It depends on what you gonna publish. If you build and publish an sdist (source distribution, .tar.gz) — Python version doesn’t matter. If you build and publish a source wheel — Python version doesn’t matter. If you build and publish a binary wheel with compiled extension(s) written in C — Python version matters very much and you must build wheels for every major Python version separately.
Python will load a wheel based on API, ABI, OS, Architecture tags that it supports. For pure Python wheels you largely get fairly universal compatibility.
For wheels with C extensions however, the ABI version must match the interpreter. CPython 3.10 will not load a wheel built for Python 3.9.
The only C extensions that are forward-compatible are ones built with the Stable API. These extensions declare what Python version they’re built against and can only use a limited subset of the API. In exchange they declare an abi3 ABI and can be loaded by newer Python versions. Note that pybind11 does NOT support the stable API.
To see the full list of tag combinations for your Python run:
$ python -m pip debug --verbose
...
Compatible tags: 450
...
cp310-cp310-macosx_10_7_universal2
cp310-cp310-macosx_10_6_universal2
...
cp310-abi3-macosx_10_7_universal2
cp310-abi3-macosx_10_6_universal2
...
cp310-none-macosx_10_5_universal2
cp310-none-macosx_10_4_universal2
cp39-abi3-macosx_13_0_arm64
cp39-abi3-macosx_13_0_universal2
...
So my Python 3.10 will load wheels built for Python 3.10, stable API wheels built all the way back to 3.2, and pure-python wheels. That’s it.
The practical way to deal with this is to use a tool like cibuildwheel in your CI pipeline (like GitHub Actions) to build all the combinations.