How to interpret the "Package would be ignored" warning generated by setuptools?
Question:
I work on several python packages that contain data within them. I add them via the MANIFEST.in file, passing include_package_data=True to setup. For example:
# MANIFEST.in
graft mypackage/plugins
graft mypackage/data
Up to now, this has worked without warnings as far as I know. However, in setuptools 62.3.0, I get the following message:
SetuptoolsDeprecationWarning: Installing 'mypackage.plugins' as data is deprecated, please list it in `packages`.
07:53:53 !!
07:53:53
07:53:53
07:53:53 ############################
07:53:53 # Package would be ignored #
07:53:53 ############################
07:53:53 Python recognizes 'mypackage.plugins' as an importable package, however it is
07:53:53 included in the distribution as "data".
07:53:53 This behavior is likely to change in future versions of setuptools (and
07:53:53 therefore is considered deprecated).
07:53:53
07:53:53 Please make sure that 'mypackage.plugins' is included as a package by using
07:53:53 setuptools' `packages` configuration field or the proper discovery methods
07:53:53 (for example by using `find_namespace_packages(...)`/`find_namespace:`
07:53:53 instead of `find_packages(...)`/`find:`).
07:53:53
07:53:53 You can read more about "package discovery" and "data files" on setuptools
07:53:53 documentation page.
I get the above warning for pretty much every directory within mypackage that contains data and is included by MANIFEST.in.
My goal is to include arbitrary data (which could even include python files in the case of a plugin interface) in a package so that it can be accessed by users who install via wheel or tarball. I would also like that applications built by, e.g., pyinstaller, that pull my package in can easily collect the data with collect_data_files, which for me has worked without any additional setup with the current methodology.
What is the proper way to do this going forward?
Answers:
The TL;DR is that in Python since PEP 420, directories count as packages, even if they don’t have a __init__.py file.
The main difference is that directories without __init__.py are called "namespace packages".
Accordingly, if a project wants to distribute directories without a __init__.py file, it should use packages=find_namespace_packages() (setup.py) or packages = find_namespace: (setup.cfg). Details on how to use those tools can be found on these docs. Doing this change should make the error go away.
The MANIFEST.in or the include_package_data=True should be fine.
For those who face this issue, a bit more detailed steps on answer above. In your setup.py file, import the find_namespace_packages:
from setuptools import find_namespace_packages
and then in your setup() section, add this two lines:
include_package_data=True,
packages=find_namespace_packages(exclude=["*.tests", "*.tests.*", "tests.*", "tests"]),
In my case, I previously had packages=find_packages(exclude=["*.tests", "*.tests.*", "tests.*", "tests"]), so I just needed to replace find_packages to find_namespace_packages.
I work on several python packages that contain data within them. I add them via the MANIFEST.in file, passing include_package_data=True to setup. For example:
# MANIFEST.in
graft mypackage/plugins
graft mypackage/data
Up to now, this has worked without warnings as far as I know. However, in setuptools 62.3.0, I get the following message:
SetuptoolsDeprecationWarning: Installing 'mypackage.plugins' as data is deprecated, please list it in `packages`.
07:53:53 !!
07:53:53
07:53:53
07:53:53 ############################
07:53:53 # Package would be ignored #
07:53:53 ############################
07:53:53 Python recognizes 'mypackage.plugins' as an importable package, however it is
07:53:53 included in the distribution as "data".
07:53:53 This behavior is likely to change in future versions of setuptools (and
07:53:53 therefore is considered deprecated).
07:53:53
07:53:53 Please make sure that 'mypackage.plugins' is included as a package by using
07:53:53 setuptools' `packages` configuration field or the proper discovery methods
07:53:53 (for example by using `find_namespace_packages(...)`/`find_namespace:`
07:53:53 instead of `find_packages(...)`/`find:`).
07:53:53
07:53:53 You can read more about "package discovery" and "data files" on setuptools
07:53:53 documentation page.
I get the above warning for pretty much every directory within mypackage that contains data and is included by MANIFEST.in.
My goal is to include arbitrary data (which could even include python files in the case of a plugin interface) in a package so that it can be accessed by users who install via wheel or tarball. I would also like that applications built by, e.g., pyinstaller, that pull my package in can easily collect the data with collect_data_files, which for me has worked without any additional setup with the current methodology.
What is the proper way to do this going forward?
The TL;DR is that in Python since PEP 420, directories count as packages, even if they don’t have a __init__.py file.
The main difference is that directories without __init__.py are called "namespace packages".
Accordingly, if a project wants to distribute directories without a __init__.py file, it should use packages=find_namespace_packages() (setup.py) or packages = find_namespace: (setup.cfg). Details on how to use those tools can be found on these docs. Doing this change should make the error go away.
The MANIFEST.in or the include_package_data=True should be fine.
For those who face this issue, a bit more detailed steps on answer above. In your setup.py file, import the find_namespace_packages:
from setuptools import find_namespace_packages
and then in your setup() section, add this two lines:
include_package_data=True,
packages=find_namespace_packages(exclude=["*.tests", "*.tests.*", "tests.*", "tests"]),
In my case, I previously had packages=find_packages(exclude=["*.tests", "*.tests.*", "tests.*", "tests"]), so I just needed to replace find_packages to find_namespace_packages.