How to use setup.py console_script with native namespace packages
Question:
I’m unable to define console_scripts
for my Python package that utilizes the Python namespace packages structure.
I am following the Python namespace packages outlined in the Python packaging document. Specifically, I am using native namespace packaging in Python 3.8.2.
I have created my project like this:
src
└── main
└── python
└── mynamespace-mypackage
├── mynamespace
│ └── mypackage
│ ├── __init__.py
│ ├── __main__.py
│ ├── module_a.py
│ └── module_b.py
└── setup.py
In setup.py
, I write:
#!/usr/bin/env python3
from setuptools import setup, find_namespace_packages
PACKAGE = "mynamespace-mypackage"
VERSION = "0.1a"
setup(
name=PACKAGE,
version=VERSION,
packages=find_namespace_packages(include=["mynamespace.*"]),
description="My fancy package",
entry_points={
"console_scripts": {
"do-some-stuff = mypackage.__main__:main"
}
}
)
When I install my project using pip by calling pip install -e src/main/python/mynamespace-mypackage
, the package gets installed successfully. However, when I call do-some-stuff
from command line, I get ModuleNotFoundError: No module named 'mypackage'
.
I also tried modifying the console_scripts
to be:
entry_points={
"console_scripts": {
"do-some-stuff = mynamespace.mypackage.__main__:main"
}
}
but this also gives ModuleNotFoundError: No module named 'mynamespace.mypackage
when I run do-some-stuff
from the command line.
How should I define the entry point of my program to correctly point to the __main__:main
function of my package? Any help is much appreciated.
Potential Workaround:
Based on setuptools documentation, If I add __init__.py
to mynamespace
directory with the following content
__import__("pkg_resources").declare_namespace(__name__)
And modify setup.py
by adding namespace_packages=["mynamespace"]
, the entry points work.
I can confirm that this solves my problem. However, this is not ideal in my opinion. The Python packaging documentation explicitly states that when using native packaging, I should not add any __init__.py
to the namespace directory.
A minimal reproducible example:
src/main/python/mynamespace-mypackage/mynamespace/mypackage/__init__.py
:
#!/usr/bin/env python3
from .module_a import Hello
from .module_b import GoodBye
__all__ = ["Hello", "GoodBye"]
src/main/python/mynamespace-mypackage/mynamespace/mypackage/__main__.py
:
#!/usr/bin/env python3
from . import Hello, GoodBye
def main() -> None:
h = Hello()
h.say_hi()
g = GoodBye()
g.say_bye()
if __name__ == "__main__":
main()
src/main/python/mynamespace-mypackage/mynamespace/mypackage/module_a.py
:
#!/usr/bin/env python3
class Hello:
def say_hi(self) -> None:
print("Hello!")
src/main/python/mynamespace-mypackage/mynamespace/mypackage/module_b.py
:
#!/usr/bin/env python3
class GoodBye:
def say_bye(self) -> None:
print("Good Bye!")
Answers:
It turns out the issue was an out of date setuptools
library. I was using setuptools-41.2.0
. I upgraded to setuptools-46.1.3
and now my minimal example works as expected. There is no need for the workaround.
pip install -U setuptools
Using pkgutil-style namespace packages was the only way for me to get this to work even with setuptools-52.0.0
src/main/python/mynamespace-mypackage/mynamespace/__init__.py
:
__path__ = __import__("pkgutil").extend_path(__path__, __name__)
And adding the namespace to packages in setup.py
:
packages=["mynamespace", "mynamespace.mypackage"])
I’m unable to define console_scripts
for my Python package that utilizes the Python namespace packages structure.
I am following the Python namespace packages outlined in the Python packaging document. Specifically, I am using native namespace packaging in Python 3.8.2.
I have created my project like this:
src
└── main
└── python
└── mynamespace-mypackage
├── mynamespace
│ └── mypackage
│ ├── __init__.py
│ ├── __main__.py
│ ├── module_a.py
│ └── module_b.py
└── setup.py
In setup.py
, I write:
#!/usr/bin/env python3
from setuptools import setup, find_namespace_packages
PACKAGE = "mynamespace-mypackage"
VERSION = "0.1a"
setup(
name=PACKAGE,
version=VERSION,
packages=find_namespace_packages(include=["mynamespace.*"]),
description="My fancy package",
entry_points={
"console_scripts": {
"do-some-stuff = mypackage.__main__:main"
}
}
)
When I install my project using pip by calling pip install -e src/main/python/mynamespace-mypackage
, the package gets installed successfully. However, when I call do-some-stuff
from command line, I get ModuleNotFoundError: No module named 'mypackage'
.
I also tried modifying the console_scripts
to be:
entry_points={
"console_scripts": {
"do-some-stuff = mynamespace.mypackage.__main__:main"
}
}
but this also gives ModuleNotFoundError: No module named 'mynamespace.mypackage
when I run do-some-stuff
from the command line.
How should I define the entry point of my program to correctly point to the __main__:main
function of my package? Any help is much appreciated.
Potential Workaround:
Based on setuptools documentation, If I add __init__.py
to mynamespace
directory with the following content
__import__("pkg_resources").declare_namespace(__name__)
And modify setup.py
by adding namespace_packages=["mynamespace"]
, the entry points work.
I can confirm that this solves my problem. However, this is not ideal in my opinion. The Python packaging documentation explicitly states that when using native packaging, I should not add any __init__.py
to the namespace directory.
A minimal reproducible example:
src/main/python/mynamespace-mypackage/mynamespace/mypackage/__init__.py
:
#!/usr/bin/env python3
from .module_a import Hello
from .module_b import GoodBye
__all__ = ["Hello", "GoodBye"]
src/main/python/mynamespace-mypackage/mynamespace/mypackage/__main__.py
:
#!/usr/bin/env python3
from . import Hello, GoodBye
def main() -> None:
h = Hello()
h.say_hi()
g = GoodBye()
g.say_bye()
if __name__ == "__main__":
main()
src/main/python/mynamespace-mypackage/mynamespace/mypackage/module_a.py
:
#!/usr/bin/env python3
class Hello:
def say_hi(self) -> None:
print("Hello!")
src/main/python/mynamespace-mypackage/mynamespace/mypackage/module_b.py
:
#!/usr/bin/env python3
class GoodBye:
def say_bye(self) -> None:
print("Good Bye!")
It turns out the issue was an out of date setuptools
library. I was using setuptools-41.2.0
. I upgraded to setuptools-46.1.3
and now my minimal example works as expected. There is no need for the workaround.
pip install -U setuptools
Using pkgutil-style namespace packages was the only way for me to get this to work even with setuptools-52.0.0
src/main/python/mynamespace-mypackage/mynamespace/__init__.py
:
__path__ = __import__("pkgutil").extend_path(__path__, __name__)
And adding the namespace to packages in setup.py
:
packages=["mynamespace", "mynamespace.mypackage"])