How to use setup.py console_script with native namespace packages

Question:

I’m unable to define console_scripts for my Python package that utilizes the Python namespace packages structure.

I am following the Python namespace packages outlined in the Python packaging document. Specifically, I am using native namespace packaging in Python 3.8.2.

I have created my project like this:

src
└── main
   └── python
      └── mynamespace-mypackage
         ├── mynamespace
         │  └── mypackage
         │     ├── __init__.py
         │     ├── __main__.py
         │     ├── module_a.py
         │     └── module_b.py
         └── setup.py

In setup.py, I write:


#!/usr/bin/env python3

from setuptools import setup, find_namespace_packages

PACKAGE = "mynamespace-mypackage"
VERSION = "0.1a"

setup(
    name=PACKAGE,
    version=VERSION,
    packages=find_namespace_packages(include=["mynamespace.*"]),
    description="My fancy package",
    entry_points={
        "console_scripts": {
            "do-some-stuff = mypackage.__main__:main"
        }
    }
)

When I install my project using pip by calling pip install -e src/main/python/mynamespace-mypackage, the package gets installed successfully. However, when I call do-some-stuff from command line, I get ModuleNotFoundError: No module named 'mypackage'.

I also tried modifying the console_scripts to be:


    entry_points={
        "console_scripts": {
            "do-some-stuff = mynamespace.mypackage.__main__:main"
        }
    }

but this also gives ModuleNotFoundError: No module named 'mynamespace.mypackage when I run do-some-stuff from the command line.

How should I define the entry point of my program to correctly point to the __main__:main function of my package? Any help is much appreciated.

Potential Workaround:

Based on setuptools documentation, If I add __init__.py to mynamespace directory with the following content

__import__("pkg_resources").declare_namespace(__name__)

And modify setup.py by adding namespace_packages=["mynamespace"], the entry points work.
I can confirm that this solves my problem. However, this is not ideal in my opinion. The Python packaging documentation explicitly states that when using native packaging, I should not add any __init__.py to the namespace directory.

A minimal reproducible example:

src/main/python/mynamespace-mypackage/mynamespace/mypackage/__init__.py:

#!/usr/bin/env python3

from .module_a import Hello
from .module_b import GoodBye

__all__ = ["Hello", "GoodBye"]

src/main/python/mynamespace-mypackage/mynamespace/mypackage/__main__.py:

#!/usr/bin/env python3

from . import Hello, GoodBye

def main() -> None:
    h = Hello()
    h.say_hi()
    g = GoodBye()
    g.say_bye()

if __name__ == "__main__":
    main()

src/main/python/mynamespace-mypackage/mynamespace/mypackage/module_a.py:

#!/usr/bin/env python3


class Hello:
    def say_hi(self) -> None:
        print("Hello!")

src/main/python/mynamespace-mypackage/mynamespace/mypackage/module_b.py:

#!/usr/bin/env python3


class GoodBye:
    def say_bye(self) -> None:
        print("Good Bye!")

Asked By: Ashkan

||

Answers:

It turns out the issue was an out of date setuptools library. I was using setuptools-41.2.0. I upgraded to setuptools-46.1.3 and now my minimal example works as expected. There is no need for the workaround.

pip install -U setuptools
Answered By: Ashkan

Using pkgutil-style namespace packages was the only way for me to get this to work even with setuptools-52.0.0

src/main/python/mynamespace-mypackage/mynamespace/__init__.py:

__path__ = __import__("pkgutil").extend_path(__path__, __name__)

And adding the namespace to packages in setup.py:

packages=["mynamespace", "mynamespace.mypackage"])
Answered By: Matthias Kuhn