Why is pip editable install causing issue with subpackages?
Question:
The Problem
I’m having an issue using editable install for my package. Inside my package I have three subpackages as shown in my setup.py file. Everything works fine if I install with pip install .
, but if I install with pip install -e .
then I’ll get the following error when attempting to run the CLI relctl
:
Traceback (most recent call last):
File "/Users/my_username/.virtualenvs/rawp3/bin/relctl", line 11, in <module>
load_entry_point('tech-control', 'console_scripts', 'relctl')()
File "/Users/my_username/.virtualenvs/rawp3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 490, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/Users/my_username/.virtualenvs/rawp3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2859, in load_entry_point
return ep.load()
File "/Users/my_username/.virtualenvs/rawp3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2450, in load
return self.resolve()
File "/Users/my_username/.virtualenvs/rawp3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2456, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
ModuleNotFoundError: No module named 'tech_control.cli'
On further inspection I find that I’m not able to import any subpackages. E.g. The following will fail:
import tech_control.boards
Fails with: ModuleNotFoundError: No module named 'tech_control.boards'
My Package Directory
tech_control/
├── __init__.py
├── boards
│ ├── __init__.py
│ ├── board_base.py
│ ├── tech8ch.py
│ └── wiring.py
├── cli
│ ├── __init__.py
│ └── cli.py
└── setup.py
Setup.py
from setuptools import setup
setup(
name="tech_control",
version="0.1.0",
description="CLI for Tech Boards",
packages=["tech_control", "tech_control.boards", "tech_control.cli"],
package_dir={
"tech_control": "",
"tech_control.boards": "boards",
"tech_control.cli": "cli"},
install_requires=["pyusb"],
entry_points={
"console_scripts": [
"relctl = tech_control.cli.cli:main"
]
},
)
Question
Why is my editable install preventing me from accessing subpackages?
Answers:
As far as I know, editable installations do not work with changes to the directory structure, as done with package_dir
.
Changing the root directory is OK, though:
package_dir={'': 'path/to/root'}
References:
- https://github.com/pypa/pip/issues/3160
- https://github.com/pypa/setuptools/issues/463
- https://github.com/pypa/setuptools/issues/230
- https://docs.python.org/3/distutils/setupscript.html?highlight=package_dir#listing-whole-packages
- https://docs.python.org/3/distutils/examples.html?highlight=package_dir#pure-python-distribution-by-package
- https://setuptools.readthedocs.io/en/latest/setuptools.html?highlight=package_dir#using-find-packages
- https://setuptools.readthedocs.io/en/latest/setuptools.html?highlight=package_dir#using-a-src-layout
Aside:
There is currently an ongoing effort towards a new design for editable installations. Most likely the combination with the rewrite of the directory structure is probably not in scope, as it’s probably not a use case that comes up often enough to make it worth the additional code complexity to support this.
The Problem
I’m having an issue using editable install for my package. Inside my package I have three subpackages as shown in my setup.py file. Everything works fine if I install with pip install .
, but if I install with pip install -e .
then I’ll get the following error when attempting to run the CLI relctl
:
Traceback (most recent call last):
File "/Users/my_username/.virtualenvs/rawp3/bin/relctl", line 11, in <module>
load_entry_point('tech-control', 'console_scripts', 'relctl')()
File "/Users/my_username/.virtualenvs/rawp3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 490, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/Users/my_username/.virtualenvs/rawp3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2859, in load_entry_point
return ep.load()
File "/Users/my_username/.virtualenvs/rawp3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2450, in load
return self.resolve()
File "/Users/my_username/.virtualenvs/rawp3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2456, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
ModuleNotFoundError: No module named 'tech_control.cli'
On further inspection I find that I’m not able to import any subpackages. E.g. The following will fail:
import tech_control.boards
Fails with: ModuleNotFoundError: No module named 'tech_control.boards'
My Package Directory
tech_control/
├── __init__.py
├── boards
│ ├── __init__.py
│ ├── board_base.py
│ ├── tech8ch.py
│ └── wiring.py
├── cli
│ ├── __init__.py
│ └── cli.py
└── setup.py
Setup.py
from setuptools import setup
setup(
name="tech_control",
version="0.1.0",
description="CLI for Tech Boards",
packages=["tech_control", "tech_control.boards", "tech_control.cli"],
package_dir={
"tech_control": "",
"tech_control.boards": "boards",
"tech_control.cli": "cli"},
install_requires=["pyusb"],
entry_points={
"console_scripts": [
"relctl = tech_control.cli.cli:main"
]
},
)
Question
Why is my editable install preventing me from accessing subpackages?
As far as I know, editable installations do not work with changes to the directory structure, as done with package_dir
.
Changing the root directory is OK, though:
package_dir={'': 'path/to/root'}
References:
- https://github.com/pypa/pip/issues/3160
- https://github.com/pypa/setuptools/issues/463
- https://github.com/pypa/setuptools/issues/230
- https://docs.python.org/3/distutils/setupscript.html?highlight=package_dir#listing-whole-packages
- https://docs.python.org/3/distutils/examples.html?highlight=package_dir#pure-python-distribution-by-package
- https://setuptools.readthedocs.io/en/latest/setuptools.html?highlight=package_dir#using-find-packages
- https://setuptools.readthedocs.io/en/latest/setuptools.html?highlight=package_dir#using-a-src-layout
Aside:
There is currently an ongoing effort towards a new design for editable installations. Most likely the combination with the rewrite of the directory structure is probably not in scope, as it’s probably not a use case that comes up often enough to make it worth the additional code complexity to support this.