Python relative import: ModuleNotFoundError: No module named 'sib1'
Question:
I have a project hierarchy like below, when I run python src/bot/main I didn’t get error. While if I run python -m src.bot.main I got an error. Why?
This is my file hierarchy:
MyProject
└── src
├── __init__.py
├── bot
│ ├── __init__.py
│ ├── main.py
│ └── sib1.py
└── mod
├── __init__.py
└── module1.py
This is the content of main.py:
import sys
if __name__ == "__main__":
# sys.path will print the parent folder.
print(sys.path, end="nn")
# my problem is on this line.
import sib1
sib1.test()
The error:
Traceback (most recent call last):
File "/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/Users/me/Desktop/test_py/src/bot/main.py", line 16, in <module>
import sib1
ModuleNotFoundError: No module named 'sib1'
Some conclusion I’ve made so far:
Since the output of sys.path in both cases include /Users/me/Desktop/MyProject, the reason should not related to scope?
The output of sys.path of both python -m src.bot.main and python src/bot/main:
(test) ✔ me Desktop/test_py % python -m src.bot.main
['/Users/me/Desktop/test_py', '/usr/local/Caskroom/miniconda/base/envs/test/lib/python39.zip', '/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.9', '/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.9/lib-dynload', '/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.9/site-packages']
Answers:
I will try my best to clarify each of my confusions in a Q&A form, and organize @Brain’s comments along the way:
Q1: when I run python src/bot/main I got no error.
The sys.path will include the current directory containing the file main.py, i.e. the interpreter will see the file MyProject/src/bot:
import sib1
is logically equivalent to:
import "MyProject/src/bot" + "/sib1.py"
Hence, no error.
Q2: While if I run python -m src.bot.main I got error. Why?
Now it’s time to quote @Brain’s valuable (first) comment:
Using python -m src.bot.main tells Python that src is a top-level package. Everything below src in the directory structure will be considered submodules/subpackages of src. The proper name for sib1 under that organization is src.bot.sib1. No top-level module named sib1 exists as far as Python is concerned.
(emphasis by me)
So:
- The
-m option will define the scope (and thus the top-level package) for the file you’re going to run.
- All packages not starting from
src. will be regarded as built-in or third-party libraries installed in your virtual environment. Since I didn’t install any package called sib1, you got the error. (the sys.path is intentionally ignored in this case, as we there is no implicitly relative import starting from Python 3.3)
- All your relative imports should not go outside of the top-level package
src (by prepending too many .‘s in the import).
For example, this will work:
from . import sib1
from ..mod import module1 # The `..` is equivalent to the MyProject/src.
module1.hello()
sib1.test()
Finally, don’t test your package by inserting many if __name__ == '__main__'. Do so by professional tools:
If you only need to run submodules for testing purpose, you could consider using more robust testing tools like doctest (lightweight) or unittest (heavyweight).
This is a good read I had put 300 bounty on it years ago, you must find a time to read it.
I have a project hierarchy like below, when I run python src/bot/main I didn’t get error. While if I run python -m src.bot.main I got an error. Why?
This is my file hierarchy:
MyProject
└── src
├── __init__.py
├── bot
│ ├── __init__.py
│ ├── main.py
│ └── sib1.py
└── mod
├── __init__.py
└── module1.py
This is the content of main.py:
import sys
if __name__ == "__main__":
# sys.path will print the parent folder.
print(sys.path, end="nn")
# my problem is on this line.
import sib1
sib1.test()
The error:
Traceback (most recent call last):
File "/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/Users/me/Desktop/test_py/src/bot/main.py", line 16, in <module>
import sib1
ModuleNotFoundError: No module named 'sib1'
Some conclusion I’ve made so far:
Since the output of
sys.pathin both cases include/Users/me/Desktop/MyProject, the reason should not related to scope?
The output of sys.path of both python -m src.bot.main and python src/bot/main:
(test) ✔ me Desktop/test_py % python -m src.bot.main
['/Users/me/Desktop/test_py', '/usr/local/Caskroom/miniconda/base/envs/test/lib/python39.zip', '/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.9', '/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.9/lib-dynload', '/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.9/site-packages']
I will try my best to clarify each of my confusions in a Q&A form, and organize @Brain’s comments along the way:
Q1: when I run python src/bot/main I got no error.
The sys.path will include the current directory containing the file main.py, i.e. the interpreter will see the file MyProject/src/bot:
import sib1
is logically equivalent to:
import "MyProject/src/bot" + "/sib1.py"
Hence, no error.
Q2: While if I run python -m src.bot.main I got error. Why?
Now it’s time to quote @Brain’s valuable (first) comment:
Using
python -m src.bot.maintells Python thatsrcis a top-level package. Everything below src in the directory structure will be considered submodules/subpackages of src. The proper name forsib1under that organization issrc.bot.sib1. No top-level module named sib1 exists as far as Python is concerned.
(emphasis by me)
So:
- The
-moption will define the scope (and thus the top-level package) for the file you’re going to run. - All packages not starting from
src.will be regarded as built-in or third-party libraries installed in your virtual environment. Since I didn’t install any package calledsib1, you got the error. (thesys.pathis intentionally ignored in this case, as we there is no implicitly relative import starting from Python 3.3) - All your relative imports should not go outside of the top-level package
src(by prepending too many.‘s in theimport).
For example, this will work:
from . import sib1
from ..mod import module1 # The `..` is equivalent to the MyProject/src.
module1.hello()
sib1.test()
Finally, don’t test your package by inserting many if __name__ == '__main__'. Do so by professional tools:
If you only need to run submodules for testing purpose, you could consider using more robust testing tools like doctest (lightweight) or unittest (heavyweight).
This is a good read I had put 300 bounty on it years ago, you must find a time to read it.