Can't do Python imports from another dir
Question:
I’m unable to import from Python file in another directory. Directory structure:
some_root/
- __init__.py
- dir_0/
- __init__.py
- dir_1/
- __init__.py
- file_1.py
- dir_2/
- __init__.py
- file_2.py
file_1.py has some exported member:
# file_1.py
def foo():
pass
file_2.py tries to import member from file_1.py:
# file_2.py
from dir_0.dir_1.file_1 import foo
But not absolute nor relative import seems work. How to do Python’s imports correctly? If I could avoid using sys.path.insert it would be nice, but if there’s no way around this then I guess that’s how things stand.
Answers:
By default, the Python search path includes directories that are effectively fixed when Python is installed: places like /usr/lib/python/site-packages.
When you run a script, the directory containing the script being executed is added to the path. Most importantly here, the current working directory is not added to the path, for security purposes. (You don’t want your script to work differently, possibly importing modules you did not expect, based on its calling environment. This is the same reason it is strongly recommended to keep . out of your PATH variable.)
When you execute file_2.py, some_root/dir_0/dir_2 is added to the path, but if some_root is the current working directory, some_root itself is not on the search path. That’s why import dir_0.dir_1.file_1 fails. A relative import fails as well because some_root is not the root (or any) package used to resolve the relative import.
Typically, if you wanted to run your code without installing it, you keep your scripts in some_root itself:
some_root/
file_a.py
- __init__.py
- dir_0/
- __init__.py
- dir_1/
- __init__.py
- file_1.py
- dir_2/
- __init__.py
file_2.py
file_a.py can be as simple as
from dir_0.dir_2.file_2 import main
if __name__ == '__main__':
main()
with file_2 providing an entry point rather than being directly executable itself. (It still can be executable itself; it just uses main the same as any other script importing file_2.py rather than just jumping straight into its code.)
Don’t mess with the search path
You are right not messing around with sys.path. This is not recommended and always just an ugly workaround. There are better solutions for this.
Restructure your folder layout
See official Python docs about packaging.
Distinguish between project folder and package folder. We assume your some_root folder as the package folder which is also the name of the package (used in import statements). And it is recommended to put the package folder into a folder named src. Above it is the project folder some_project. This project folder layout is also known as the "Src-Layout".
In your case it should look like this.
some_project
└── src
└── some_root
├── dir_0
│ ├── dir_1
│ │ ├── file_1.py
│ │ └── __init__.py
│ ├── dir_2
│ │ ├── file_2.py
│ │ └── __init__.py
│ └── __init__.py
└── __init__.py
Make your package installable
Create a some_project/setup.cfg with that content. Keep the line breaks and indention in line 5 and 6. They have to be like this but I don’t know why.
[metadata]
name = some_project
[options]
package_dir=
=src
packages = find:
zip_safe = False
python_requires = >= 3
[options.packages.find]
where = src
exclude =
tests*
.gitignore
Create some_project/setup.py with that content:
from setuptools import setup
setup()
"Install" the package
This is not a usual installation. Please see Developement Mode to understand what this really means. The package is not copied into /usr/lib/python/site-packages; only links are created.
Navigate into the project folder some_project and run
python3 -m pip install --editable .
Don’t forget the . at the end. Depending on your OS and environment maybe you have to replace python3 with py -3 or python or something else.
Import
Your file_2.py
import some_root
import some_root.dir_0
import some_root.dir_0.dir_1
from some_root.dir_0.dir_1 import file_1
file_1.foo()
But as others said in the comments. Improve your structure of files and folders and reduce its complexity.
I’m unable to import from Python file in another directory. Directory structure:
some_root/
- __init__.py
- dir_0/
- __init__.py
- dir_1/
- __init__.py
- file_1.py
- dir_2/
- __init__.py
- file_2.py
file_1.py has some exported member:
# file_1.py
def foo():
pass
file_2.py tries to import member from file_1.py:
# file_2.py
from dir_0.dir_1.file_1 import foo
But not absolute nor relative import seems work. How to do Python’s imports correctly? If I could avoid using sys.path.insert it would be nice, but if there’s no way around this then I guess that’s how things stand.
By default, the Python search path includes directories that are effectively fixed when Python is installed: places like /usr/lib/python/site-packages.
When you run a script, the directory containing the script being executed is added to the path. Most importantly here, the current working directory is not added to the path, for security purposes. (You don’t want your script to work differently, possibly importing modules you did not expect, based on its calling environment. This is the same reason it is strongly recommended to keep . out of your PATH variable.)
When you execute file_2.py, some_root/dir_0/dir_2 is added to the path, but if some_root is the current working directory, some_root itself is not on the search path. That’s why import dir_0.dir_1.file_1 fails. A relative import fails as well because some_root is not the root (or any) package used to resolve the relative import.
Typically, if you wanted to run your code without installing it, you keep your scripts in some_root itself:
some_root/
file_a.py
- __init__.py
- dir_0/
- __init__.py
- dir_1/
- __init__.py
- file_1.py
- dir_2/
- __init__.py
file_2.py
file_a.py can be as simple as
from dir_0.dir_2.file_2 import main
if __name__ == '__main__':
main()
with file_2 providing an entry point rather than being directly executable itself. (It still can be executable itself; it just uses main the same as any other script importing file_2.py rather than just jumping straight into its code.)
Don’t mess with the search path
You are right not messing around with sys.path. This is not recommended and always just an ugly workaround. There are better solutions for this.
Restructure your folder layout
See official Python docs about packaging.
Distinguish between project folder and package folder. We assume your some_root folder as the package folder which is also the name of the package (used in import statements). And it is recommended to put the package folder into a folder named src. Above it is the project folder some_project. This project folder layout is also known as the "Src-Layout".
In your case it should look like this.
some_project
└── src
└── some_root
├── dir_0
│ ├── dir_1
│ │ ├── file_1.py
│ │ └── __init__.py
│ ├── dir_2
│ │ ├── file_2.py
│ │ └── __init__.py
│ └── __init__.py
└── __init__.py
Make your package installable
Create a some_project/setup.cfg with that content. Keep the line breaks and indention in line 5 and 6. They have to be like this but I don’t know why.
[metadata]
name = some_project
[options]
package_dir=
=src
packages = find:
zip_safe = False
python_requires = >= 3
[options.packages.find]
where = src
exclude =
tests*
.gitignore
Create some_project/setup.py with that content:
from setuptools import setup
setup()
"Install" the package
This is not a usual installation. Please see Developement Mode to understand what this really means. The package is not copied into /usr/lib/python/site-packages; only links are created.
Navigate into the project folder some_project and run
python3 -m pip install --editable .
Don’t forget the . at the end. Depending on your OS and environment maybe you have to replace python3 with py -3 or python or something else.
Import
Your file_2.py
import some_root
import some_root.dir_0
import some_root.dir_0.dir_1
from some_root.dir_0.dir_1 import file_1
file_1.foo()
But as others said in the comments. Improve your structure of files and folders and reduce its complexity.