Can't figure out the correct python project files hierarchy

Question

I have a python project with hierarchy kind of like this: (but with more folders under src)

ip-rep/
│
├── ip_rep/
│   ├── __init__.py
│   ├── run.py
│   ├── src/
│       ├── __init__.py
│       ├── cli_tool/
│           ├── __init__.py
│           ├---- cli_loader.py
├── tests/
│   ├── test_cli_loader
│       ├── test_cli_loader.py
│
├── dockefile
├── bash_start.sh

now in the dockerfile I have something like this (simplified):

RUN mkdir app
WORKDIR /app
ADD ip_rep /app/ip_rep
ADD load_ips.sh /app/
RUN chmod +x /app/load_ips.sh
ENTRYPOINT ["/bin/bash", "load_ips.sh"]

and this is the bash file (again simplified):

python ip_rep/run.py

and this is an example of an import in run.py file:

from ip_rep.src.cli_tool.cli_loader import CliTool

When I try to run it I get the error:

Traceback (most recent call last):
  File "ip_rep/run.py", line 9, in <module>
    from ip_rep.src.cli_tool.cli_loader import CliTool
ModuleNotFoundError: No module named 'ip_rep'

I can’t really understand what I did wrong.
I can’t remove the ip_rep in run.py and do only src. file because then the tests stop working.

What am I missing?

Asked By: Ema Il

||

Source

Answer 1

Use this structure instead

ip-rep/
│
├── ip_rep/
│   ├── __init__.py
│   ├── cli_tool/
│       ├── __init__.py
│       ├---- cli_loader.py
├── scripts/
│   ├── run.py
│
├── tests/
│   ├── test_cli_loader
│       ├── test_cli_loader.py
│
├── dockerfile
├── bash_start.sh

(You can ignore the ${SOMEDIR}/ part, but it’s needed as example below.)

That moves run.py into a separate directory at the same level as the ip_rep package directory, but not inside it; similar to the tests.

It also removes the unnecessary src/ directory, as commented by Lenormju.

Your run.py file should now have the import line:

from ip_rep.cli_tool.cli_loader import CliTool

and your bash_start.sh should (at least) have something like

PYTHONPATH=app/ip_rep

python app/ip_rep/scripts/run.py

It is not unlikely that you don’t need to set PYTHONPATH, if your bash script is already in the app/ip_rep directory: Python will automatically put the current directory on its search path. The script could also then be relative, and the whole thing becomes

python scripts/run.py

The first variant is just more explicit, and allows for putting the bash script elsewhere.

Naturally, the best part is to install the package and script(s) properly, so that they are on the system PATH and PYTHONPATH. That would require something like pip install . to be run, with a valid setup.py or pyproject.toml file for the project. With that, the bash script becomes simply run.py (just that one line) or even run, depending on how you configured things. Of course, "run" is a generic and bad name for a script, but that should be what you could be working towards.

Answered By: 9769953

Can't figure out the correct python project files hierarchy

Question:

Answers: