How to configure a PYTHONPATH env variable in the Dockerfile?

Question

I am modifying the Machine Learning Inference to perform inference using my trained model and to this it is need to import a set of modules. So the project tree is now:

.
├── Dockerfile
├── __init__.py
├── app.py
├── requirements.txt
**
└── maskrcnn
    ├── config.py
    ├── __init__.py
    ├── m_rcnn.py
    ├── visualize.py
    ├── mask_rcnn_coco.h5
    └── model.py
**

Dockerfile:

# Pull the base image with python 3.8 as a runtime for your Lambda
FROM public.ecr.aws/lambda/python:3.8

# Copy the earlier created requirements.txt file to the container
COPY requirements.txt ./

# Install the python requirements from requirements.txt
RUN python3.8 -m pip install -r requirements.txt 

# Create the model directory for mounting the EFS 
RUN mkdir -p /mnt/ml

# Copy the earlier created app.py file to the container
COPY app.py ./
COPY maskrcnn/ ./maskrcnn


# Set the CMD to your handler
CMD ["app.lambda_handler"]

To import the python modules in app.py I used the following command line:

from maskrcnn.m_rcnn import *
from maskrcnn.visualize import random_colors, get_mask_contours, draw_mask

When running my docker image, I get an import error:

{"errorMessage": "Unable to import module 'app': No module named 'maskrcnn'", 

"errorType": "Runtime.ImportModuleError", "stackTrace": []}

My other problem is that in the module m_rcnn.py the file mask_rcnn_coco.h5 is accessed to do some procedures.

m_rcnn.py:

ROOT_DIR = os.path.abspath("/maskrcnn")


# Local path to trained weights file
COCO_MODEL_PATH = os.path.join(ROOT_DIR, "mask_rcnn_coco.h5")

Because of this code I am getting the following error:

{"errorMessage": "[Errno 2] No such file or directory: 

'/maskrcnn/mask_rcnn_coco.h5'", "errorType": "FileNotFoundError"},

Asked By: BodeTech

||

Source

Answer 1

You can try to add this lines before importing maskrcnn:


import sys
from pathlib import Path

sys.path.append(str(Path(__file__).parents[0]))

from maskrcnn.m_rcnn import *
from maskrcnn.visualize import random_colors, get_mask_contours, draw_mask

This way, the module maskrcnn will be searched at the same level as app.py so the imports should work correctly.

Answered By: Cuartero

Answer 2

PYTHONPATH tells Python where to look for modules and packages.

The /maskrcnn directory is itself a package due to the presence of the __init__.py file.

Therefore, adding ./maskrcnn to PYTHONPATH accomplishes nothing! You need the parent directory of maskrcnn added to the Python search path, which is just the current directory ., which is usually added to the Python search path by default.

Furthermore, you added the directory /maskrcnn to PYTHONPATH, and not ./maskrcnn. As far as I can tell from the Dockerfile, the former directory doesn’t even exist.

As per the answer here, all this is moot anyway because you forgot to copy the maskrcnn directory in to the image! ~~I actually don’t know the cause of the error as-written; that will require further debugging. But~~ Once you fix that problem, this Dockerfile should work as-is if you simply remove the ENV PYTHONPATH directive.

If that still doesn’t work, then possibly . is not in the package search path, so try ENV PYTHONPATH "." to ensure that it is added.

Answered By: shadowtalker

Answer 3

If you want to use the Python module maskrcnn from your lokal working directory, you have to copy it into the Docker image. Adding a COPY statement for maskrcnn/* could solve the issue:

# Dockerfile

# Copy the earlier created app.py file to the container
COPY app.py ./
COPY maskrcnn/ ./maskrcnn

And please remove the line ENV PYTHONPATH.... I don’t see any indication to mess around with PYTHONPATH.

Answered By: Konstantin A. Magg

How to configure a PYTHONPATH env variable in the Dockerfile?

Question:

Answers: