How to use poetry with docker?

Question:

How do I install poetry in my image? (should I use pip?)

Which version of poetry should I use?

Do I need a virtual environment?

There are many examples and opinions in the wild which offer different solutions.

Asked By: Soof Golan

||

Answers:

TL;DR

Install poetry with pip, configure virtualenv, install dependencies, run your app.

FROM python:3.10

# Configure Poetry
ENV POETRY_VERSION=1.2.0
ENV POETRY_HOME=/opt/poetry
ENV POETRY_VENV=/opt/poetry-venv
ENV POETRY_CACHE_DIR=/opt/.cache

# Install poetry separated from system interpreter
RUN python3 -m venv $POETRY_VENV 
    && $POETRY_VENV/bin/pip install -U pip setuptools 
    && $POETRY_VENV/bin/pip install poetry==${POETRY_VERSION}

# Add `poetry` to PATH
ENV PATH="${PATH}:${POETRY_VENV}/bin"

WORKDIR /app

# Install dependencies
COPY poetry.lock pyproject.toml ./
RUN poetry install

# Run your app
COPY . /app
CMD [ "poetry", "run", "python", "-c", "print('Hello, World!')" ]

In Detail

Installing Poetry

How do I install poetry in my image? (should I use pip?)

Install it with pip

You should install poetry with pip. but you need to isolate it from the system interpreter and the project’s virtual environment.

For maximum control in your CI environment, installation with pip is fully supported … offers the best debugging experience, and leaves you subject to the fewest external tools.

ENV POETRY_VERSION=1.2.0
ENV POETRY_VENV=/opt/poetry-venv

# Install poetry separated from system interpreter
RUN python3 -m venv $POETRY_VENV 
    && $POETRY_VENV/bin/pip install -U pip setuptools 
    && $POETRY_VENV/bin/pip install poetry==${POETRY_VERSION}

# Add `poetry` to PATH
ENV PATH="${PATH}:${POETRY_VENV}/bin"

Poetry Version

Which version of poetry should I use?

Specify the latest stable version explicitly in your installation.

Forgetting to specify POETRY_VERSION will result in undeterministic builds, as the installer will always install the latest version – which may introduce breaking changes

Virtual Environment (virtualenv)

Do I need a virtual environment?

Yes, and you need to configure it a bit.

ENV POETRY_CACHE_DIR=/opt/.cache

The reasons for this are somewhat off topic:

By default, poetry creates a virtual environment in $HOME/.cache/pypoetry/virtualenvs to isolate the system interpreter from your application. This is the desired behavior for most development scenarios. When using a container, the $HOME variable may be changed by certain runtimes, so creating the virtual environment in an independent directory solves any reproducibility issues that may arise.

Bringing It All Together

To use poetry in a docker image you need to:

  1. Install your desired version of poetry
  2. Configure virtual environment location
  3. Install your dependencies
  4. Use poetry run python ... to run your application

A Working Example:

This is a minimal flask project managed with poetry.

You can copy these contents to your machine to test it out (expect for poerty.lock)

Project structure

python-poetry-docker/
|- Dockerfile
|- app.py
|- pyproject.toml
|- poetry.lock

Dockerfile

FROM python:3.10 as python-base

# https://python-poetry.org/docs#ci-recommendations
ENV POETRY_VERSION=1.2.0
ENV POETRY_HOME=/opt/poetry
ENV POETRY_VENV=/opt/poetry-venv

# Tell Poetry where to place its cache and virtual environment
ENV POETRY_CACHE_DIR=/opt/.cache

# Create stage for Poetry installation
FROM python-base as poetry-base

# Creating a virtual environment just for poetry and install it with pip
RUN python3 -m venv $POETRY_VENV 
    && $POETRY_VENV/bin/pip install -U pip setuptools 
    && $POETRY_VENV/bin/pip install poetry==${POETRY_VERSION}

# Create a new stage from the base python image
FROM python-base as example-app

# Copy Poetry to app image
COPY --from=poetry-base ${POETRY_VENV} ${POETRY_VENV}

# Add Poetry to PATH
ENV PATH="${PATH}:${POETRY_VENV}/bin"

WORKDIR /app

# Copy Dependencies
COPY poetry.lock pyproject.toml ./

# [OPTIONAL] Validate the project is properly configured
RUN poetry check

# Install Dependencies
RUN poetry install --no-interaction --no-cache --without dev

# Copy Application
COPY . /app

# Run Application
EXPOSE 5000
CMD [ "poetry", "run", "python", "-m", "flask", "run", "--host=0.0.0.0" ]

app.py

from flask import Flask
app = Flask(__name__)

@app.route('/')
def hello_world():
    return 'Hello, Docker!'

pyproject.toml

[tool.poetry]
name = "python-poetry-docker-example"
version = "0.1.0"
description = ""
authors = ["Someone <[email protected]>"]

[tool.poetry.dependencies]
python = "^3.10"
Flask = "^2.1.2"

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

poetry.lock

[[package]]
name = "click"
version = "8.1.3"
description = "Composable command line interface toolkit"
category = "main"
optional = false
python-versions = ">=3.7"

[package.dependencies]
... more lines ommitted

Full contents in gist.

Answered By: Soof Golan

I prefer to use multistage builds so I can get rid of poetry in my actual release images and keep those images slim.

FROM python:3.10-slim AS builder

ENV POETRY_HOME="/opt/poetry" 
    POETRY_VIRTUALENVS_IN_PROJECT=1 
    POETRY_NO_INTERACTION=1

# to run poetry directly as soon as it's installed
ENV PATH="$POETRY_HOME/bin:$PATH"

# install poetry
RUN apt-get update 
    && apt-get install -y --no-install-recommends curl 
    && curl -sSL https://install.python-poetry.org | python3 -

WORKDIR /app

# copy only pyproject.toml and poetry.lock file nothing else here
COPY poetry.lock pyproject.toml ./

# this will create the folder /app/.venv (might need adjustment depending on which poetry version you are using)
RUN poetry install --no-root --no-ansi --without dev

# ---------------------------------------------------------------------

FROM python:3.10-slim

ENV PYTHONDONTWRITEBYTECODE=1 
    PYTHONUNBUFFERED=1 
    PATH="/app/.venv/bin:$PATH"

WORKDIR /app

# copy the venv folder from builder image 
COPY --from=builder /app/.venv ./.venv
Answered By: Sven Becker

@Soof Golan’s answer got me 99% of the way there, thank you!

I ran into an issue where the virtualenv created by poetry install was broken, which lead to the CMD creating a new venv.

The virtual environment found in /app/.venv seems to be broken.
Recreating virtualenv thatchbot-9TtSrW0h-py3.11 in /opt/.cache/virtualenvs/thatchbot-9TtSrW0h-py3.11
Command not found: gunicorn

I was able to work around this by turning poetry’s virtualenv off.

RUN poetry config virtualenvs.create false

My thinking being we already installed poetry in a virtualenv so the nesting must be causing the issue, but I’m having trouble testing this theory out. Is this defeating the point of having poetry?

Answered By: Karun Kannan