python argparse in a docker container based on ubuntu

Question

I am struggling to find a way to pass arguments to a python script within a docker container based on ubuntu. I am working with docker-compose.yml.

Please find the example below!

docker-compose.yml

version: "3"
services:
   bcp:
      image: ubuntu:18.04
      restart: always
      tty: true
      entrypoint: ["/bin/bash", "/ingestion/bcp-entrypoint.sh"]
      volumes:
          - ./services/bcp:/ingestion/services/bcp
          - ./bcp-entrypoint.sh:/ingestion/bcp-entrypoint.sh

bcp-entrypoint.sh

apt-get update
apt-get upgrade -y
apt-get clean -y
apt-get install -y python3-pip
...

python script

required_args.add_argument("--database", metavar="str", type=str, help="database from where to extract", required=True)

The way I call the script – in container and on host machine – is python3 -m services.bcp --database foo and it works just perfect. The question is, how can I achieve the same from host machine on docker container?

Basically, I am looking for something like docker-compose exec services.bcp --database foo.

I do not want to use dockerfile! Ideally everything is based on docker-compose.

Asked By: abe

||

Source

Answer 1

Here is a solution for both docker and docker compose of how to parse arguments within the docker container:

python script

#!/usr/bin/env python
import argparse

parser = argparse.ArgumentParser()
parser.add_argument("name", help="Name of person to greet")

args = parser.parse_args()

print(f"Hello there, {args.name}!")

With a custom dockerfile (seems like the most straight-forward for this situation):

FROM python:3.8

COPY main.py .

ENTRYPOINT ["./main.py"]

Then build and run the containerized cli:

# Should make the script executable to invoke directly
$ chmod +x main.py

$ docker build -t dockerized-cli .
$ docker run -it --rm dockerized-cli -h
usage: main.py [-h] name

positional arguments:
  name        Name of person to greet

optional arguments:
  -h, --help  show this help message and exit
$ docker run -it --rm dockerized-cli Jeff
Hello there, Jeff!

With just docker-compose (Original question)

version: "3"
services:
  app:
    image: python:3.8
    volumes:
      - .:/opt/app
    entrypoint: ["/opt/app/main.py"]

Then to run,

$ docker-compose run app Jeff
Hello there, Jeff!

If you need to use the ubuntu:18.04 image and still can’t build your own image off of it (to install python outside of the entrypoint), then you need to do what you already have which is create an entrypoint script wherein you first install python and then invoke your script. Something of note is that now your entrypoint is the entrypoint script itself and not your python CLI so you need to propogate any arguments from the shell script to your python script. Below is a simple example of how to do so – notice the bash variable "$@":

entrypoint.sh

#!/bin/bash

# install python here...

python print_args.py $@

print_args.py

import sys
print(sys.argv)

Answered By: ccchoy

Answer 2

The only solution I found is the following: you can pass on startup(passing arguments on building is much easier) arguments from docker-compose to Dockerfile via ARG keyword.
The issue here is that ARG has a very bizarre scope, so ENTRYPOINT cannot see defined ARGs, but does see ENVs.
So you have to copy ARG to ENV.

ingest_data.py

#!/usr/bin/env python
import argparse

if __name__ == '__main__':    
    
    parser = argparse.ArgumentParser(description='Ingest CSV data to Postgres')

    parser.add_argument('--host', required=True, help='host for postgres')
    parser.add_argument('--port', required=True, help='port for postgres')
    parser.add_argument('--db', required=True, help='database name for postgres')
    parser.add_argument('--table_name', required=True, help='name of the table where we will write the results to')

    args = parser.parse_args()

In a Dockerfile. Note: To pass arguments to ENTRYPOINT you should run it in a shell form, not in a exec form!:

FROM python:3.9.1

RUN pip install pandas sqlalchemy psycopg2 pyarrow

WORKDIR /app
COPY ingest_data.py ingest_data.py

ARG host
ARG port
ARG db
ARG table_name

ENV host=${host} port=${port} db=${db} table_name=${table_name}
RUN echo "Host: ${host}, Port: ${port}, DB: $db"

ENTRYPOINT python ingest_data.py -host=$host --port=$port --db=$db --table_name=$table_name

In a docker-compose.yaml

version: "3"
services:
  app:
    build: 
      context: .
      args: 
        - host=pgdatabase
        - port=5432
        - db=ny_taxi
        - table_name=yellow_taxi_trips_n

Finally to run it,

$ docker-compose up -d

Answered By: Artem Zaika

python argparse in a docker container based on ubuntu

Question:

Answers: