Use Python package installed with apk in Alpine Linux
Question:
I’d like to install some Python packages in Alpine Linux using apk. I use numpy as an example in the following.
Dockerfile
FROM python:3-alpine
RUN apk add --update py3-numpy
I build my Docker image
$ docker build -t python-numpy .
Sending build context to Docker daemon 2.048kB
Step 1/2 : FROM python:3-alpine
---> 930a7e894675
Step 2/2 : RUN apk add --update py3-numpy
---> Running in b30470090cde
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/community/x86_64/APKINDEX.tar.gz
(1/6) Installing libgcc (8.3.0-r0)
(2/6) Installing libquadmath (8.3.0-r0)
(3/6) Installing libgfortran (8.3.0-r0)
(4/6) Installing openblas (0.3.6-r0)
(5/6) Installing python3 (3.7.3-r0)
(6/6) Installing py3-numpy (1.16.4-r1)
Executing busybox-1.30.1-r2.trigger
OK: 113 MiB in 41 packages
Removing intermediate container b30470090cde
---> 5a82ffa67522
Successfully built 5a82ffa67522
Successfully tagged python-numpy:latest
run it and try to import the package in python
$ docker run -it --rm python-numpy python -c "import numpy"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'numpy'
However, it is not found. Running pip install numpy after apk add --update py3-numpy does not consider the apk package py3-numpy and downloads another version:
Collecting numpy
Downloading https://files.pythonhosted.org/packages/da/32/1b8f2bb5fb50e4db68543eb85ce37b9fa6660cd05b58bddfafafa7ed62da/numpy-1.17.0.zip (6.5MB)
...
If I specify the same version as of py3-numpy (see output of docker build) in pip install numpy==1.16.4-r1, it leads to
Collecting numpy==1.16.4-r1
ERROR: Could not find a version that satisfies the requirement numpy==1.16.4-r1 (from versions: 1.3.0, 1.4.1, 1.5.0, 1.5.1, 1.6.0, 1.6.1, 1.6.2, 1.7.0, 1.7.1, 1.7.2, 1.8.0, 1.8.1, 1.8.2, 1.9.0, 1.9.1, 1.9.2, 1.9.3, 1.10.0.post2, 1.10.1, 1.10.2, 1.10.4, 1.11.0b3, 1.11.0rc1, 1.11.0rc2, 1.11.0, 1.11.1rc1, 1.11.1, 1.11.2rc1, 1.11.2, 1.11.3, 1.12.0b1, 1.12.0rc1, 1.12.0rc2, 1.12.0, 1.12.1rc1, 1.12.1, 1.13.0rc1, 1.13.0rc2, 1.13.0, 1.13.1, 1.13.3, 1.14.0rc1, 1.14.0, 1.14.1, 1.14.2, 1.14.3, 1.14.4, 1.14.5, 1.14.6, 1.15.0rc1, 1.15.0rc2, 1.15.0, 1.15.1, 1.15.2, 1.15.3, 1.15.4, 1.16.0rc1, 1.16.0rc2, 1.16.0, 1.16.1, 1.16.2, 1.16.3, 1.16.4, 1.17.0rc1, 1.17.0rc2, 1.17.0)
ERROR: No matching distribution found for numpy==1.16.4-r1
What am I missing?
Answers:
Install it as python package will work fine as the base image is python so I will recommend you to install it as a python package. As installing with the alpine package manager it does not exist in the python packages nor in alpine packages. So below will work fine.
FROM python:3-alpine
RUN apk add g++ && \
pip install numpy
Now run the container
docker run -it --rm abc python -c "import numpy"
EDIT: This approach is not recommended, since the apk packages are for a different python version (see Itamar Turner-Trauring’s answer).
apk installs python packages in /usr/lib/python3.7/site-packages. This path is not part of Python’s sys.path (in Docker images). Hence, packages installed with apk are not found. The installation directory has to be added to the search path:
FROM python:3.7-alpine
RUN apk add --update py3-numpy
ENV PYTHONPATH /usr/lib/python3.7/site-packages
Note that there is no /usr/lib/python3/site-packages and even if you use python:3.6-alpine, apk add --update py3-numpy creates a directory /usr/lib/python3.7 and not /usr/lib/python3.6.
By the way, you have a similar issue if you are using python:3, but the installation directory is different
FROM python:3
RUN apt-get update && apt-get install -y python3-numpy
ENV PYTHONPATH /usr/lib/python3/dist-packages
The issue is that python:3-alpine has two Pythons: the one provided by Alpine, and an additional one added by the Python Docker image. Installing packages in one will not be reflected in the other.
Some options:
- Switch to just alpine base image,
FROM alpine:3.10. Then you’ll just have the Python installed via apk.
- Stop using Alpine, and switch to
FROM python:3.7-slim-buster (my personal recommendation: https://pythonspeed.com/articles/base-image-python-docker-images/). This will allow you to pip install numpy without having to compile anything—binary wheels don’t work on Alpine, but will work on the (Debian) Buster image.
This worked
FROM python:3-alpine
RUN apk add g++ &&
pip install numpy
But, our process needs the install on pipenv and below fails with error pipenv not found
FROM python:3-alpine
RUN apk add g++ &&
pipenv install numpy
Note pipenv is installed in earlier docker statements. However, even the below fails, with pipenv not found
FROM python:3-alpine
RUN apk add g++ &&
pip install –user pipenv &
pipenv install numpy
Any ideas?
I’d like to install some Python packages in Alpine Linux using apk. I use numpy as an example in the following.
Dockerfile
FROM python:3-alpine
RUN apk add --update py3-numpy
I build my Docker image
$ docker build -t python-numpy .
Sending build context to Docker daemon 2.048kB
Step 1/2 : FROM python:3-alpine
---> 930a7e894675
Step 2/2 : RUN apk add --update py3-numpy
---> Running in b30470090cde
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/community/x86_64/APKINDEX.tar.gz
(1/6) Installing libgcc (8.3.0-r0)
(2/6) Installing libquadmath (8.3.0-r0)
(3/6) Installing libgfortran (8.3.0-r0)
(4/6) Installing openblas (0.3.6-r0)
(5/6) Installing python3 (3.7.3-r0)
(6/6) Installing py3-numpy (1.16.4-r1)
Executing busybox-1.30.1-r2.trigger
OK: 113 MiB in 41 packages
Removing intermediate container b30470090cde
---> 5a82ffa67522
Successfully built 5a82ffa67522
Successfully tagged python-numpy:latest
run it and try to import the package in python
$ docker run -it --rm python-numpy python -c "import numpy"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'numpy'
However, it is not found. Running pip install numpy after apk add --update py3-numpy does not consider the apk package py3-numpy and downloads another version:
Collecting numpy
Downloading https://files.pythonhosted.org/packages/da/32/1b8f2bb5fb50e4db68543eb85ce37b9fa6660cd05b58bddfafafa7ed62da/numpy-1.17.0.zip (6.5MB)
...
If I specify the same version as of py3-numpy (see output of docker build) in pip install numpy==1.16.4-r1, it leads to
Collecting numpy==1.16.4-r1
ERROR: Could not find a version that satisfies the requirement numpy==1.16.4-r1 (from versions: 1.3.0, 1.4.1, 1.5.0, 1.5.1, 1.6.0, 1.6.1, 1.6.2, 1.7.0, 1.7.1, 1.7.2, 1.8.0, 1.8.1, 1.8.2, 1.9.0, 1.9.1, 1.9.2, 1.9.3, 1.10.0.post2, 1.10.1, 1.10.2, 1.10.4, 1.11.0b3, 1.11.0rc1, 1.11.0rc2, 1.11.0, 1.11.1rc1, 1.11.1, 1.11.2rc1, 1.11.2, 1.11.3, 1.12.0b1, 1.12.0rc1, 1.12.0rc2, 1.12.0, 1.12.1rc1, 1.12.1, 1.13.0rc1, 1.13.0rc2, 1.13.0, 1.13.1, 1.13.3, 1.14.0rc1, 1.14.0, 1.14.1, 1.14.2, 1.14.3, 1.14.4, 1.14.5, 1.14.6, 1.15.0rc1, 1.15.0rc2, 1.15.0, 1.15.1, 1.15.2, 1.15.3, 1.15.4, 1.16.0rc1, 1.16.0rc2, 1.16.0, 1.16.1, 1.16.2, 1.16.3, 1.16.4, 1.17.0rc1, 1.17.0rc2, 1.17.0)
ERROR: No matching distribution found for numpy==1.16.4-r1
What am I missing?
Install it as python package will work fine as the base image is python so I will recommend you to install it as a python package. As installing with the alpine package manager it does not exist in the python packages nor in alpine packages. So below will work fine.
FROM python:3-alpine
RUN apk add g++ && \
pip install numpy
Now run the container
docker run -it --rm abc python -c "import numpy"
EDIT: This approach is not recommended, since the apk packages are for a different python version (see Itamar Turner-Trauring’s answer).
apk installs python packages in /usr/lib/python3.7/site-packages. This path is not part of Python’s sys.path (in Docker images). Hence, packages installed with apk are not found. The installation directory has to be added to the search path:
FROM python:3.7-alpine
RUN apk add --update py3-numpy
ENV PYTHONPATH /usr/lib/python3.7/site-packages
Note that there is no /usr/lib/python3/site-packages and even if you use python:3.6-alpine, apk add --update py3-numpy creates a directory /usr/lib/python3.7 and not /usr/lib/python3.6.
By the way, you have a similar issue if you are using python:3, but the installation directory is different
FROM python:3
RUN apt-get update && apt-get install -y python3-numpy
ENV PYTHONPATH /usr/lib/python3/dist-packages
The issue is that python:3-alpine has two Pythons: the one provided by Alpine, and an additional one added by the Python Docker image. Installing packages in one will not be reflected in the other.
Some options:
- Switch to just alpine base image,
FROM alpine:3.10. Then you’ll just have the Python installed via apk. - Stop using Alpine, and switch to
FROM python:3.7-slim-buster(my personal recommendation: https://pythonspeed.com/articles/base-image-python-docker-images/). This will allow you to pip install numpy without having to compile anything—binary wheels don’t work on Alpine, but will work on the (Debian) Buster image.
This worked
FROM python:3-alpine
RUN apk add g++ &&
pip install numpy
But, our process needs the install on pipenv and below fails with error pipenv not found
FROM python:3-alpine
RUN apk add g++ &&
pipenv install numpy
Note pipenv is installed in earlier docker statements. However, even the below fails, with pipenv not found
FROM python:3-alpine
RUN apk add g++ &&
pip install –user pipenv &
pipenv install numpy
Any ideas?