Installing numpy on Docker Alpine
Question:
I’m trying to install numpy in a docker container based on Alpine 3.1. I’m using the following Dockerfile:
FROM alpine:3.1
RUN apk add --update make cmake gcc g++ gfortran
RUN apk add --update python py-pip python-dev
RUN pip install cython
RUN pip install numpy
This runs fine until pip install numpy when I get the following error:
error: Command "gcc -fno-strict-aliasing -Os -fomit-frame-pointer -DNDEBUG -Os -fomit-frame-pointer -fPIC -Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/include/python2.7 -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -c build/src.linux-x86_64-2.7/numpy/core/src/npymath/ieee754.c -o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/npymath/ieee754.o" failed with exit status 1
easy_install-2.7 numpy gives the same error.
Are there any config/installation steps I’m missing?
Answers:
If you don’t necessary need to install numpy from pypi, you could install it from alpine repositories. Package is named py-numpy and is in testing repository, see here. Minimal Dockerfile example that works for me
FROM alpine:3.2
ADD repositories /etc/apk/repositories
RUN apk add --update python python-dev gfortran py-pip build-base py-numpy@community
Content of repositories file
http://dl-cdn.alpinelinux.org/alpine/v3.2/main
@community http://dl-cdn.alpinelinux.org/alpine/edge/community
I’ve been having a bit of trouble with this myself and, long story short, I would encourage you to ask if it’s really worth the hassle. Numpy is enormous when you start adding things to the stack like pandas, gpus, and scipy so the benefit of building it on alpine is limited, the savings over using Debian, Arch, or even Ubuntu are relatively modest when 500MB of your space is on this library anyway.
That having been said, I threw together an image that does it. I needed as build-time dependencies musl-dev, linux-headers, and g++. I also wound up needing to add openblas from edge for something later in the stack so it’s possible that some dependencies from that are required too. But I believe just adding the three former libraries with
apk --no-cache add musl-dev linux-headers g++
should be sufficient to prevent the gcc error you are getting. You can view the image at https://hub.docker.com/r/o76923/alpine-numpy-stack/
Try this:
RUN apk --no-cache --update-cache add gcc gfortran python python-dev py-pip build-base wget freetype-dev libpng-dev openblas-dev
RUN ln -s /usr/include/locale.h /usr/include/xlocale.h
RUN pip install pandas
This one is about 311MB according to my docker images:
FROM python:3.6-alpine
RUN apk add g++
RUN pip install numpy
(Meanwhile python:3.6 is ~900MB by itself)
Have you tried NOT having gcc installed? It might be conflicting? Not sure. This one worked for me as a minimal numpy installation and wanted to share.
A package is now available in the Alpine repository: py3-numpy. But you won’t be able to use it straightaway.
py3-numpy installs libraries into /usr/lib/python3.8/site-packages directory but the default Python module path does not use it:
$ docker run -it python:3.8-alpine sh
/ # apk add --update --no-cache py3-numpy
/ # python
>>> import numpy
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'numpy'
>>> import sys
>>> sys.path
['', '/usr/local/lib/python38.zip', '/usr/local/lib/python3.8', '/usr/local/lib/python3.8/lib-dynload', '/usr/local/lib/python3.8/site-packages']
This can be fixed by setting the $PYTHONPATH environment variable to the path of the site-packages in /usr/lib:
FROM python:3.8-alpine
RUN apk add --update --no-cache py3-numpy
ENV PYTHONPATH=/usr/lib/python3.8/site-packages
With optimizations such as removing build dependencies after build and removing unneeded tests (they are here because we’re building the module, not just installing it):
FROM frolvlad/alpine-python3
RUN apk add --no-cache
--virtual=.build-dependencies
g++ file binutils
musl-dev python3-dev cython &&
apk add libstdc++ openblas &&
ln -s locale.h /usr/include/xlocale.h &&
pip install numpy &&
rm -r /root/.cache &&
find /usr/lib/python3.*/ -name 'tests' -exec rm -r '{}' + &&
find /usr/lib/python3.*/site-packages/ -name '*.so' -print -exec sh -c 'file "{}" | grep -q "not stripped" && strip -s "{}"' ; &&
rm /usr/include/xlocale.h &&
apk del .build-dependencies
Resulting size ~157MB.
simply use a docker image with numpy pre-installed: https://hub.docker.com/r/adreeve/python-numpy/
Alpine is built with musl, which is incompatible with python wheels. That means that either all the dependencies should be installed via apk or they should be compiled manually. For a smoother experience with python pypi dependencies it looks more optimal to use debian, cropped to the minimum size (python:slim) as a starting point:
FROM python:slim
CMD pip install numpy
123Mb
This approach is way simpler than the accepted answer and the resulting image is more compact than in the other answers.
I’m trying to install numpy in a docker container based on Alpine 3.1. I’m using the following Dockerfile:
FROM alpine:3.1
RUN apk add --update make cmake gcc g++ gfortran
RUN apk add --update python py-pip python-dev
RUN pip install cython
RUN pip install numpy
This runs fine until pip install numpy when I get the following error:
error: Command "gcc -fno-strict-aliasing -Os -fomit-frame-pointer -DNDEBUG -Os -fomit-frame-pointer -fPIC -Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/include/python2.7 -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -c build/src.linux-x86_64-2.7/numpy/core/src/npymath/ieee754.c -o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/npymath/ieee754.o" failed with exit status 1
easy_install-2.7 numpy gives the same error.
Are there any config/installation steps I’m missing?
If you don’t necessary need to install numpy from pypi, you could install it from alpine repositories. Package is named py-numpy and is in testing repository, see here. Minimal Dockerfile example that works for me
FROM alpine:3.2
ADD repositories /etc/apk/repositories
RUN apk add --update python python-dev gfortran py-pip build-base py-numpy@community
Content of repositories file
http://dl-cdn.alpinelinux.org/alpine/v3.2/main
@community http://dl-cdn.alpinelinux.org/alpine/edge/community
I’ve been having a bit of trouble with this myself and, long story short, I would encourage you to ask if it’s really worth the hassle. Numpy is enormous when you start adding things to the stack like pandas, gpus, and scipy so the benefit of building it on alpine is limited, the savings over using Debian, Arch, or even Ubuntu are relatively modest when 500MB of your space is on this library anyway.
That having been said, I threw together an image that does it. I needed as build-time dependencies musl-dev, linux-headers, and g++. I also wound up needing to add openblas from edge for something later in the stack so it’s possible that some dependencies from that are required too. But I believe just adding the three former libraries with
apk --no-cache add musl-dev linux-headers g++
should be sufficient to prevent the gcc error you are getting. You can view the image at https://hub.docker.com/r/o76923/alpine-numpy-stack/
Try this:
RUN apk --no-cache --update-cache add gcc gfortran python python-dev py-pip build-base wget freetype-dev libpng-dev openblas-dev
RUN ln -s /usr/include/locale.h /usr/include/xlocale.h
RUN pip install pandas
This one is about 311MB according to my docker images:
FROM python:3.6-alpine
RUN apk add g++
RUN pip install numpy
(Meanwhile python:3.6 is ~900MB by itself)
Have you tried NOT having gcc installed? It might be conflicting? Not sure. This one worked for me as a minimal numpy installation and wanted to share.
A package is now available in the Alpine repository: py3-numpy. But you won’t be able to use it straightaway.
py3-numpy installs libraries into /usr/lib/python3.8/site-packages directory but the default Python module path does not use it:
$ docker run -it python:3.8-alpine sh
/ # apk add --update --no-cache py3-numpy
/ # python
>>> import numpy
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'numpy'
>>> import sys
>>> sys.path
['', '/usr/local/lib/python38.zip', '/usr/local/lib/python3.8', '/usr/local/lib/python3.8/lib-dynload', '/usr/local/lib/python3.8/site-packages']
This can be fixed by setting the $PYTHONPATH environment variable to the path of the site-packages in /usr/lib:
FROM python:3.8-alpine
RUN apk add --update --no-cache py3-numpy
ENV PYTHONPATH=/usr/lib/python3.8/site-packages
With optimizations such as removing build dependencies after build and removing unneeded tests (they are here because we’re building the module, not just installing it):
FROM frolvlad/alpine-python3
RUN apk add --no-cache
--virtual=.build-dependencies
g++ file binutils
musl-dev python3-dev cython &&
apk add libstdc++ openblas &&
ln -s locale.h /usr/include/xlocale.h &&
pip install numpy &&
rm -r /root/.cache &&
find /usr/lib/python3.*/ -name 'tests' -exec rm -r '{}' + &&
find /usr/lib/python3.*/site-packages/ -name '*.so' -print -exec sh -c 'file "{}" | grep -q "not stripped" && strip -s "{}"' ; &&
rm /usr/include/xlocale.h &&
apk del .build-dependencies
Resulting size ~157MB.
simply use a docker image with numpy pre-installed: https://hub.docker.com/r/adreeve/python-numpy/
Alpine is built with musl, which is incompatible with python wheels. That means that either all the dependencies should be installed via apk or they should be compiled manually. For a smoother experience with python pypi dependencies it looks more optimal to use debian, cropped to the minimum size (python:slim) as a starting point:
FROM python:slim
CMD pip install numpy
123Mb
This approach is way simpler than the accepted answer and the resulting image is more compact than in the other answers.