"The headers or library files could not be found for jpeg" installing Pillow on Alpine Linux
Question:
I’m trying to run Python’s Scrapy in a Docker container based on python:alpine. It was working before, but now I’d like to use Scrapy’s Image Pipeline which requires me to install Pillow.
As a simplified example, I tried the following Dockerfile:
FROM python:alpine
RUN apk --update add libxml2-dev libxslt-dev libffi-dev gcc musl-dev libgcc openssl-dev curl
RUN apk add libjpeg zlib tiff freetype lcms libwebp tcl openjpeg
RUN pip install Pillow
However, when I try to build this I get an error which contains the following:
Traceback (most recent call last):
File "/tmp/pip-build-ft5yzzuv/Pillow/setup.py", line 744, in <module>
zip_safe=not debug_build(), )
File "/usr/local/lib/python3.6/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/local/lib/python3.6/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/usr/local/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.6/site-packages/setuptools/command/install.py", line 61, in run
return orig.install.run(self)
File "/usr/local/lib/python3.6/distutils/command/install.py", line 545, in run
self.run_command('build')
File "/usr/local/lib/python3.6/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/local/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.6/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/usr/local/lib/python3.6/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/local/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.6/distutils/command/build_ext.py", line 339, in run
self.build_extensions()
File "/tmp/pip-build-ft5yzzuv/Pillow/setup.py", line 545, in build_extensions
raise RequiredDependencyException(f)
__main__.RequiredDependencyException: jpeg
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-ft5yzzuv/Pillow/setup.py", line 756, in <module>
raise RequiredDependencyException(msg)
__main__.RequiredDependencyException:
The headers or library files could not be found for jpeg,
a required dependency when compiling Pillow from source.
Please see the install instructions at:
https://pillow.readthedocs.io/en/latest/installation.html
I went through the requirements on https://pillow.readthedocs.io/en/latest/installation.html and tried to find the corresponding packages for Alpine, although one I couldn’t find was libimagequant, so this might be the ‘culprit’. Nonetheless, it the traceback and error message seem to be saying that jpeg is missing, whereas I have installed openjpeg.
How can I modify the Dockerfile so that pip install Pillow runs?
Answers:
have you seen this QA? Fail during installation of Pillow (Python module) in Linux
they say you have to update your pip and install libjpeg-dev.
Have you try to create a requirements.txt and insert this in Dockerfile:
RUN apt-get update -qq && apt-get install build-essential g++ flex bison gperf ruby perl
mysql-client
libsqlite3-dev libmysqlclient-dev libfontconfig1-dev libicu-dev libfreetype6 libssl-dev
libpng-dev libjpeg-dev python libx11-dev libxext-dev -y
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
ADD requirements.txt /code/
COPY . /code
RUN pip install -r requirements.txt
ADD . /code/
example of requirements.txt
Django==1.9.7
django-appconf==1.0.1
django-filer==1.2.5
django-filter==0.15.0
django-grappelli==2.8.1
django-image-cropping==1.0.3
django-mptt==0.8.6
django-nested-admin==3.0.10
django-nested-inline==0.3.6
django-polymorphic==0.8.1
django-taggit==0.21.2
django-tinymce==2.4.0
dnspython==1.15.0
easy-thumbnails==2.3
enum34==1.1.2
funcsigs==0.4
idna==2.1
ipaddress==1.0.17
mercurial==3.7.3
mock==1.3.0
mysql-python
ndg-httpsclient==0.4.2
parsedatetime==2.1
pbr==1.8.0
Pillow==3.3.1
psutil==3.4.2
pyasn1==0.1.9
PyICU==1.9.2
pyOpenSSL==16.1.0
pyRFC3339==1.0
python-augeas==0.5.0
python-monkey-business==1.0.0
python2-pythondialog==3.3.0
pytz==2014.10
requests==2.11.1
six==1.10.0
Unidecode==0.4.19
urllib3==1.16
zope.component==4.2.2
zope.event==4.2.0
zope.hookable==4.0.4
zope.interface==4.1.3
In a comment that appears to have been deleted later, someone pointed me to https://github.com/python-pillow/Pillow/blob/c05099f45c0d94a2a98c3609a96bdb6cf7446627/depends/alpine_Dockerfile. Based on that Dockerfile I modified my own as follows:
FROM python:alpine
RUN apk --update add libxml2-dev libxslt-dev libffi-dev gcc musl-dev libgcc openssl-dev curl
RUN apk add jpeg-dev zlib-dev freetype-dev lcms2-dev openjpeg-dev tiff-dev tk-dev tcl-dev
RUN pip install Pillow
Now it builds successfully.
For debian
sudo apt install libjpeg-dev zlib1g-dev
pip install Pillow
Just adding “RUN apk add jpeg-dev” to Dockerfile fixed the problem for me.
I ran into this problem with docker image python:3.6-alpine I solved it by adding these packages apk add jpeg-dev zlib-dev.
If you’re missing libjpeg.so package in runtime, try this (alpine linux):
apk add --no-cache jpeg
Just in case anyone else is still struggling like I was you can see the official alpine Dockerfile for Pillow here: https://github.com/python-pillow/docker-images/blob/master/alpine/Dockerfile#L20
It states the following dependencies:
RUN apk --no-cache add python3
...
# Pillow dependencies
jpeg-dev
zlib-dev
freetype-dev
lcms2-dev
openjpeg-dev
tiff-dev
tk-dev
tcl-dev
harfbuzz-dev
fribidi-dev
In short, this helps:
RUN apt-get update -qq && apt-get install -y build-essential libsqlite3-dev
libpng-dev libjpeg-dev
Detailed:
I have the same error with python:3.8-slim-buster image. Solution presented by @pierangelo-orizio worked for me, but I just cleaned it to a minimal required packages list. So here is my Dockerfile:
FROM python:3.8-slim-buster
RUN apt-get update -qq && apt-get install -y build-essential libsqlite3-dev
libpng-dev libjpeg-dev
COPY requirements.txt .
RUN pip install --upgrade pip
RUN pip install -r requirements.txt
EXPOSE 8000
VOLUME /usr/src/app
WORKDIR /usr/src/app
CMD python manage.py runserver 0.0.0.0:8000
And requirements.txt:
Django>=2.1,<2.2
wagtail>=2.4,<2.5
django-cors-headers==2.5.3
python-dotenv==0.10.3
For macOS:
brew install libtiff libjpeg webp little-cms2
Reason:
https://pillow.readthedocs.io/en/latest/installation.html
Run into the same issue on python 3.9. Fixed it by bumping pillow version in requirements.txt to Pillow>=8.0. There’s a nice matrix of which pillow version supports which python version here.
For yum users, this should work:
sudo yum -y install libjpeg-turbo-devel
pip3 install Pillow
For macOS users, just run:
brew install libjpeg libtiff little-cms2 openjpeg webp
I’m trying to run Python’s Scrapy in a Docker container based on python:alpine. It was working before, but now I’d like to use Scrapy’s Image Pipeline which requires me to install Pillow.
As a simplified example, I tried the following Dockerfile:
FROM python:alpine
RUN apk --update add libxml2-dev libxslt-dev libffi-dev gcc musl-dev libgcc openssl-dev curl
RUN apk add libjpeg zlib tiff freetype lcms libwebp tcl openjpeg
RUN pip install Pillow
However, when I try to build this I get an error which contains the following:
Traceback (most recent call last):
File "/tmp/pip-build-ft5yzzuv/Pillow/setup.py", line 744, in <module>
zip_safe=not debug_build(), )
File "/usr/local/lib/python3.6/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/local/lib/python3.6/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/usr/local/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.6/site-packages/setuptools/command/install.py", line 61, in run
return orig.install.run(self)
File "/usr/local/lib/python3.6/distutils/command/install.py", line 545, in run
self.run_command('build')
File "/usr/local/lib/python3.6/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/local/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.6/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/usr/local/lib/python3.6/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/local/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.6/distutils/command/build_ext.py", line 339, in run
self.build_extensions()
File "/tmp/pip-build-ft5yzzuv/Pillow/setup.py", line 545, in build_extensions
raise RequiredDependencyException(f)
__main__.RequiredDependencyException: jpeg
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-ft5yzzuv/Pillow/setup.py", line 756, in <module>
raise RequiredDependencyException(msg)
__main__.RequiredDependencyException:
The headers or library files could not be found for jpeg,
a required dependency when compiling Pillow from source.
Please see the install instructions at:
https://pillow.readthedocs.io/en/latest/installation.html
I went through the requirements on https://pillow.readthedocs.io/en/latest/installation.html and tried to find the corresponding packages for Alpine, although one I couldn’t find was libimagequant, so this might be the ‘culprit’. Nonetheless, it the traceback and error message seem to be saying that jpeg is missing, whereas I have installed openjpeg.
How can I modify the Dockerfile so that pip install Pillow runs?
have you seen this QA? Fail during installation of Pillow (Python module) in Linux
they say you have to update your pip and install libjpeg-dev.
Have you try to create a requirements.txt and insert this in Dockerfile:
RUN apt-get update -qq && apt-get install build-essential g++ flex bison gperf ruby perl
mysql-client
libsqlite3-dev libmysqlclient-dev libfontconfig1-dev libicu-dev libfreetype6 libssl-dev
libpng-dev libjpeg-dev python libx11-dev libxext-dev -y
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
ADD requirements.txt /code/
COPY . /code
RUN pip install -r requirements.txt
ADD . /code/
example of requirements.txt
Django==1.9.7
django-appconf==1.0.1
django-filer==1.2.5
django-filter==0.15.0
django-grappelli==2.8.1
django-image-cropping==1.0.3
django-mptt==0.8.6
django-nested-admin==3.0.10
django-nested-inline==0.3.6
django-polymorphic==0.8.1
django-taggit==0.21.2
django-tinymce==2.4.0
dnspython==1.15.0
easy-thumbnails==2.3
enum34==1.1.2
funcsigs==0.4
idna==2.1
ipaddress==1.0.17
mercurial==3.7.3
mock==1.3.0
mysql-python
ndg-httpsclient==0.4.2
parsedatetime==2.1
pbr==1.8.0
Pillow==3.3.1
psutil==3.4.2
pyasn1==0.1.9
PyICU==1.9.2
pyOpenSSL==16.1.0
pyRFC3339==1.0
python-augeas==0.5.0
python-monkey-business==1.0.0
python2-pythondialog==3.3.0
pytz==2014.10
requests==2.11.1
six==1.10.0
Unidecode==0.4.19
urllib3==1.16
zope.component==4.2.2
zope.event==4.2.0
zope.hookable==4.0.4
zope.interface==4.1.3
In a comment that appears to have been deleted later, someone pointed me to https://github.com/python-pillow/Pillow/blob/c05099f45c0d94a2a98c3609a96bdb6cf7446627/depends/alpine_Dockerfile. Based on that Dockerfile I modified my own as follows:
FROM python:alpine
RUN apk --update add libxml2-dev libxslt-dev libffi-dev gcc musl-dev libgcc openssl-dev curl
RUN apk add jpeg-dev zlib-dev freetype-dev lcms2-dev openjpeg-dev tiff-dev tk-dev tcl-dev
RUN pip install Pillow
Now it builds successfully.
For debian
sudo apt install libjpeg-dev zlib1g-dev
pip install Pillow
Just adding “RUN apk add jpeg-dev” to Dockerfile fixed the problem for me.
I ran into this problem with docker image python:3.6-alpine I solved it by adding these packages apk add jpeg-dev zlib-dev.
If you’re missing libjpeg.so package in runtime, try this (alpine linux):
apk add --no-cache jpeg
Just in case anyone else is still struggling like I was you can see the official alpine Dockerfile for Pillow here: https://github.com/python-pillow/docker-images/blob/master/alpine/Dockerfile#L20
It states the following dependencies:
RUN apk --no-cache add python3
...
# Pillow dependencies
jpeg-dev
zlib-dev
freetype-dev
lcms2-dev
openjpeg-dev
tiff-dev
tk-dev
tcl-dev
harfbuzz-dev
fribidi-dev
In short, this helps:
RUN apt-get update -qq && apt-get install -y build-essential libsqlite3-dev
libpng-dev libjpeg-dev
Detailed:
I have the same error with python:3.8-slim-buster image. Solution presented by @pierangelo-orizio worked for me, but I just cleaned it to a minimal required packages list. So here is my Dockerfile:
FROM python:3.8-slim-buster
RUN apt-get update -qq && apt-get install -y build-essential libsqlite3-dev
libpng-dev libjpeg-dev
COPY requirements.txt .
RUN pip install --upgrade pip
RUN pip install -r requirements.txt
EXPOSE 8000
VOLUME /usr/src/app
WORKDIR /usr/src/app
CMD python manage.py runserver 0.0.0.0:8000
And requirements.txt:
Django>=2.1,<2.2
wagtail>=2.4,<2.5
django-cors-headers==2.5.3
python-dotenv==0.10.3
For macOS:
brew install libtiff libjpeg webp little-cms2
Reason:
https://pillow.readthedocs.io/en/latest/installation.html
Run into the same issue on python 3.9. Fixed it by bumping pillow version in requirements.txt to Pillow>=8.0. There’s a nice matrix of which pillow version supports which python version here.
For yum users, this should work:
sudo yum -y install libjpeg-turbo-devel
pip3 install Pillow
For macOS users, just run:
brew install libjpeg libtiff little-cms2 openjpeg webp