Cython For Appengine
Question:
Is it possible to use Cython in appengine (with Python 2.7 specifically)? I could see using it when some code is not performing well, and could use a speed boost by having a statically defined type, or loop optimization, etc…
Answers:
No, this is very clearly not possible. GAE has very strong restrictions on what you can run inside its sandbox, and one of the most definitively spelled-out restrictions is that you cannot run arbitrary C code:
All code for the Python runtime environment must be pure Python, and not include any C extensions or other code that must be compiled.
This is now possible with Managed VMs.
The following project will serve as an example:
https://github.com/GoogleCloudPlatform/appengine-vm-fortunespeak-python
The answers above are dated, but the question is still a good one. You can run Cython with gcloud now provided that you cythonize at build time.
For example:
# Load python version
FROM python:3.11
# Allow statements and log messages to immediately appear in the Knative logs
ENV PYTHONUNBUFFERED True
# Copy local code to the container image.
ENV APP_HOME /app
WORKDIR $APP_HOME
COPY . ./
# Install production dependencies and cythonize .pyx files
RUN pip install -r requirements.txt
RUN python setup.py build_ext --inplace
# Run the web service on container startup. Here we use the gunicorn
CMD exec gunicorn --bind :$PORT --workers 1 --threads 8 --timeout 0 run:app
Now, there is an interesting twist if you use a SLIM build of python. In that case gcc is not included and so cythonizing will fail. So then you have to load gcc by hand:
# The slim python doesnt include gcc
FROM python:3.11-slim
# Allow statements and log messages to immediately appear in the Knative logs
ENV PYTHONUNBUFFERED True
# Copy local code to the container image.
ENV APP_HOME /app
WORKDIR $APP_HOME
COPY . ./
# Install production dependencies
RUN pip install -r requirements.txt
# Install gcc and cythonize
RUN apt-get update
RUN apt-get -y install gcc
RUN python setup_pyd.py build_ext --inplace
# Run the web service on container startup. Here we use the gunicorn
CMD exec gunicorn --bind :$PORT --workers 1 --threads 8 --timeout 0 run:app
Is it possible to use Cython in appengine (with Python 2.7 specifically)? I could see using it when some code is not performing well, and could use a speed boost by having a statically defined type, or loop optimization, etc…
No, this is very clearly not possible. GAE has very strong restrictions on what you can run inside its sandbox, and one of the most definitively spelled-out restrictions is that you cannot run arbitrary C code:
All code for the Python runtime environment must be pure Python, and not include any C extensions or other code that must be compiled.
This is now possible with Managed VMs.
The following project will serve as an example:
https://github.com/GoogleCloudPlatform/appengine-vm-fortunespeak-python
The answers above are dated, but the question is still a good one. You can run Cython with gcloud now provided that you cythonize at build time.
For example:
# Load python version
FROM python:3.11
# Allow statements and log messages to immediately appear in the Knative logs
ENV PYTHONUNBUFFERED True
# Copy local code to the container image.
ENV APP_HOME /app
WORKDIR $APP_HOME
COPY . ./
# Install production dependencies and cythonize .pyx files
RUN pip install -r requirements.txt
RUN python setup.py build_ext --inplace
# Run the web service on container startup. Here we use the gunicorn
CMD exec gunicorn --bind :$PORT --workers 1 --threads 8 --timeout 0 run:app
Now, there is an interesting twist if you use a SLIM build of python. In that case gcc is not included and so cythonizing will fail. So then you have to load gcc by hand:
# The slim python doesnt include gcc
FROM python:3.11-slim
# Allow statements and log messages to immediately appear in the Knative logs
ENV PYTHONUNBUFFERED True
# Copy local code to the container image.
ENV APP_HOME /app
WORKDIR $APP_HOME
COPY . ./
# Install production dependencies
RUN pip install -r requirements.txt
# Install gcc and cythonize
RUN apt-get update
RUN apt-get -y install gcc
RUN python setup_pyd.py build_ext --inplace
# Run the web service on container startup. Here we use the gunicorn
CMD exec gunicorn --bind :$PORT --workers 1 --threads 8 --timeout 0 run:app