Firebase-Admin with AWS Lambda Python
Question:
I am currently writing a Python server to deploy on AWS Lambda. I want to use the firebase-admin package to send notifications with FCM and read data from cloud firestore. however when I try deploying my function to AWS Lambda with the .zip file archives, I get this error on execution:
[ERROR] Runtime.ImportModuleError: Unable to import module 'lambda_function': Failed to import the Cloud Firestore library for Python. Make sure to install the "google-cloud-firestore" module.
I installed the module with this: pip install --target . firebase-admin into a folder, added my code files (to the root as instructed), zipped it recursively and uploaded it with the aws-cli, I can clearly see that there is a google-cloud-firestore folder inside the .zip so i’m not sure whats going on. any help is appreciated!
Answers:
What you’re looking for is a LambdaLayer.
There are a bunch of resources on that online, but essentially you want to make a Zip file in this format*:
myLayer.zip/
— python/
—- lib/
—— pythonX.Y/
——– site-packages/
———- firebase-admin
———- …
*where X.Y is the python version you’re targetting
Then, either in the AWS console or in your CI/CD, you can add that LambdaLayer in your Function’s configuration, which lets you import all the libraries in it.
From the look of it you have bundled your code correctly and deployed successfully. The error occurs because Firestore relies on a C-based implementation of GRPC. By default this does not work on AWS Lambda. I’m currently creating a work-around and will update this post with my results.
The problem is that the firebase-admin package uses some libraries that are compiled differently on different OSes, so you need to create a docker image and install everything there, zip it up and upload it as a layer. Here is an example docker file:
FROM amazonlinux:2.0.20210126.0
RUN yum -y groupinstall "Development Tools" &&
yum -y install openssl-devel bzip2-devel libffi-devel &&
yum -y install wget &&
wget https://www.python.org/ftp/python/3.8.7/Python-3.8.7.tgz &&
yum install -y tar &&
yum install -y gzip &&
tar xvf Python-3.8.7.tgz &&
cd Python-3.8*/ &&
./configure --enable-optimizations &&
make altinstall &&
yum install -y zip &&
yum clean all
RUN python3.8 -m pip install --upgrade pip &&
python3.8 -m pip install virtualenv
RUN python3.8 -m venv myvenv
RUN source myvenv/bin/activate
RUN pip install firebase-admin -t ./python
RUN deactivate
RUN zip -r python.zip ./python/
And then copy the zip file:
docker cp container_name:python.zip ./Desktop/
Would be useful for next people if someone could explain how to run a docker file.
I’m not an expert, so what I’ve done was to create a docker instance based on the image mentioned, then I executed each command individually within the CLI, and lastly, since I wasn’t able to zip and copy the files from my container to the desktop, I had to find the folder through my windows explorer within the Linux virtual enviroment so that I could ZIP and bring.
It actually worked, but again, will be very helpful if someone can complement the best answer by letting everyone know how to build and execute a docker file.
Best!
I am currently writing a Python server to deploy on AWS Lambda. I want to use the firebase-admin package to send notifications with FCM and read data from cloud firestore. however when I try deploying my function to AWS Lambda with the .zip file archives, I get this error on execution:
[ERROR] Runtime.ImportModuleError: Unable to import module 'lambda_function': Failed to import the Cloud Firestore library for Python. Make sure to install the "google-cloud-firestore" module.
I installed the module with this: pip install --target . firebase-admin into a folder, added my code files (to the root as instructed), zipped it recursively and uploaded it with the aws-cli, I can clearly see that there is a google-cloud-firestore folder inside the .zip so i’m not sure whats going on. any help is appreciated!
What you’re looking for is a LambdaLayer.
There are a bunch of resources on that online, but essentially you want to make a Zip file in this format*:
myLayer.zip/
— python/
—- lib/
—— pythonX.Y/
——– site-packages/
———- firebase-admin
———- …
*where X.Y is the python version you’re targetting
Then, either in the AWS console or in your CI/CD, you can add that LambdaLayer in your Function’s configuration, which lets you import all the libraries in it.
From the look of it you have bundled your code correctly and deployed successfully. The error occurs because Firestore relies on a C-based implementation of GRPC. By default this does not work on AWS Lambda. I’m currently creating a work-around and will update this post with my results.
The problem is that the firebase-admin package uses some libraries that are compiled differently on different OSes, so you need to create a docker image and install everything there, zip it up and upload it as a layer. Here is an example docker file:
FROM amazonlinux:2.0.20210126.0
RUN yum -y groupinstall "Development Tools" &&
yum -y install openssl-devel bzip2-devel libffi-devel &&
yum -y install wget &&
wget https://www.python.org/ftp/python/3.8.7/Python-3.8.7.tgz &&
yum install -y tar &&
yum install -y gzip &&
tar xvf Python-3.8.7.tgz &&
cd Python-3.8*/ &&
./configure --enable-optimizations &&
make altinstall &&
yum install -y zip &&
yum clean all
RUN python3.8 -m pip install --upgrade pip &&
python3.8 -m pip install virtualenv
RUN python3.8 -m venv myvenv
RUN source myvenv/bin/activate
RUN pip install firebase-admin -t ./python
RUN deactivate
RUN zip -r python.zip ./python/
And then copy the zip file:
docker cp container_name:python.zip ./Desktop/
Would be useful for next people if someone could explain how to run a docker file.
I’m not an expert, so what I’ve done was to create a docker instance based on the image mentioned, then I executed each command individually within the CLI, and lastly, since I wasn’t able to zip and copy the files from my container to the desktop, I had to find the folder through my windows explorer within the Linux virtual enviroment so that I could ZIP and bring.
It actually worked, but again, will be very helpful if someone can complement the best answer by letting everyone know how to build and execute a docker file.
Best!