Using numpy in AWS Lambda


I’m looking for a work around to use numpy in AWS lambda. I am not using EC2 just lambda for this so if anyone has a suggestion that’d be appreciated. Currently getting the error:

cannot import name ‘multiarray’

Using grunt lambda to create the zip file and upload the function code. All the modules that I use are installed into a folder called python_modules inside the root of the lambda function which includes numpy using pip install and a requirements.txt file.

Asked By: Sebastian Łącki



Updated to include the solution here, rather than a link:

After much effort, I found that I had to create my deployment package from within a python3.6 virtualenv, rather than directly from the host machine. I did the following within a Ubuntu 16.04 docker image. This assumes that you have python3.6, virtualenv and awscli already installed/configured, and that your lambda function code is in the ~/lambda_code directory:

1) cd ~ (We’ll build the virtualenv in the home directory)

2) virtualenv venv --python=python3.6 (Create the virtual environment)

3) source venv/bin/activate (Activate the virtual environment)

4) pip install numpy

5) cp -r ~/venv/lib/python3.6/site-packages/* ~/lambda_code (Copy all installed packages into root level of lambda_code directory. This will include a few unnecessary files, but you can remove those yourself if needed)

6) cd ~/lambda_code

7) zip -r9 ~/ . (Zip up the lambda package)

8) aws lambda update-function-code --function-name my_lambda_function --zip-file fileb://~/ (Upload to AWS)

Your lambda function should now be able to import numpy with no problems.

If you want a more out-of-the-box solution, you could consider using serverless to deploy your lambda function. Before I found the above solution, I followed the guide here and was able to run numpy successfully in a python3.6 lambda function.

Answered By: cailan s

I would add this answer as well:

Using pipenv includes all of the needed .so files as well.

Answered By: alexpotato

I was unable to find a good solution using serverless plugins, but I did find a good way with layers. See Serverless – Numpy – Unable to find good bind path format

Answered By: launchpadmcquack

As of 2018 it’s best to just use the inbuilt layers functionality.

AWS have actually released a pre-made one with numpy in it:

Answered By: DaveRGP

1.) Do a Pip install of numpy to a folder on your local machine.

2.) once complete, zip the entire folder and create a zip file.

3.) Go to AWS lambda console, create a layer and upload zip file created in step 2 there and save the layer.

4.) After you create your lambda function, click add layer and add the layer you created. That’s it, import numpy will start working.

Answered By: appleios

An easy way to make your lambda function support the numpy library for python 3.7:

  1. Go to your lambda function page
  2. Find the Layers section at the bottom of the page.
  3. Click on Add a layer.
  4. Choose AWS layers as layer source.
  5. Select AWSLambda-Python37-Scipy1x as AWS layers.
  6. Select 37 for version.
  7. And finally click on Add.

Now your lambda function is ready to support numpy.

Answered By: user_5

Add numpy layer in this way:

-Go on your lambda function

  • select add a new layer

  • add it using this arn: arn:aws:lambda:eu-central-1:770693421928:layer:Klayers-p39-numpy:7

(change your zone if you are not in eu-central-1)

Let me know if it will work

Answered By: Saro