Importing the numpy C-extensions failed (Amplify)

Question:

Cross posted on GitHub

I’m working with AWS Amplify and pipenv for my python 3.9 lambda. I’m attempting to use pandas to create a dataframe, do some processing and write it back to CSV for sagemaker inference.

Reproducing code example:

import pandas as pd

(Code immediately fails after this)

Error message:

Here’s the full error message:

[ERROR] Runtime.ImportModuleError: Unable to import module 'index': 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.9 from "/var/lang/bin/python3.9"
  * The NumPy version is: "1.21.2"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: No module named 'numpy.core._multiarray_umath'

I first downloaded with pipenv install pandas which automatically installs numpy

To solve I’ve tried:

  • pipenv install numpy / pipenv uninstall numpy
  • pipenv uninstall pandas / pipenv install pandas
  • pipenv uninstall setuptools / pipenv install setuptools

Important to note I’m on Windows 10

Asked By: cmcnphp

||
Source

Answers:

I had ran into a similar issue. After much research it looks like the Lambda Layer AWSLambda-Python38-SciPy1x provided by Amazon is your best bet. More Info is here.

You can manually add the Layer via the Console like so: Picture for you

Or you can add the layer via the Amplify CLI. I ran the following commands on an existing lambda function:

amplify function update

  • Then select the Lambda function (serverless function) you’d like to
    add the layer to.

  • Go to Lambda layers configuration.

  • Provide the ARN under Provide existing Lambda Layer ARN.

  • I used arn:aws:lambda:us-west-2:420165488524:layer:AWSLambda-Python38-SciPy1x:29

    amplify push

That should do the trick.

Answered By: Michael Angelo

switching to lower python version works for me. from python3.8 to python3.7

Answered By: Qing Yuan

I faced the same problem because I was building the packages in a different
python version from the AWS Lambda runtime I was trying to import the pandas layer.

So check your python version and create a lambda function in the same python version, then add the layer you’re gonna make.

It worked in my case.

Answered By: Lucas Advir

It is helpful for debugging to go to the numpy.core and see what’s inside.

In my case reason of such error was wrong platform of .so files used in created by me Lambda Layer.

enter image description here
(correct files on the left, wrong on the right)

I made a layer on my Mac OS where pip by default downloads numpy for darwin platform. But actually I had to download packages for Lambda’s amazonlinux2 environment with

pip install 
 --platform=manylinux1_x86_64 
 --only-binary=:all: 
 --target layer-dir/python 
 numpy;
Answered By: juggernaut

I wasted hours on this one. You need to make sure if you build with pip for version 3.7 of python to select the same supported platform in the lambda layer. This one worked for me.

Answered By: Nik V