XGBoost Library (libxgboost.so) could not be loaded

Question:

Problem

In a serverless project, I want to import XGBoost into a lambda written on Python. But when I’m trying to invoke the lambda, I see this error on CloudWatch:

[ERROR] XGBoostError: XGBoost Library (libxgboost.so) could not be loaded.
Likely causes:
* OpenMP runtime is not installed (vcomp140.dll or libgomp-1.dll for Windows, libgomp.so for UNIX-like OSes)
* You are running 32-bit Python on a 64-bit OS
Error message(s): [‘libgomp.so.1: cannot open shared object file: No such file or directory’]

What I tried?

  1. I’m running sls deploy on macOS so I’ve added dockerizePip: true to my serverless.yml

  2. I tried to fix missing dependency by using custom Dockerfile:

    FROM lambci/lambda:build-python3.6

RUN apt-get update && apt-get install libaio1

    I also must specify library path in dockerExtraFiles but I have no idea where libgomp.so should be located on Linux. So, I stuck on this point.

My code

serverless.yml:

app: improve
org: kvadrug
service: testservice

provider:
  name: aws
  runtime: python3.8
  versionFunctions: false
  stage: dev
  region: us-west-2
  timeout: 30

plugins:
  - serverless-python-requirements

custom:
  pythonRequirements:
    dockerFile: Dockerfile
    zip: true
    dockerizePip: true

functions:
  hello:
    handler: hello.hello
    events:
      - http:
          path: hello
          method: post
          private: true

package.json:

{
    "name": "testservice",
    "version": "1.0.0",
    "description": "Test service",
    "dependencies": {},
    "devDependencies": {
        "serverless-python-requirements": "^5.1.0"
    }
}

requirements.txt:

xgboost==1.0.2
Asked By: kelin

||
Source

Answers:

On Linux, the libraries you need are installed into

/usr/lib/x86_64-linux-gnu/libgomp.so.1
/usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0

by

apt-get install libgomp1

On newer Ubuntu and CentOS, libgomp1 is no longer installed by default (I have just checked – I have it on xenial, don’t have it on a fresh bionic).

This is on a AWS machine, unfortunately not very recent (I have no other readily available):

lrwxrwxrwx 1 root root     16 Oct  4  2019 /usr/lib/x86_64-linux-gnu/libgomp.so.1 -> libgomp.so.1.0.0
-rw-r--r-- 1 root root 138448 Oct  4  2019 /usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0
Answered By: LSerni

Working configuration – Python 3.7

Tested on Mojave 10.14.6 (18G4032) and Docker v2.1.0.2 (37877).

Steps:

sls requirements clean
rm -Rf ~/Library/Caches/serverless-python-requirements/
sls deploy

requirements.txt:

xgboost==1.0.2

serverless.yml:

service: xgboost
provider:
  name: aws
  timeout:60
  runtime: python3.7

plugins:
  - serverless-python-requirements

custom:
  pythonRequirements:
    zip: true
    dockerizePip: non-linux
    dockerExtraFiles:
      - /usr/lib64/libgomp.so.1

functions:
  hello:
    handler: handler.hello
    events:
      - http:
          path: hello
          method: post
          private: true

handler.py :

    try:
        import unzip_requirements
    except ImportError:
        pass

    import sys
    import glob
    import os

    def hello(event, context):
        pkgdir = '/tmp/sls-py-req/'

        print("-------- Sys Path --------")
        for p in sys.path:
            print(p)

        if os.path.exists(pkgdir):
            print("-------- Pkg Dir ----------")
            os.chdir(pkgdir)
            for file in glob.glob("*"):
                print(file)

        print("-------- Var Task ----------")
        os.chdir("/var/task")
        for file in glob.glob("*"):
            print(file)

        try:
            import xgboost as xgb
            print(xgb)
        except Exception as ex:
            template = "An exception of type {0} occurred. Arguments:n{1!r}"
            message = template.format(type(ex).__name__, ex.args)
            print(message)

        return True

Python 3.8

For some reason, Lambda running Python 3.8 ignores the entry for ‘/tmp/sls-py-req’ on ‘sys.path’. Therefore you need to manually add the library file ‘libgomp.so.1’ to the root of your application.

To give it a try, update ‘runtime: python3.7’ to ‘runtime: python3.8’ on serverless.yml and follow these steps:

sls requirements clean
rm -Rf ~/Library/Caches/serverless-python-requirements/
sls package
cp .serverless/requirements/libgomp.so.1 ./
sls deploy

FYI – How to find the libraries location within the Lambda docker image

Run:

docker run --rm -ti --entrypoint /bin/sh -u 0 lambci/lambda:build-python3.8

Then:

find / -name libgomp.so.1
Answered By: Alexandre Andrade

For some reason, these solutions did not work for me (Python 3.6, xgboost==1.4, Ubuntu 18.04, aarch64).

However, I was able to get things working by pointing to the libgomp library packaged with the xgboost Python package.

For example, export LD_PRELOAD=/usr/local/lib/python3.6/dist-packages/xgboost/lib/../../xgboost.libs/libgomp-d22c30c5.so.1.0.0

Answered By: Prunus Persica