Modern/2020 way to call C++ code from Python

Question

I am trying to call a C++ function from a Python script. I have seen different solutions on Stackoverflow from 2010-2015 but they are all using complicated packages and was hoping for something easier/newer and more sophisticated. The C++ function I am trying to call takes in a double variable and returns a double.

double foo(double var1){
    double result = ...
    return result;
}

Asked By: user14347561

||

Source

Answer 1

Python has ctypes package which allows calling functions in DLLs or shared libraries. Compile your C++ project into a shared library (.so) on Linux, or DLL on Windows. Export the functions you wish to expose outside.

C++ supports function overloading, to avoid ambiguity in the binary code, additional information is added to function names, known as name mangling. To ensure no name is changed, place inside an extern "C" block.
More on importance of extern "C" at the end!

Demo:
In this dummy demo, our library has a single function, taking an int and printing it.

lib.cpp

#include <iostream>

int Function(int num) 
{
    std::cout << "Num = " << num << std::endl;
    return 0;
}

extern "C" {
    int My_Function(int a)
    {
        return Function(a);
    }
}

We will compile this into a shared object first

g++ -fPIC -shared -o libTest.so lib.cpp

Now we will utilized ctypes, to load the shared object/dll and functions.

myLib.py

import ctypes
import sys
import os 

dir_path = os.path.dirname(os.path.realpath(__file__))
handle = ctypes.CDLL(dir_path + "/libTest.so")     

handle.My_Function.argtypes = [ctypes.c_int] 
  
def My_Function(num):
    return handle.My_Function(num)

For our test, we will call the function with num = 16

test.py

from myLib import *

My_Function(16)

The expected out as well.

EDIT: Comment section does not understand the importance of extern "C". As already explained above, C++ supports function overloading and additional information is added to function names known as name mangling.

Consider the following library

#include <iostream>

int My_Function(int num) 
{
    std::cout << "Num = " << num << std::endl;
    return 0;
}

Compiled the same: g++ -fPIC -shared -o libTest.so lib.cpp. Listing the exported symbols with nm -gD libTest.so results in:

Notice how the function name in the exported symbols is changed to _Z11My_Functioni. Running test.py now fails as it can not find the symbol.

You’d have to change myLib.py to reflect the change. However, you do not compile your library, take a look at the resulting symbol and build your module extension because there’s no guarantee that re-compiling in the future with different version and additional code will result in the same symbol names. This is why one uses extern "C". Notice how in the first code the function name is unchanged.

Answered By: Tony Tannous

Answer 2

Replace double foo(double var1) with

#define PY_SSIZE_T_CLEAN
#include <Python.h>

extern "C" PyObject* foo(PyObject* self,PyObject* args, PyObject* kwargs)
{
    double var1;
    char* arr[] = {(char*)"var1",NULL};
    int err = PyArgs_ParseTupleAndKeywords(args,kwargs,"d",arr,&var1);
    if (err==0) {return NULL;}
    var result = ...;
    return Py_BuildValue("d",result);
}

Then add the function

PyMethodDef pymethods[2]:
PyModuleDef pymodule;
extern "C" PyObject* PyInit_bar() // Replace bar with your C++ Source file's name without its extension
{
    pymethods[0].ml_meth = (PyCFunction)foo;
    pymethods[0].ml_name = "foo";
    pymethods[0].ml_flags = METH_VAARGS | METH_KEYWORDS;
    pymethods[0].ml_doc = "";
    pymethods[1] = NULL;
    pymodule.m_name = "bar"; // Replace bar here too
    pymodule.m_methods = pymethods;
    pymodule.m_doc = "";
    PyObject* mod = PyModule_Create(&pymodule);
    return rtn;
}

Compile it as dll and change its file extension to .pyd and then you can import that pyd from your python code

Example

import bar
print(bar.foo(5.5))

Answered By: Supergamer

Modern/2020 way to call C++ code from Python

Question:

Answers:

Example