Calling a python method from C/C++, and extracting its return value
Question:
I’d like to call a custom function that is defined in a Python module from C. I have some preliminary code to do that, but it just prints the output to stdout.
mytest.py
import math
def myabs(x):
return math.fabs(x)
test.cpp
#include <Python.h>
int main() {
Py_Initialize();
PyRun_SimpleString("import sys; sys.path.append('.')");
PyRun_SimpleString("import mytest;");
PyRun_SimpleString("print mytest.myabs(2.0)");
Py_Finalize();
return 0;
}
How can I extract the return value into a C double and use it in C?
Answers:
You have to extract the python method somehow and run it with PyObject_CallObject(). To do that, you can provide Python a way to set the function, as the Extending and Embedding Python Tutorial example does.
A complete example of calling a Python function and retrieving the result is located at http://docs.python.org/release/2.6.5/extending/embedding.html#pure-embedding:
#include <Python.h>
int
main(int argc, char *argv[])
{
PyObject *pName, *pModule, *pDict, *pFunc;
PyObject *pArgs, *pValue;
int i;
if (argc < 3) {
fprintf(stderr,"Usage: call pythonfile funcname [args]n");
return 1;
}
Py_Initialize();
pName = PyString_FromString(argv[1]);
/* Error checking of pName left out */
pModule = PyImport_Import(pName);
Py_DECREF(pName);
if (pModule != NULL) {
pFunc = PyObject_GetAttrString(pModule, argv[2]);
/* pFunc is a new reference */
if (pFunc && PyCallable_Check(pFunc)) {
pArgs = PyTuple_New(argc - 3);
for (i = 0; i < argc - 3; ++i) {
pValue = PyInt_FromLong(atoi(argv[i + 3]));
if (!pValue) {
Py_DECREF(pArgs);
Py_DECREF(pModule);
fprintf(stderr, "Cannot convert argumentn");
return 1;
}
/* pValue reference stolen here: */
PyTuple_SetItem(pArgs, i, pValue);
}
pValue = PyObject_CallObject(pFunc, pArgs);
Py_DECREF(pArgs);
if (pValue != NULL) {
printf("Result of call: %ldn", PyInt_AsLong(pValue));
Py_DECREF(pValue);
}
else {
Py_DECREF(pFunc);
Py_DECREF(pModule);
PyErr_Print();
fprintf(stderr,"Call failedn");
return 1;
}
}
else {
if (PyErr_Occurred())
PyErr_Print();
fprintf(stderr, "Cannot find function "%s"n", argv[2]);
}
Py_XDECREF(pFunc);
Py_DECREF(pModule);
}
else {
PyErr_Print();
fprintf(stderr, "Failed to load "%s"n", argv[1]);
return 1;
}
Py_Finalize();
return 0;
}
If you assign the return value to a variable, then you can use something like PyEval_GetGlobals() and PyDict_GetItemString() to get the PyObject. From there, PyNumber_Float can get you the value you want.
I suggest browsing through the whole API – certain things become obvious when you see the different methods that are available to you, and there may well be a better method than the one I’ve described.
As explained before, using PyRun_SimpleString seems to be a bad idea.
You should definitely use the methods provided by the C-API (http://docs.python.org/c-api/).
Reading the introduction is the first thing to do to understand the way it works.
First, you have to learn about PyObject that is the basic object for the C API. It can represent any kind of python basic types (string, float, int,…).
Many functions exist to convert for example python string to char* or PyFloat to double.
First, import your module :
PyObject* myModuleString = PyString_FromString((char*)"mytest");
PyObject* myModule = PyImport_Import(myModuleString);
Then getting a reference to your function :
PyObject* myFunction = PyObject_GetAttrString(myModule,(char*)"myabs");
PyObject* args = PyTuple_Pack(1,PyFloat_FromDouble(2.0));
Then getting your result :
PyObject* myResult = PyObject_CallObject(myFunction, args)
And getting back to a double :
double result = PyFloat_AsDouble(myResult);
You should obviously check the errors (cf. link given by Mark Tolonen).
If you have any question, don’t hesitate. Good luck.
I have done it using BOOST to embedded Python to C++ [This working C module should help]
#include <boost/python.hpp>
void main()
{
using namespace boost::python;
Py_Initialize();
PyObject* filename = PyString_FromString((char*)"memory_leak_test");
PyObject* imp = PyImport_Import(filename);
PyObject* func = PyObject_GetAttrString(imp,(char*)"begin");
PyObject* args = PyTuple_Pack(1,PyString_FromString("CacheSetup"));
PyObject* retured_value = PyObject_CallObject(func, args); // if you have arg
double retured_value = PyFloat_AsDouble(myResult);
std::cout << result << std::endl;
Py_Finalize();
}
Here is a sample code I wrote (with the help of various online sources) to send a string to a Python code, then return a value.
Here is the C code call_function.c:
#include <Python.h>
#include <stdlib.h>
int main()
{
// Set PYTHONPATH TO working directory
setenv("PYTHONPATH",".",1);
PyObject *pName, *pModule, *pDict, *pFunc, *pValue, *presult;
// Initialize the Python Interpreter
Py_Initialize();
// Build the name object
pName = PyString_FromString((char*)"arbName");
// Load the module object
pModule = PyImport_Import(pName);
// pDict is a borrowed reference
pDict = PyModule_GetDict(pModule);
// pFunc is also a borrowed reference
pFunc = PyDict_GetItemString(pDict, (char*)"someFunction");
if (PyCallable_Check(pFunc))
{
pValue=Py_BuildValue("(z)",(char*)"something");
PyErr_Print();
printf("Let's give this a shot!n");
presult=PyObject_CallObject(pFunc,pValue);
PyErr_Print();
} else
{
PyErr_Print();
}
printf("Result is %dn",PyInt_AsLong(presult));
Py_DECREF(pValue);
// Clean up
Py_DECREF(pModule);
Py_DECREF(pName);
// Finish the Python Interpreter
Py_Finalize();
return 0;
}
Here is the Python code, in file arbName.py:
def someFunction(text):
print 'You passed this Python program '+text+' from C! Congratulations!'
return 12345
I use the command gcc call_function.c -I/usr/include/python2.6 -lpython2.6 ; ./a.out to run this process. I’m on redhat. I recommend using PyErr_Print(); for error checking.
To prevent the extra .py file as in the other answers, you can just retrieve the __main__ module, which is created by the first call to PyRun_SimpleString:
PyObject *moduleMainString = PyString_FromString("__main__");
PyObject *moduleMain = PyImport_Import(moduleMainString);
PyRun_SimpleString(
"def mul(a, b): n"
" return a * b n"
);
PyObject *func = PyObject_GetAttrString(moduleMain, "mul");
PyObject *args = PyTuple_Pack(2, PyFloat_FromDouble(3.0), PyFloat_FromDouble(4.0));
PyObject *result = PyObject_CallObject(func, args);
printf("mul(3,4): %.2fn", PyFloat_AsDouble(result)); // 12
Here is a simple and direct answer to your question:
#include <iostream>
#include <Python.h>
using namespace std;
int main()
{
const char *scriptDirectoryName = "/yourDir";
Py_Initialize();
PyObject *sysPath = PySys_GetObject("path");
PyObject *path = PyString_FromString(scriptDirectoryName);
int result = PyList_Insert(sysPath, 0, path);
PyObject *pModule = PyImport_ImportModule("mytest");
PyObject* myFunction = PyObject_GetAttrString(pModule,(char*)"myabs");
PyObject* args = PyTuple_Pack(1,PyFloat_FromDouble(-2.0));
PyObject* myResult = PyObject_CallObject(myFunction, args);
double getResult = PyFloat_AsDouble(myResult);
return 0;
}
Here’s a minimal executable version that also works with Python 3 (tested with Python 2.7 and 3.9).
Links into the docs are included in the comments, but all are accessible under https://docs.python.org/3/c-api/
#include <Python.h>
#include <stdio.h>
int main()
{
// Initialize the Python Interpreter
Py_Initialize();
// see https://docs.python.org/3/c-api/structures.html
// NULL objects are special and Py_CLEAR knows this
PyObject *module = NULL, *result = NULL;
// https://docs.python.org/3/c-api/import.html
module = PyImport_ImportModule("mytest");
if (!module) {
// Python generally uses exceptions to indicate an error state which
// gets flagged in the C-API (a NULL pointer in this case) indicating
// "something" failed. the PyErr_* API should be used to get more
// details
goto done;
}
// see https://docs.python.org/3/c-api/call.html#c.PyObject_CallMethod
// and https://docs.python.org/3/c-api/arg.html#building-values
result = PyObject_CallMethod(module, "myabs", "f", 3.14);
if (!result) {
goto done;
}
// make sure we got our number back
if (PyFloat_Check(result)) {
printf("Successfully got a float: %fn", PyFloat_AsDouble(result));
} else {
printf("Successfully got something unexpected!n");
}
done:
// see https://docs.python.org/3/c-api/exceptions.html
PyErr_Print();
// see https://docs.python.org/3/c-api/refcounting.html
Py_CLEAR(result);
Py_CLEAR(module);
// Optionally release Python Interpreter
Py_Finalize();
return 0;
}
This uses the OP’s Python code mytest.py, or this one-line equivalent:
from math import fabs as myabs
Building is going to be OS/Python version specific, but the following works for me:
cc -o test -I/usr/include/python3.9 /usr/lib/libpython3.9.so test.c
As others have already mentioned, this is answered at the Python docs. However, since I come from Python and don’t have that much experience using C/C++, I had some issues running it with Python 3. So here is my full working example to get the Python docs running after spending some time on other posts of stackoverflow:
File c_function.c
#define PY_SSIZE_T_CLEAN
#include <Python.h>
int main(int argc, char *argv[])
{
PyObject *pName, *pModule, *pFunc;
PyObject *pArgs, *pValue;
int i;
if (argc < 3) {
fprintf(stderr,"Usage: call pythonfile funcname [args]n");
return 1;
}
Py_Initialize();
// I had to add the following two lines to make it work
PyRun_SimpleString("import sys");
PyRun_SimpleString("sys.path.append(".")");
pName = PyUnicode_DecodeFSDefault(argv[1]);
/* Error checking of pName left out */
pModule = PyImport_Import(pName);
Py_DECREF(pName);
if (pModule != NULL) {
pFunc = PyObject_GetAttrString(pModule, argv[2]);
/* pFunc is a new reference */
if (pFunc && PyCallable_Check(pFunc)) {
pArgs = PyTuple_New(argc - 3);
for (i = 0; i < argc - 3; ++i) {
pValue = PyLong_FromLong(atoi(argv[i + 3]));
if (!pValue) {
Py_DECREF(pArgs);
Py_DECREF(pModule);
fprintf(stderr, "Cannot convert argumentn");
return 1;
}
/* pValue reference stolen here: */
PyTuple_SetItem(pArgs, i, pValue);
}
pValue = PyObject_CallObject(pFunc, pArgs);
Py_DECREF(pArgs);
if (pValue != NULL) {
printf("Result of call: %ldn", PyLong_AsLong(pValue));
Py_DECREF(pValue);
}
else {
Py_DECREF(pFunc);
Py_DECREF(pModule);
PyErr_Print();
fprintf(stderr,"Call failedn");
return 1;
}
}
else {
if (PyErr_Occurred())
PyErr_Print();
fprintf(stderr, "Cannot find function "%s"n", argv[2]);
}
Py_XDECREF(pFunc);
Py_DECREF(pModule);
}
else {
PyErr_Print();
fprintf(stderr, "Failed to load "%s"n", argv[1]);
return 1;
}
if (Py_FinalizeEx() < 0) {
return 120;
}
return 0;
}
File multiply.py
def multiply(a,b):
print("Will compute", a, "times", b)
c = 0
for i in range(0, a):
c = c + b
return c
We need to compile and link it. This can be done with the command:
gcc c_function.c -c $(python3.6-config --cflags) -fPIC
followed by
gcc c_function.o $(python3.6-config --ldflags) -o call
for the example of Python 3.6. Afterwards, the example from the Python docs can just be executed by
./call multiply multiply 3 2
I’d like to call a custom function that is defined in a Python module from C. I have some preliminary code to do that, but it just prints the output to stdout.
mytest.py
import math
def myabs(x):
return math.fabs(x)
test.cpp
#include <Python.h>
int main() {
Py_Initialize();
PyRun_SimpleString("import sys; sys.path.append('.')");
PyRun_SimpleString("import mytest;");
PyRun_SimpleString("print mytest.myabs(2.0)");
Py_Finalize();
return 0;
}
How can I extract the return value into a C double and use it in C?
You have to extract the python method somehow and run it with PyObject_CallObject(). To do that, you can provide Python a way to set the function, as the Extending and Embedding Python Tutorial example does.
A complete example of calling a Python function and retrieving the result is located at http://docs.python.org/release/2.6.5/extending/embedding.html#pure-embedding:
#include <Python.h>
int
main(int argc, char *argv[])
{
PyObject *pName, *pModule, *pDict, *pFunc;
PyObject *pArgs, *pValue;
int i;
if (argc < 3) {
fprintf(stderr,"Usage: call pythonfile funcname [args]n");
return 1;
}
Py_Initialize();
pName = PyString_FromString(argv[1]);
/* Error checking of pName left out */
pModule = PyImport_Import(pName);
Py_DECREF(pName);
if (pModule != NULL) {
pFunc = PyObject_GetAttrString(pModule, argv[2]);
/* pFunc is a new reference */
if (pFunc && PyCallable_Check(pFunc)) {
pArgs = PyTuple_New(argc - 3);
for (i = 0; i < argc - 3; ++i) {
pValue = PyInt_FromLong(atoi(argv[i + 3]));
if (!pValue) {
Py_DECREF(pArgs);
Py_DECREF(pModule);
fprintf(stderr, "Cannot convert argumentn");
return 1;
}
/* pValue reference stolen here: */
PyTuple_SetItem(pArgs, i, pValue);
}
pValue = PyObject_CallObject(pFunc, pArgs);
Py_DECREF(pArgs);
if (pValue != NULL) {
printf("Result of call: %ldn", PyInt_AsLong(pValue));
Py_DECREF(pValue);
}
else {
Py_DECREF(pFunc);
Py_DECREF(pModule);
PyErr_Print();
fprintf(stderr,"Call failedn");
return 1;
}
}
else {
if (PyErr_Occurred())
PyErr_Print();
fprintf(stderr, "Cannot find function "%s"n", argv[2]);
}
Py_XDECREF(pFunc);
Py_DECREF(pModule);
}
else {
PyErr_Print();
fprintf(stderr, "Failed to load "%s"n", argv[1]);
return 1;
}
Py_Finalize();
return 0;
}
If you assign the return value to a variable, then you can use something like PyEval_GetGlobals() and PyDict_GetItemString() to get the PyObject. From there, PyNumber_Float can get you the value you want.
I suggest browsing through the whole API – certain things become obvious when you see the different methods that are available to you, and there may well be a better method than the one I’ve described.
As explained before, using PyRun_SimpleString seems to be a bad idea.
You should definitely use the methods provided by the C-API (http://docs.python.org/c-api/).
Reading the introduction is the first thing to do to understand the way it works.
First, you have to learn about PyObject that is the basic object for the C API. It can represent any kind of python basic types (string, float, int,…).
Many functions exist to convert for example python string to char* or PyFloat to double.
First, import your module :
PyObject* myModuleString = PyString_FromString((char*)"mytest");
PyObject* myModule = PyImport_Import(myModuleString);
Then getting a reference to your function :
PyObject* myFunction = PyObject_GetAttrString(myModule,(char*)"myabs");
PyObject* args = PyTuple_Pack(1,PyFloat_FromDouble(2.0));
Then getting your result :
PyObject* myResult = PyObject_CallObject(myFunction, args)
And getting back to a double :
double result = PyFloat_AsDouble(myResult);
You should obviously check the errors (cf. link given by Mark Tolonen).
If you have any question, don’t hesitate. Good luck.
I have done it using BOOST to embedded Python to C++ [This working C module should help]
#include <boost/python.hpp>
void main()
{
using namespace boost::python;
Py_Initialize();
PyObject* filename = PyString_FromString((char*)"memory_leak_test");
PyObject* imp = PyImport_Import(filename);
PyObject* func = PyObject_GetAttrString(imp,(char*)"begin");
PyObject* args = PyTuple_Pack(1,PyString_FromString("CacheSetup"));
PyObject* retured_value = PyObject_CallObject(func, args); // if you have arg
double retured_value = PyFloat_AsDouble(myResult);
std::cout << result << std::endl;
Py_Finalize();
}
Here is a sample code I wrote (with the help of various online sources) to send a string to a Python code, then return a value.
Here is the C code call_function.c:
#include <Python.h>
#include <stdlib.h>
int main()
{
// Set PYTHONPATH TO working directory
setenv("PYTHONPATH",".",1);
PyObject *pName, *pModule, *pDict, *pFunc, *pValue, *presult;
// Initialize the Python Interpreter
Py_Initialize();
// Build the name object
pName = PyString_FromString((char*)"arbName");
// Load the module object
pModule = PyImport_Import(pName);
// pDict is a borrowed reference
pDict = PyModule_GetDict(pModule);
// pFunc is also a borrowed reference
pFunc = PyDict_GetItemString(pDict, (char*)"someFunction");
if (PyCallable_Check(pFunc))
{
pValue=Py_BuildValue("(z)",(char*)"something");
PyErr_Print();
printf("Let's give this a shot!n");
presult=PyObject_CallObject(pFunc,pValue);
PyErr_Print();
} else
{
PyErr_Print();
}
printf("Result is %dn",PyInt_AsLong(presult));
Py_DECREF(pValue);
// Clean up
Py_DECREF(pModule);
Py_DECREF(pName);
// Finish the Python Interpreter
Py_Finalize();
return 0;
}
Here is the Python code, in file arbName.py:
def someFunction(text):
print 'You passed this Python program '+text+' from C! Congratulations!'
return 12345
I use the command gcc call_function.c -I/usr/include/python2.6 -lpython2.6 ; ./a.out to run this process. I’m on redhat. I recommend using PyErr_Print(); for error checking.
To prevent the extra .py file as in the other answers, you can just retrieve the __main__ module, which is created by the first call to PyRun_SimpleString:
PyObject *moduleMainString = PyString_FromString("__main__");
PyObject *moduleMain = PyImport_Import(moduleMainString);
PyRun_SimpleString(
"def mul(a, b): n"
" return a * b n"
);
PyObject *func = PyObject_GetAttrString(moduleMain, "mul");
PyObject *args = PyTuple_Pack(2, PyFloat_FromDouble(3.0), PyFloat_FromDouble(4.0));
PyObject *result = PyObject_CallObject(func, args);
printf("mul(3,4): %.2fn", PyFloat_AsDouble(result)); // 12
Here is a simple and direct answer to your question:
#include <iostream>
#include <Python.h>
using namespace std;
int main()
{
const char *scriptDirectoryName = "/yourDir";
Py_Initialize();
PyObject *sysPath = PySys_GetObject("path");
PyObject *path = PyString_FromString(scriptDirectoryName);
int result = PyList_Insert(sysPath, 0, path);
PyObject *pModule = PyImport_ImportModule("mytest");
PyObject* myFunction = PyObject_GetAttrString(pModule,(char*)"myabs");
PyObject* args = PyTuple_Pack(1,PyFloat_FromDouble(-2.0));
PyObject* myResult = PyObject_CallObject(myFunction, args);
double getResult = PyFloat_AsDouble(myResult);
return 0;
}
Here’s a minimal executable version that also works with Python 3 (tested with Python 2.7 and 3.9).
Links into the docs are included in the comments, but all are accessible under https://docs.python.org/3/c-api/
#include <Python.h>
#include <stdio.h>
int main()
{
// Initialize the Python Interpreter
Py_Initialize();
// see https://docs.python.org/3/c-api/structures.html
// NULL objects are special and Py_CLEAR knows this
PyObject *module = NULL, *result = NULL;
// https://docs.python.org/3/c-api/import.html
module = PyImport_ImportModule("mytest");
if (!module) {
// Python generally uses exceptions to indicate an error state which
// gets flagged in the C-API (a NULL pointer in this case) indicating
// "something" failed. the PyErr_* API should be used to get more
// details
goto done;
}
// see https://docs.python.org/3/c-api/call.html#c.PyObject_CallMethod
// and https://docs.python.org/3/c-api/arg.html#building-values
result = PyObject_CallMethod(module, "myabs", "f", 3.14);
if (!result) {
goto done;
}
// make sure we got our number back
if (PyFloat_Check(result)) {
printf("Successfully got a float: %fn", PyFloat_AsDouble(result));
} else {
printf("Successfully got something unexpected!n");
}
done:
// see https://docs.python.org/3/c-api/exceptions.html
PyErr_Print();
// see https://docs.python.org/3/c-api/refcounting.html
Py_CLEAR(result);
Py_CLEAR(module);
// Optionally release Python Interpreter
Py_Finalize();
return 0;
}
This uses the OP’s Python code mytest.py, or this one-line equivalent:
from math import fabs as myabs
Building is going to be OS/Python version specific, but the following works for me:
cc -o test -I/usr/include/python3.9 /usr/lib/libpython3.9.so test.c
As others have already mentioned, this is answered at the Python docs. However, since I come from Python and don’t have that much experience using C/C++, I had some issues running it with Python 3. So here is my full working example to get the Python docs running after spending some time on other posts of stackoverflow:
File c_function.c
#define PY_SSIZE_T_CLEAN
#include <Python.h>
int main(int argc, char *argv[])
{
PyObject *pName, *pModule, *pFunc;
PyObject *pArgs, *pValue;
int i;
if (argc < 3) {
fprintf(stderr,"Usage: call pythonfile funcname [args]n");
return 1;
}
Py_Initialize();
// I had to add the following two lines to make it work
PyRun_SimpleString("import sys");
PyRun_SimpleString("sys.path.append(".")");
pName = PyUnicode_DecodeFSDefault(argv[1]);
/* Error checking of pName left out */
pModule = PyImport_Import(pName);
Py_DECREF(pName);
if (pModule != NULL) {
pFunc = PyObject_GetAttrString(pModule, argv[2]);
/* pFunc is a new reference */
if (pFunc && PyCallable_Check(pFunc)) {
pArgs = PyTuple_New(argc - 3);
for (i = 0; i < argc - 3; ++i) {
pValue = PyLong_FromLong(atoi(argv[i + 3]));
if (!pValue) {
Py_DECREF(pArgs);
Py_DECREF(pModule);
fprintf(stderr, "Cannot convert argumentn");
return 1;
}
/* pValue reference stolen here: */
PyTuple_SetItem(pArgs, i, pValue);
}
pValue = PyObject_CallObject(pFunc, pArgs);
Py_DECREF(pArgs);
if (pValue != NULL) {
printf("Result of call: %ldn", PyLong_AsLong(pValue));
Py_DECREF(pValue);
}
else {
Py_DECREF(pFunc);
Py_DECREF(pModule);
PyErr_Print();
fprintf(stderr,"Call failedn");
return 1;
}
}
else {
if (PyErr_Occurred())
PyErr_Print();
fprintf(stderr, "Cannot find function "%s"n", argv[2]);
}
Py_XDECREF(pFunc);
Py_DECREF(pModule);
}
else {
PyErr_Print();
fprintf(stderr, "Failed to load "%s"n", argv[1]);
return 1;
}
if (Py_FinalizeEx() < 0) {
return 120;
}
return 0;
}
File multiply.py
def multiply(a,b):
print("Will compute", a, "times", b)
c = 0
for i in range(0, a):
c = c + b
return c
We need to compile and link it. This can be done with the command:
gcc c_function.c -c $(python3.6-config --cflags) -fPIC
followed by
gcc c_function.o $(python3.6-config --ldflags) -o call
for the example of Python 3.6. Afterwards, the example from the Python docs can just be executed by
./call multiply multiply 3 2