How do I wrap a C++ class with Cython?
Question:
I have a C++ class. It’s made up of one .ccp file and one .h file. It compiles (I can write a main method that uses it successfully in c++). How do I wrap this class with Cython to make it available in Python?
I’ve read the docs and don’t follow. They talk about generating the cpp file. When I’ve tried to follow the docs, my already existing cpp gets blown away…
What am I meant to put in the pyx file? I’ve been told the class definition but how much of it? Just the public methods?
Do I need a .pxd file? I don’t understand when this file is or isn’t required.
I’ve tried asking these question in the #python IRC channel and can’t get an answer.
Answers:
Cython is mainly for C development, to integrate C++ with Python I would recommend Boost.Python. Their excellent documentation should get you started pretty quickly.
Even Cython is generally for use with C, it can generate C++ code, too. When compiling, you add the --cplus flag.
Now, creating a wrapper for the class is simple and not much different from wrapping a structure. It mainly differs from declaring the extern, but that’s not much difference at all.
Suppose you have a class MyCppClass in mycppclass.h.
cdef extern from "mycppclass.h":
cppclass MyCppClass:
int some_var
MyCppClass(int, char*)
void doStuff(void*)
char* getStuff(int)
cdef class MyClass:
# the public-modifier will make the attribute public for cython,
# not for python. Maybe you need to access the internal C++ object from
# outside of the class. If not, you better declare it as private by just
# leaving out the `private` modifier.
# ---- EDIT ------
# Sorry, this statement is wrong. The `private` modifier would make it available to Python,
# so the following line would cause an error es the Pointer to MyCppClass
# couldn't be converted to a Python object.
#>> cdef public MyCppClass* cobj
# correct is:
cdef MyCppClass* obj
def __init__(self, int some_var, char* some_string):
self.cobj = new MyCppClass(some_var, some_string)
if self.cobj == NULL:
raise MemoryError('Not enough memory.')
def __del__(self):
del self.cobj
property some_var:
def __get__(self):
return self.cobj.some_var
def __set__(self, int var):
self.cobj.some_var = var
Note that the new keyword is only available when the --cplus flag is set, otherwise use malloc from <stdlib.h> by externing it.
Also note that you don’t need to dereference the pointer (->) to call the method. Cython tracks the object’s type and applies what fits.
.pxd files are for seperating declarations from implementation, or to avoid namespace colliding. Imagine you’d like to name you Python-wrapper like the C++ class. Simply put in your .pxd file the extern declarations and cimport the pxd file in the .pyx.
cimport my_pxd
cdef my_pxd.SomeExternedType obj
Note that you can not write implementations in a .pxd file.
So after lots of poking, trial and error, screaming and tearing my hair out, I finally got this to work. First though, I had to re-write my C++ into C, which for me really just involved converting all my std::string variables to char* and keeping track of some lengths.
Once done I had my .h and .c files. I wanted to make a single function from the C code available in Python. It turns out that Cython can compile your C files into the extension for you and link any libraries all in one go, so starting with my setup.py, it ended up looking like this:
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
ext_modules=[
Extension("myext",
["myext.pyx", "../stuff.c"],
libraries=["ssl", "crypto"]
)
]
setup(
name = "myext",
cmdclass = {"build_ext": build_ext},
ext_modules = ext_modules
)
As you can see, the second argument to the Extension simply lists all the files that need to be compiled, Cython works out how to compile them depending on their file extension as far as I can tell. The libraries array tells the Cython compiler what needs to be linked in (in this case I was wrapping some crypto stuff that I couldn’t seem to mimick directly through existing Python libs).
To actually make my C function available in the .pyx file, you write a small wrapper in the .pxd. My myext.pxd looked as below:
cdef extern from "../stuff.h":
char* myfunc(char* arg1, char* arg2, char* arg3)
In the .pyx you then use the cimport declaration to import this function, which is then available for use as if it were any other Python function:
cimport myext
def my_python_func(arg1, arg2, arg3):
result = myext.myfunc(arg1, arg2, arg3)
return result
When you build this (on Mac at least) you get a .so that you can import in python and run the functions from the .pyx. There may be better, more correct way to get this all working but that comes from experience and this was a first encounter that I managed to work out. I’d be very interested on pointers where I may have gone wrong.
Update:
After further use of Cython, I found it was super simple to integrate it with C++ too, once you know what you’re doing. Making C++’s string available is as simple as from libcpp.string cimport string in your pyx/pyd. Declaring the C++ class is similarly easy as:
cdef extern from "MyCPPClass.h":
cdef cppclass MyCPPClass:
int foo;
string bar;
Sure you have to basically redeclare the .h definition of your class in a Pythonic format, but that’s a small price to pay for getting access to your already written C++ functions.
The answers above have more or less answered the OP’s question.
But its now passing the mid of 2020 and I’d thought to put a contribution (in the form of resource summary) here for those who want to explore Python-C++ bindings via python packages that can be ‘pip installed’ (or in the PyPI).
I’d reckon for calling C++ from Python / wrapping over third party libs , one can look at cppyy. Its a relatively young project but it certainly looks promising and backed with a modern compiler. See the link below:
https://cppyy.readthedocs.io/en/latest/
Also, there’s always pybind11 ….
https://github.com/pybind/pybind11
Cython also supports C++ natively (for most of the C++ languages) ; for more information see https://cython.readthedocs.io/en/latest/src/userguide/wrapping_CPlusPlus.html
I have a C++ class. It’s made up of one .ccp file and one .h file. It compiles (I can write a main method that uses it successfully in c++). How do I wrap this class with Cython to make it available in Python?
I’ve read the docs and don’t follow. They talk about generating the cpp file. When I’ve tried to follow the docs, my already existing cpp gets blown away…
What am I meant to put in the pyx file? I’ve been told the class definition but how much of it? Just the public methods?
Do I need a .pxd file? I don’t understand when this file is or isn’t required.
I’ve tried asking these question in the #python IRC channel and can’t get an answer.
Cython is mainly for C development, to integrate C++ with Python I would recommend Boost.Python. Their excellent documentation should get you started pretty quickly.
Even Cython is generally for use with C, it can generate C++ code, too. When compiling, you add the --cplus flag.
Now, creating a wrapper for the class is simple and not much different from wrapping a structure. It mainly differs from declaring the extern, but that’s not much difference at all.
Suppose you have a class MyCppClass in mycppclass.h.
cdef extern from "mycppclass.h":
cppclass MyCppClass:
int some_var
MyCppClass(int, char*)
void doStuff(void*)
char* getStuff(int)
cdef class MyClass:
# the public-modifier will make the attribute public for cython,
# not for python. Maybe you need to access the internal C++ object from
# outside of the class. If not, you better declare it as private by just
# leaving out the `private` modifier.
# ---- EDIT ------
# Sorry, this statement is wrong. The `private` modifier would make it available to Python,
# so the following line would cause an error es the Pointer to MyCppClass
# couldn't be converted to a Python object.
#>> cdef public MyCppClass* cobj
# correct is:
cdef MyCppClass* obj
def __init__(self, int some_var, char* some_string):
self.cobj = new MyCppClass(some_var, some_string)
if self.cobj == NULL:
raise MemoryError('Not enough memory.')
def __del__(self):
del self.cobj
property some_var:
def __get__(self):
return self.cobj.some_var
def __set__(self, int var):
self.cobj.some_var = var
Note that the new keyword is only available when the --cplus flag is set, otherwise use malloc from <stdlib.h> by externing it.
Also note that you don’t need to dereference the pointer (->) to call the method. Cython tracks the object’s type and applies what fits.
.pxd files are for seperating declarations from implementation, or to avoid namespace colliding. Imagine you’d like to name you Python-wrapper like the C++ class. Simply put in your .pxd file the extern declarations and cimport the pxd file in the .pyx.
cimport my_pxd
cdef my_pxd.SomeExternedType obj
Note that you can not write implementations in a .pxd file.
So after lots of poking, trial and error, screaming and tearing my hair out, I finally got this to work. First though, I had to re-write my C++ into C, which for me really just involved converting all my std::string variables to char* and keeping track of some lengths.
Once done I had my .h and .c files. I wanted to make a single function from the C code available in Python. It turns out that Cython can compile your C files into the extension for you and link any libraries all in one go, so starting with my setup.py, it ended up looking like this:
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
ext_modules=[
Extension("myext",
["myext.pyx", "../stuff.c"],
libraries=["ssl", "crypto"]
)
]
setup(
name = "myext",
cmdclass = {"build_ext": build_ext},
ext_modules = ext_modules
)
As you can see, the second argument to the Extension simply lists all the files that need to be compiled, Cython works out how to compile them depending on their file extension as far as I can tell. The libraries array tells the Cython compiler what needs to be linked in (in this case I was wrapping some crypto stuff that I couldn’t seem to mimick directly through existing Python libs).
To actually make my C function available in the .pyx file, you write a small wrapper in the .pxd. My myext.pxd looked as below:
cdef extern from "../stuff.h":
char* myfunc(char* arg1, char* arg2, char* arg3)
In the .pyx you then use the cimport declaration to import this function, which is then available for use as if it were any other Python function:
cimport myext
def my_python_func(arg1, arg2, arg3):
result = myext.myfunc(arg1, arg2, arg3)
return result
When you build this (on Mac at least) you get a .so that you can import in python and run the functions from the .pyx. There may be better, more correct way to get this all working but that comes from experience and this was a first encounter that I managed to work out. I’d be very interested on pointers where I may have gone wrong.
Update:
After further use of Cython, I found it was super simple to integrate it with C++ too, once you know what you’re doing. Making C++’s string available is as simple as from libcpp.string cimport string in your pyx/pyd. Declaring the C++ class is similarly easy as:
cdef extern from "MyCPPClass.h":
cdef cppclass MyCPPClass:
int foo;
string bar;
Sure you have to basically redeclare the .h definition of your class in a Pythonic format, but that’s a small price to pay for getting access to your already written C++ functions.
The answers above have more or less answered the OP’s question.
But its now passing the mid of 2020 and I’d thought to put a contribution (in the form of resource summary) here for those who want to explore Python-C++ bindings via python packages that can be ‘pip installed’ (or in the PyPI).
I’d reckon for calling C++ from Python / wrapping over third party libs , one can look at cppyy. Its a relatively young project but it certainly looks promising and backed with a modern compiler. See the link below:
https://cppyy.readthedocs.io/en/latest/
Also, there’s always pybind11 ….
https://github.com/pybind/pybind11
Cython also supports C++ natively (for most of the C++ languages) ; for more information see https://cython.readthedocs.io/en/latest/src/userguide/wrapping_CPlusPlus.html