Running C++ code alongside and interacting with Python
Question:
So my current project is mostly in Python, but I’m looking to rewrite the most computationally expensive portions in C++ to try and boost performance. Much of this I can achieve via simple functions loaded from DLL files, but not everything. I have a multidimensional array in Python that I want to perform operations on in C++ (particularly A* pathfinding), but I’m not really sure how to translate them, and constantly sending data one piece at a time into a loaded function seems really inefficient (the array’s first two dimensions are in the low hundreds, and the functions will need to deal with scores, if not hundreds, of elements in the array at a time).
My idea was to have a class in C++ that creates its own copy of the array at setup (where performance isn’t as much of an issue) and has methods that are performed on the array and return data to the main Python program. However, I’m not sure how to accomplish this, and even if this is the proper way to go about such a thing; that would seem to imply having C++ code running parallel to the main Python program, and intuition tells me that’s a bad idea.
I don’t know much about integrating C++ and Python beyond how to load simple functions via cTypes in Python, so I’d really appreciate some pointers here. Keep in mind I’m relatively new to C++ at all; most of my programming experience is in Python. What would be the best way to fit the two together in this situation?
Answers:
Take a look at Cython.
First and foremost, when you are working with multidimensional arrays in Python, you should really be using NumPy. Chances are that your program is already fast enough when you let NumPy do the number crunching (use Array arithmetic instead of Python for loops).
If this is not enough, consider writing parts of your program using Cython. Cython also supports NumPy arrays and provides a painless way to write C code using Python-like syntax.
If it really must be C++, I highly recommend using Boost.Python. Bridging Python and C++ has never been this easy. Plus, Boost.Python comes with NumPy support as well (boost::numeric::array).
So my current project is mostly in Python, but I’m looking to rewrite the most computationally expensive portions in C++ to try and boost performance. Much of this I can achieve via simple functions loaded from DLL files, but not everything. I have a multidimensional array in Python that I want to perform operations on in C++ (particularly A* pathfinding), but I’m not really sure how to translate them, and constantly sending data one piece at a time into a loaded function seems really inefficient (the array’s first two dimensions are in the low hundreds, and the functions will need to deal with scores, if not hundreds, of elements in the array at a time).
My idea was to have a class in C++ that creates its own copy of the array at setup (where performance isn’t as much of an issue) and has methods that are performed on the array and return data to the main Python program. However, I’m not sure how to accomplish this, and even if this is the proper way to go about such a thing; that would seem to imply having C++ code running parallel to the main Python program, and intuition tells me that’s a bad idea.
I don’t know much about integrating C++ and Python beyond how to load simple functions via cTypes in Python, so I’d really appreciate some pointers here. Keep in mind I’m relatively new to C++ at all; most of my programming experience is in Python. What would be the best way to fit the two together in this situation?
Take a look at Cython.
First and foremost, when you are working with multidimensional arrays in Python, you should really be using NumPy. Chances are that your program is already fast enough when you let NumPy do the number crunching (use Array arithmetic instead of Python for loops).
If this is not enough, consider writing parts of your program using Cython. Cython also supports NumPy arrays and provides a painless way to write C code using Python-like syntax.
If it really must be C++, I highly recommend using Boost.Python. Bridging Python and C++ has never been this easy. Plus, Boost.Python comes with NumPy support as well (boost::numeric::array).