pybind11 modify numpy array from C++

Question

EDIT: It works now, I do not know why. Don’t think I changed anything

I want to pass in and modify a large numpy array with pybind11. Because it’s large I want to avoid copying it and returning a new one.

Here’s the code:

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include <pybind11/numpy.h>
#include <vector>

// C++ code
void calc_sum_cost(float* ptr, int N, int M, float* ptr_cost) {
  for(int32_t i = 1; i < N; i++) {
    for(int32_t j = 1; j < M; j++) {
      float upc = ptr[(i-1) * M + j];
      float leftc = ptr[i * M + j - 1];
      float diagc = ptr[(i-1) * M + j - 1];
      float transition_cost = std::min(upc, std::min(leftc, diagc));
      if (transition_cost == diagc) {
        transition_cost += 2 * ptr_cost[i*M + j];
      } else {
        transition_cost += ptr_cost[i*M + j];
      }
      std::cout << transition_cost << std::endl;
      ptr[i * M + j] = transition_cost;
    }
  }
}

// Interface

namespace py = pybind11;

// wrap C++ function with NumPy array IO
py::object wrapper(py::array_t<float> array,
                  py::array_t<float> arrayb) {
  // check input dimensions
  if ( array.ndim()     != 2 )
    throw std::runtime_error("Input should be 2-D NumPy array");

  auto buf = array.request();
  auto buf2 = arrayb.request();
  if (buf.size != buf2.size) throw std::runtime_error("sizes do not match!");

  int N = array.shape()[0], M = array.shape()[1];

  float* ptr = (float*) buf.ptr;
  float* ptr_cost = (float*) buf2.ptr;
  // call pure C++ function
  calc_sum_cost(ptr, N, M, ptr_cost);
  return py::cast<py::none>(Py_None);
}

PYBIND11_MODULE(fast,m) {
  m.doc() = "pybind11 plugin";
  m.def("calc_sum_cost", &wrapper, "Calculate the length of an array of vectors");
}

I think the py::array::forcecast is causing a conversion and so leaving the input matrix unmodified (in python). When I remove that though I get a runtime error, when I remove ::c_style it runs but again in python the numpy array is the same.

Basically my question is how can one pass and modify a numpy array with pybind11?

Asked By: Nimitz14

||

Source

Answer 1

I just had the same problem. If, from Python, you pass a numpy array of the type matching the C++ argument then no conversion happens, and you can modify the data in-place i.e. for py::array_t<float> argument pass in a numpy np.float32 array. If you happen to pass in a np.float64 array (the default type) then pybind11 does the conversion due to the py::array::forcecast template parameter (default on py::array_t<T>), so your C++ function only gets a converted copy of a numpy array, and any changes are lost after returning.

Answered By: Saul

Answer 2

You should define the wrapper function to pass parameters by reference:

py::object wrapper(py::array_t<float> &array,
                  py::array_t<float> &arrayb)

And just as Saul’s answer shows, you should pass numpy array with type of np.float32 to match the C++ array type, otherwise you can not change the passed arrays from Python in C++ function.

Answered By: JasVixBan

pybind11 modify numpy array from C++

Question:

Answers: