C++ & Python: Pass and return a 2D double pointer array from python to c++
Question:
I want to pass a 2D array from Python to a C++ function and then return an array of the same type, same dimensions, to Python. I am aware this question has already been asked several times, but I haven’t been able to find a relevant answer to my question. For my problem, I must use a double pointer array and have the function returning a double pointer array (not void as many examples show).
My C++ function is:
#include <stdio.h>
#include <stdlib.h>
extern "C" double** dot(double **a, int m, int n){
double **arr = (double **)malloc(m * sizeof(double *));
for (int i=0; i<m; i++)
arr[i] = (double*)malloc(n * sizeof(double));
for (int i=0; i < m; i++){
for (int j=0; j < n; j++){
arr[i][j] = a[i][j];
}
}
return arr;
}
For the moment, I have used Ctypes. I know I could use the Swig interface but I would prefer avoiding it given that I don’t know it very well. However, I am still open to any suggestion. My problem if I had to use Swig is that, if I’m not mistaking, I would have to use a Typemap in order to decompose the pointer structure, and it’s a part I don’t understand very well.
What I have tried for the moment in Python is:
import ctypes as c
import numpy as np
ty_ = np.ctypeslib._ctype_ndarray(c.POINTER(c.POINTER(c.c_double)), (3,3))
x = np.arange(9.).reshape(3,3)
_dll = ctypes.CDLL('./double_2D.so')
_foobar = _dll.dot
_foobar.argtype = type(y)
_foobar.restype = type(y)
d = _foobar(y, 3, 3) #I would like d to be a nice matrix like x
I have also tried
c.cast(_foobar(y,3,3), c.POINTER(c.POINTER(c.c_double)))
But none of the examples above work. So therefore, any suggestion for defining the argtype or restype, or a snippet for Typemap in Swig would be of great help.
Answers:
Listing [Python.Docs]: ctypes – A foreign function library for Python.
Couple of thoughts:
-
"double pointer array" is misleading:
-
There is no array
-
"double pointer" might mean either pointer to double or pointer to pointer to something (including double)
-
The solution with double pointers (to double 🙂 ) seems a bit complex, (as also specified in the comments). I tend to think it’s an XY Problem. Normally, one should only deal with simple pointers, especially if their knowledge in this area isn’t very strong (and this seems to apply here, as I noticed from the other questions identical (or very similar) to this one, that you submitted and then deleted)
Anyway, here’s a simple example for demo purposes.
dll00.c:
#include <stdlib.h>
#include <stdio.h>
#if defined(_WIN32)
# define DLL00_EXPORT_API __declspec(dllexport)
#else
# define DLL00_EXPORT_API
#endif
#if defined(__cplusplus)
extern "C" {
#endif
DLL00_EXPORT_API double** init(double **ppMat, int m, int n);
DLL00_EXPORT_API int cleanup(double **ppMat, int m);
#if defined(__cplusplus)
}
#endif
DLL00_EXPORT_API double** init(double **ppMat, int m, int n)
{
const double factor = 7.0;
printf("n----- FROM C: Multiplying input matrix by: %.3fn", factor);
double **ret = malloc(m * sizeof(double*));
for (int i = 0; i < m; i++) {
ret[i] = malloc(n * sizeof(double));
for (int j = 0; j < n; j++) {
ret[i][j] = ppMat[i][j] * factor;
}
}
return ret;
}
DLL00_EXPORT_API int cleanup(double **ppMat, int m)
{
int ret = 0;
if (ppMat) {
printf("n----- FROM C: freen");
for (int i = 0; i < m; i++) {
free(ppMat[i]);
ret++;
ppMat[i] = NULL;
}
free(ppMat);
}
return ++ret;
}
code00.py:
#!/usr/bin/env python
import ctypes as cts
import sys
from pprint import pprint as pp
DLL_NAME = "./dll00.{:s}".format("dll" if sys.platform[:3].lower() == "win" else "so")
def ptr2d_to_mat(ptr, rows, cols):
return tuple(tuple(ptr[i][j] for j in range(cols)) for i in range(rows))
def main(*argv):
dll00 = cts.CDLL(DLL_NAME)
init = dll00.init
cleanup = dll00.cleanup
rows = 4
cols = 6
DblPtr = cts.POINTER(cts.c_double)
DblPtrPtr = cts.POINTER(DblPtr)
init.argtypes = (DblPtrPtr, cts.c_int, cts.c_int)
init.restype = DblPtrPtr
cleanup.argtypes = (DblPtrPtr, cts.c_int)
cleanup.restype = cts.c_int
DblPtrArr = DblPtr * rows
DblArr = cts.c_double * cols
DblArrArr = DblArr * rows
first_value = 6
in_mat = tuple(tuple(range(cols * i + first_value, cols * (i + 1) + first_value)) for i in range(rows))
print("Input matrix:")
pp(in_mat)
in_arr = DblArrArr(*in_mat)
in_ptr = cts.cast(DblPtrArr(*(cts.cast(row, DblPtr) for row in in_arr)), DblPtrPtr) # Cast each row and the final array to (corresponding) pointers
out_ptr = init(in_ptr, rows, cols)
out_mat = ptr2d_to_mat(out_ptr, rows, cols)
cleanup(out_ptr, rows)
print("nOutput matrix:")
pp(out_mat)
if __name__ == "__main__":
print("Python {:s} {:03d}bit on {:s}n".format(" ".join(item.strip() for item in sys.version.split("n")),
64 if sys.maxsize > 0x100000000 else 32, sys.platform))
rc = main(*sys.argv[1:])
print("nDone.n")
sys.exit(rc)
Output:
[cfati@CFATI-5510-0:e:WorkDevStackOverflowq058226790]> sopr.bat
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###
[prompt]> "c:Installx86MicrosoftVisual Studio Community2017VCAuxiliaryBuildvcvarsall.bat" x64>nul
[prompt]> dir /b
code00.py
dll00.c
[prompt]> cl /nologo /DDLL dll00.c /link /NOLOGO /DLL /OUT:dll00.dll
dll00.c
Creating library dll00.lib and object dll00.exp
[prompt]> dir /b
code00.py
dll00.c
dll00.dll
dll00.exp
dll00.lib
dll00.obj
[prompt]> "e:WorkDevVEnvspy_064_03.07.03_test0Scriptspython.exe" code00.py
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)] 064bit on win32
Input matrix:
((6, 7, 8, 9, 10, 11),
(12, 13, 14, 15, 16, 17),
(18, 19, 20, 21, 22, 23),
(24, 25, 26, 27, 28, 29))
----- FROM C: Multiplying input matrix by: 7.000
----- FROM C: free
Output matrix:
((42.0, 49.0, 56.0, 63.0, 70.0, 77.0),
(84.0, 91.0, 98.0, 105.0, 112.0, 119.0),
(126.0, 133.0, 140.0, 147.0, 154.0, 161.0),
(168.0, 175.0, 182.0, 189.0, 196.0, 203.0))
Done.
You can also take a look at [SO]: Problems with passing and getting arrays for a C function using ctypes (@CristiFati’s answer), which is very similar (almost identical, I’d say) to this one.
I want to pass a 2D array from Python to a C++ function and then return an array of the same type, same dimensions, to Python. I am aware this question has already been asked several times, but I haven’t been able to find a relevant answer to my question. For my problem, I must use a double pointer array and have the function returning a double pointer array (not void as many examples show).
My C++ function is:
#include <stdio.h>
#include <stdlib.h>
extern "C" double** dot(double **a, int m, int n){
double **arr = (double **)malloc(m * sizeof(double *));
for (int i=0; i<m; i++)
arr[i] = (double*)malloc(n * sizeof(double));
for (int i=0; i < m; i++){
for (int j=0; j < n; j++){
arr[i][j] = a[i][j];
}
}
return arr;
}
For the moment, I have used Ctypes. I know I could use the Swig interface but I would prefer avoiding it given that I don’t know it very well. However, I am still open to any suggestion. My problem if I had to use Swig is that, if I’m not mistaking, I would have to use a Typemap in order to decompose the pointer structure, and it’s a part I don’t understand very well.
What I have tried for the moment in Python is:
import ctypes as c
import numpy as np
ty_ = np.ctypeslib._ctype_ndarray(c.POINTER(c.POINTER(c.c_double)), (3,3))
x = np.arange(9.).reshape(3,3)
_dll = ctypes.CDLL('./double_2D.so')
_foobar = _dll.dot
_foobar.argtype = type(y)
_foobar.restype = type(y)
d = _foobar(y, 3, 3) #I would like d to be a nice matrix like x
I have also tried
c.cast(_foobar(y,3,3), c.POINTER(c.POINTER(c.c_double)))
But none of the examples above work. So therefore, any suggestion for defining the argtype or restype, or a snippet for Typemap in Swig would be of great help.
Listing [Python.Docs]: ctypes – A foreign function library for Python.
Couple of thoughts:
-
"double pointer array" is misleading:
-
There is no array
-
"double pointer" might mean either pointer to double or pointer to pointer to something (including double)
-
-
The solution with double pointers (to double 🙂 ) seems a bit complex, (as also specified in the comments). I tend to think it’s an XY Problem. Normally, one should only deal with simple pointers, especially if their knowledge in this area isn’t very strong (and this seems to apply here, as I noticed from the other questions identical (or very similar) to this one, that you submitted and then deleted)
Anyway, here’s a simple example for demo purposes.
dll00.c:
#include <stdlib.h>
#include <stdio.h>
#if defined(_WIN32)
# define DLL00_EXPORT_API __declspec(dllexport)
#else
# define DLL00_EXPORT_API
#endif
#if defined(__cplusplus)
extern "C" {
#endif
DLL00_EXPORT_API double** init(double **ppMat, int m, int n);
DLL00_EXPORT_API int cleanup(double **ppMat, int m);
#if defined(__cplusplus)
}
#endif
DLL00_EXPORT_API double** init(double **ppMat, int m, int n)
{
const double factor = 7.0;
printf("n----- FROM C: Multiplying input matrix by: %.3fn", factor);
double **ret = malloc(m * sizeof(double*));
for (int i = 0; i < m; i++) {
ret[i] = malloc(n * sizeof(double));
for (int j = 0; j < n; j++) {
ret[i][j] = ppMat[i][j] * factor;
}
}
return ret;
}
DLL00_EXPORT_API int cleanup(double **ppMat, int m)
{
int ret = 0;
if (ppMat) {
printf("n----- FROM C: freen");
for (int i = 0; i < m; i++) {
free(ppMat[i]);
ret++;
ppMat[i] = NULL;
}
free(ppMat);
}
return ++ret;
}
code00.py:
#!/usr/bin/env python
import ctypes as cts
import sys
from pprint import pprint as pp
DLL_NAME = "./dll00.{:s}".format("dll" if sys.platform[:3].lower() == "win" else "so")
def ptr2d_to_mat(ptr, rows, cols):
return tuple(tuple(ptr[i][j] for j in range(cols)) for i in range(rows))
def main(*argv):
dll00 = cts.CDLL(DLL_NAME)
init = dll00.init
cleanup = dll00.cleanup
rows = 4
cols = 6
DblPtr = cts.POINTER(cts.c_double)
DblPtrPtr = cts.POINTER(DblPtr)
init.argtypes = (DblPtrPtr, cts.c_int, cts.c_int)
init.restype = DblPtrPtr
cleanup.argtypes = (DblPtrPtr, cts.c_int)
cleanup.restype = cts.c_int
DblPtrArr = DblPtr * rows
DblArr = cts.c_double * cols
DblArrArr = DblArr * rows
first_value = 6
in_mat = tuple(tuple(range(cols * i + first_value, cols * (i + 1) + first_value)) for i in range(rows))
print("Input matrix:")
pp(in_mat)
in_arr = DblArrArr(*in_mat)
in_ptr = cts.cast(DblPtrArr(*(cts.cast(row, DblPtr) for row in in_arr)), DblPtrPtr) # Cast each row and the final array to (corresponding) pointers
out_ptr = init(in_ptr, rows, cols)
out_mat = ptr2d_to_mat(out_ptr, rows, cols)
cleanup(out_ptr, rows)
print("nOutput matrix:")
pp(out_mat)
if __name__ == "__main__":
print("Python {:s} {:03d}bit on {:s}n".format(" ".join(item.strip() for item in sys.version.split("n")),
64 if sys.maxsize > 0x100000000 else 32, sys.platform))
rc = main(*sys.argv[1:])
print("nDone.n")
sys.exit(rc)
Output:
[cfati@CFATI-5510-0:e:WorkDevStackOverflowq058226790]> sopr.bat ### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ### [prompt]> "c:Installx86MicrosoftVisual Studio Community2017VCAuxiliaryBuildvcvarsall.bat" x64>nul [prompt]> dir /b code00.py dll00.c [prompt]> cl /nologo /DDLL dll00.c /link /NOLOGO /DLL /OUT:dll00.dll dll00.c Creating library dll00.lib and object dll00.exp [prompt]> dir /b code00.py dll00.c dll00.dll dll00.exp dll00.lib dll00.obj [prompt]> "e:WorkDevVEnvspy_064_03.07.03_test0Scriptspython.exe" code00.py Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)] 064bit on win32 Input matrix: ((6, 7, 8, 9, 10, 11), (12, 13, 14, 15, 16, 17), (18, 19, 20, 21, 22, 23), (24, 25, 26, 27, 28, 29)) ----- FROM C: Multiplying input matrix by: 7.000 ----- FROM C: free Output matrix: ((42.0, 49.0, 56.0, 63.0, 70.0, 77.0), (84.0, 91.0, 98.0, 105.0, 112.0, 119.0), (126.0, 133.0, 140.0, 147.0, 154.0, 161.0), (168.0, 175.0, 182.0, 189.0, 196.0, 203.0)) Done.
You can also take a look at [SO]: Problems with passing and getting arrays for a C function using ctypes (@CristiFati’s answer), which is very similar (almost identical, I’d say) to this one.