How to pass Python list to C function using Cython

Question

I am using a Raspberry Pi to interface with custom hardware connected to the GPIO. The controlling software is written in Python, and the interface to the custom hardware is written in C, as it is a much faster C implementation. I now need to start calling my C functions from my Python, and have recently been learning how to wrap C in Cython. I have got everything to work, except passing a Python list to a C function.

My custom hardware needs to be sent anywhere from 1 to 32 bytes, hence the use of an array.

The Cython tutorials and other references I have read online either are really simple, and do not include how to pass lists to C, use numpy, which I am not using, or use very complicated code examples that lack sufficient documentation for me to understand it properly.

What I have now are:

test.c

#include <stdio.h>
#include "test.h"
void pop(void) {
    a[0] = 0x55;
    a[1] = 0x66;
    a[2] = 0x77;
    a[3] = '';
}
void putAll(int n, char c[]) {
    memcpy(a, c, n);
}
char *getAll(void) {
    return &a[0];
}

test.h

char a[4];

void putAll(int n, char[]);
char *getAll(void);

pytest.pyx

cimport defns

# Populate C array with values
def pypop():
    defns.pop()

# Pass python list to C
def pyPutAll(int n, char[:] pyc):
    cdef char* c = pyc
    defns.putAll(n, c)

# Get array from C
def pyGetAll():
    cdef char* c = defns.getAll()
    cdef bytes pyc = c
    print pyc

defns.pxd

cdef extern from "test.h":
    char a[4]
    void pop()
    void putAll(int n, char c[])
    char *getAll()

Using the tutorials at cython.org, my getAll() and pop() functions work, but when I include the putAll() function (taken from the process_byte_data example code found at the link, under Unicode and passing strings > Accepting strings from Python code), I get this error:

python setup.py build_ext -i

Error compiling Cython file:
------------------------------------------------------------
...

def pyputAll(int n, char[:] pyc):
                        ^
------------------------------------------------------------

pytest.pyx:13:25: Expected an identifier or literal

Now, I have a way around this – combining up to 32 bytes into an int and passing as a long int, and then pulling it apart in C – but it is very ugly.

Also, I do not require Cython for any performance gains, other than that of using the C implemented library for interfacing with my custom hardware vs a Python implemented one.

Asked By: Hengy

||

Source

Answer 1

ctypes is better suited to what you are trying to do.

For instance: (test.py)

from ctypes import create_string_buffer, c_char_p, c_int, CDLL

libtest = CDLL("./libtest.so")

_putAll = libtest.putAll
_putAll.restype = None
_putAll.argtypes = [c_int, c_char_p]

def putAll(values):
    """Expects a bytes object, bytearray, or a list of ints."""
    char_buffer = create_string_buffer(bytes(values))
    _putAll(len(char_buffer), char_buffer)

getAll = libtest.getAll
getAll.restype = c_char_p
getAll.argtypes = None

Usage:

import test
test.putAll(b"hello world")
assert test.getAll() == b"hello world"
test.putAll(bytearray(b"another string"))
test.putAll([1, 2, 3, 255])

The above is for python 3 . If you’re running python 2 then you can substitute bytes for str, but the function is less flexible. In addition, be aware that create_string_buffer creates a C-string (adds an additional NUL character on the end of the string).

To compile the shared library you need to do the following:

gcc -fPIC -g -c -Wall test.c
gcc -shared -Wl,-soname,libtest.so.1 -o libtest.so test.o

Answered By: Dunes

Answer 2

You can use Python’s bytearray with Cython and I think is cleaner and easier than ctypes:

test.py

larr = bytearray([4, 1, 2])
pyPutAll(3, larr)

This works with your original putAll C function:

test.c

...
void putAll(int n, char c[]) {
    memcpy(a, c, n);
}
...

pytest.pyx

# Pass python bytearray to C
def pyPutAll(int n, char[:] pyc):
    defns.putAll(n, &pyc[0])

If you need to pass a list, you would have to convert it to a vector inside the pyx function, and pass a reference to that instead:

pytest.pyx

def pyPutAllList(int n, list pyc):
    cdef vector[char] vec = pyc
    defns.putAll(n, &vec[0])

Answered By: bio_c

Answer 3

I managed to get this working. Here is the code I now have for anyone who needs it.

pytest.pyx:

...
def pyPutAll(int n, c):
    cdef int *ptr
    ptr = <int *>malloc(n*cython.sizeof(int))
    if ptr is NULL:
            raise MemoryError()
    for i in xrange(n):
            ptr[i] = c[i]
    defns.putAll(n, ptr)
    free(ptr)
...

test.c:

void putAll(int n, int c[])
{
    char d[n];
    int i;
    for (i=0;i<n;i++) {
            d[i] = c[i];
    }
    memcpy(addr, d, n);
}

This code is not optimal, as it uses ints in the python/cython code, then converts it to char in the C function. The pyPutAll() function in pytest.pyc accepts an ordinary python list. It then creates a C pointer and allocates memory. Iterating through the list, each value is put into a C array, and then finally passes the pointer to the C function.

It gets the job done, but I am sure someone else can give a much more efficient solution.

_{This answer was posted as an edit to the question How to pass Python list to C function using Cython by the OP Hengy under CC BY-SA 3.0.}

Answered By: vvvvv

How to pass Python list to C function using Cython

Question:

Answers: