PyCUDA + numpy, and working with strings in general
Question:
I’m relatively new to all of the things mentioned in the title, so bear with me.
Currently I’m stuck with the conversion between python and C. Since CUDA kernels are written in C, so I can’t just look at it the python way.
As the documentation is rather limited, and overly complex for a beginner, I’d like to ask how pyCuda actually converts python(or numpy) arrays for use in C.
For example, a string “stuff” is an array of characters in C, but in python it’s an immutable string. However I can do the following:
stuff = "stuff"
d_stuff = cuda.mem_alloc(len(stuff))
cuda.memcpy_htod(d_stuff, stuff)
And in the CUDA kernel, now I can use it as a char* d_stuff.
However I cannot get it back the same way, as a python string is immutable. So doing the following will obviously raise an error:
newstuff = ""
cuda.memcpy_dtoh(newstuff, d_stuff)
I know that these can be written as
d_stuff = gpuarray.to_gpu(numpy.array(stuff)) # I need numpy, as the to_gpu expects an array
newstuff = d_stuff.get()
But I have no idea how it works, and what it does behind the scenes, so I would really appreciate, if anyone could explain how the conversions work briefly. (E.g. how does the second example give back a string)
Also I have questions about arrays created with numpy. I’ve seen that they’re widely used for GPUs but I don’t know how they work.
Does giving numpy a string create an array of characters in terms of C code, if yes, does an array of strings become char, or something else? (when translated to C of course)
Writing CUDA code probably would be better with C only, but I’d like to explore the features of python, and I’m doing all this for learning purposes.
Answers:
I’d like to ask how PyCUDA actually converts python(or numpy) arrays for use in C.
It doesn’t. PyCUDA simply takes any object which supports the Python buffer protocol (typically a numpy array) and directly accesses its host memory buffer to transfer data to and from the GPU. No type conversion or data manipulation is ever performed. Types are directly inferred from the CTypes interface, (typically via a numpy dtype, given that numpy arrays are the usual data source).
Does giving numpy a string create an array of characters in terms of C code, if yes, does an array of strings become char, or something else?
That depends. For example doing this:
ttt = np.asarray([ "stuff" + str(i) for i in range(0,20) ])
print( ttt.dtype, type(ttt[0]) )
|S7 <type 'numpy.string_'>
Here numpy uses a special fixed length string data type whose length is calculated from the input data. This is effective a C ordered array of char[7]. See more here. PyCUDA automagically understands how to handle this because of the buffer protocol and the underlying direct mapping to a native C type.
However you can also do this:
ttt = np.asarray([ "stuff" + str(i) for i in range(0,20) ], dtype=object)
print( ttt.dtype, type(ttt[0]) )
object <type 'str'>
Here, the numpy array created contains Python objects (in this case a string). This is not something which can be used in PyCUDA, because the Python object doesn’t have a direct representation in C.
could you give an Example how kernel function use the strings?
%%writefile s.py
#!python
import pycuda.driver as cuda
import pycuda.autoinit
from pycuda.compiler import SourceModule
from pycuda import gpuarray
import numpy as np
mod = SourceModule("""
#include <stdio.h>
__global__ void say_hi( char *out_gpu)
{
int idx = threadIdx.x + blockIdx.x * blockDim.x;
printf("thread id: [%d] %s\n",idx,out_gpu);
//print c-str
for (int i=0;i<5;i++){
printf("%s",(out_gpu+i));
}
printf(" >end\n");
}
""")
func = mod.get_function("say_hi")
s= np.array(['stzabcdf'], dtype=np.str_)
print('shape:',s.shape)
print(s,s.data)
s_gpu = gpuarray.to_gpu(s)
func(s_gpu,block=(s.size,1,1),grid=(1,1,1))
#run the s.py, why not get full output string
!python s.py
shape: (1,)
['stzabcdf'] <memory at 0x7fc80eaf3640>
thread id: [0] s
st >end
I’m relatively new to all of the things mentioned in the title, so bear with me.
Currently I’m stuck with the conversion between python and C. Since CUDA kernels are written in C, so I can’t just look at it the python way.
As the documentation is rather limited, and overly complex for a beginner, I’d like to ask how pyCuda actually converts python(or numpy) arrays for use in C.
For example, a string “stuff” is an array of characters in C, but in python it’s an immutable string. However I can do the following:
stuff = "stuff"
d_stuff = cuda.mem_alloc(len(stuff))
cuda.memcpy_htod(d_stuff, stuff)
And in the CUDA kernel, now I can use it as a char* d_stuff.
However I cannot get it back the same way, as a python string is immutable. So doing the following will obviously raise an error:
newstuff = ""
cuda.memcpy_dtoh(newstuff, d_stuff)
I know that these can be written as
d_stuff = gpuarray.to_gpu(numpy.array(stuff)) # I need numpy, as the to_gpu expects an array
newstuff = d_stuff.get()
But I have no idea how it works, and what it does behind the scenes, so I would really appreciate, if anyone could explain how the conversions work briefly. (E.g. how does the second example give back a string)
Also I have questions about arrays created with numpy. I’ve seen that they’re widely used for GPUs but I don’t know how they work.
Does giving numpy a string create an array of characters in terms of C code, if yes, does an array of strings become char, or something else? (when translated to C of course)
Writing CUDA code probably would be better with C only, but I’d like to explore the features of python, and I’m doing all this for learning purposes.
I’d like to ask how PyCUDA actually converts python(or numpy) arrays for use in C.
It doesn’t. PyCUDA simply takes any object which supports the Python buffer protocol (typically a numpy array) and directly accesses its host memory buffer to transfer data to and from the GPU. No type conversion or data manipulation is ever performed. Types are directly inferred from the CTypes interface, (typically via a numpy dtype, given that numpy arrays are the usual data source).
Does giving numpy a string create an array of characters in terms of C code, if yes, does an array of strings become char, or something else?
That depends. For example doing this:
ttt = np.asarray([ "stuff" + str(i) for i in range(0,20) ])
print( ttt.dtype, type(ttt[0]) )
|S7 <type 'numpy.string_'>
Here numpy uses a special fixed length string data type whose length is calculated from the input data. This is effective a C ordered array of char[7]. See more here. PyCUDA automagically understands how to handle this because of the buffer protocol and the underlying direct mapping to a native C type.
However you can also do this:
ttt = np.asarray([ "stuff" + str(i) for i in range(0,20) ], dtype=object)
print( ttt.dtype, type(ttt[0]) )
object <type 'str'>
Here, the numpy array created contains Python objects (in this case a string). This is not something which can be used in PyCUDA, because the Python object doesn’t have a direct representation in C.
could you give an Example how kernel function use the strings?
%%writefile s.py
#!python
import pycuda.driver as cuda
import pycuda.autoinit
from pycuda.compiler import SourceModule
from pycuda import gpuarray
import numpy as np
mod = SourceModule("""
#include <stdio.h>
__global__ void say_hi( char *out_gpu)
{
int idx = threadIdx.x + blockIdx.x * blockDim.x;
printf("thread id: [%d] %s\n",idx,out_gpu);
//print c-str
for (int i=0;i<5;i++){
printf("%s",(out_gpu+i));
}
printf(" >end\n");
}
""")
func = mod.get_function("say_hi")
s= np.array(['stzabcdf'], dtype=np.str_)
print('shape:',s.shape)
print(s,s.data)
s_gpu = gpuarray.to_gpu(s)
func(s_gpu,block=(s.size,1,1),grid=(1,1,1))
#run the s.py, why not get full output string
!python s.py
shape: (1,)
['stzabcdf'] <memory at 0x7fc80eaf3640>
thread id: [0] s
st >end