Are numpy's basic operations vectorized, i.e. do they use SIMD operations?
Question:
I am doing some performance analysis, and i wonder, whether numpy vectorizes its standard array operations, when the datatype is known (double).
a, b = (some numpy arrays)
c = a + b #Is this vectorized?
Edit:
Is this operation vectorized, i.e. will the computation consist of SIMD operations?
Answers:
Yes, they are.
/*
* This file is for the definitions of simd vectorized operations.
*
* Currently contains sse2 functions that are built on amd64, x32 or
* non-generic builds (CFLAGS=-march=...)
* In future it may contain other instruction sets like AVX or NEON detected
* at runtime in which case it needs to be included indirectly via a file
* compiled with special options (or use gcc target attributes) so the binary
* stays portable.
*/
i notice there is a comment from Quazi Irfan on henrikstroem’s answer,which says numpy doesn’t exploit vectorization and cites a blog in which the author made a "proof" by experiments.
so i go through the blog and found there is a gap may conduct a different conclusion:for numpy-array a and b,the arithmetic a*b is different with np.dot(a,b).the arithmetic(a*b) which the blog author tested is just scalar multiplication,not a matrix multiplication(np.dot(a,b)),even not a vector inner product.but the author still used a*b to compare with the original experiment which runs np.dot(a,b).the complexity of these two arithmetics are so different!
numpy certainly exploits vectorized by SIMD and BLAS,which can be found in its source code.the official numpy distribution supports a set of parallel manipulation(like np.dot),but not every functions(like np.where,np.mean).the blog author may choose an inappropriate function（a unvectorized function） to compare.
we can also see that in respect of multi-cores CPU usage.when executing numpy.dot(),all the cores are performing a high usage.Hence numpy must have vectorized(by BLAS) to avoid just using a single core because of the CPython’s GIL restriction.
I am doing some performance analysis, and i wonder, whether numpy vectorizes its standard array operations, when the datatype is known (double).
a, b = (some numpy arrays)
c = a + b #Is this vectorized?
Edit:
Is this operation vectorized, i.e. will the computation consist of SIMD operations?
Yes, they are.
/*
* This file is for the definitions of simd vectorized operations.
*
* Currently contains sse2 functions that are built on amd64, x32 or
* non-generic builds (CFLAGS=-march=...)
* In future it may contain other instruction sets like AVX or NEON detected
* at runtime in which case it needs to be included indirectly via a file
* compiled with special options (or use gcc target attributes) so the binary
* stays portable.
*/
i notice there is a comment from Quazi Irfan on henrikstroem’s answer,which says numpy doesn’t exploit vectorization and cites a blog in which the author made a "proof" by experiments.
so i go through the blog and found there is a gap may conduct a different conclusion:for numpy-array a and b,the arithmetic a*b is different with np.dot(a,b).the arithmetic(a*b) which the blog author tested is just scalar multiplication,not a matrix multiplication(np.dot(a,b)),even not a vector inner product.but the author still used a*b to compare with the original experiment which runs np.dot(a,b).the complexity of these two arithmetics are so different!
numpy certainly exploits vectorized by SIMD and BLAS,which can be found in its source code.the official numpy distribution supports a set of parallel manipulation(like np.dot),but not every functions(like np.where,np.mean).the blog author may choose an inappropriate function（a unvectorized function） to compare.
we can also see that in respect of multi-cores CPU usage.when executing numpy.dot(),all the cores are performing a high usage.Hence numpy must have vectorized(by BLAS) to avoid just using a single core because of the CPython’s GIL restriction.