Cython Speed Boost vs. Usability


I just came across Cython, while I was looking out for ways to optimize Python code. I read various posts on Stack Overflow, the python wiki and read the article "General Rules for Optimization".

Cython is something which grasps my interest the most; instead of writing C-code for yourself, you can choose to have other data types in your python code itself.

Here is a silly test I tried,

# test.pyx
def test(value):
    for i in xrange(value):
        print i


$ time python test.pyx

real    0m16.774s 
user    0m16.745s
sys     0m0.024s

$ time cython test.pyx

real    0m0.513s 
user    0m0.196s 
sys     0m0.052s

Now, honestly, I’m dumbfounded. The code which I have used here is pure python code, and all I have changed is the interpreter. In this case, if the cython is this good, then why do people still use the traditional Python interpreter? Are there any reliability issues for Cython?

Asked By: user277465



Cython is not another interpreter. It generates c-extensions for python, from python(-like) code. cython test.pyx will only generate a ‘test.c’ file, which (once compiled) can be used by python just like a normal python library.

That means that you are only measuring the time it takes for cython to translate your python code to c, not how fast that version of your code runs.

Answered By: Steven
  • cython test.pyx doesn’t actually run your program. The cython binary is for processing your Cython code into a Python extension module. You would have to import it in Python to run it.

  • #!/usr/bin/python isn’t the best shebang line for Python scripts. #!/usr/bin/env python is generally preferred, which runs whatever python would on the command line.

    • Cython pyx files probably shouldn’t have a shebang line at all, except in the corner case they are valid Python programs.
  • You have an IndentationError in the posted code.

  • Using the traditional interpreter is simpler and more portable. Cython is reliable, but has its limitations and quirks. It might be compelling to use it tons more if it magically gave the speedups your timings make it look like it does, but it actually gives smaller ones. You’ll have to start using Cython-specific features to use C features to see a lot of speedup.

Answered By: Mike Graham

The other answers have already explained how you were just compiling the Cython code, not executing it. However, I thought that you might want to know how much faster Cython can make your code. When I compiled the code you have (though I ran the function from from a different module) with distutils, I got very marginal speed gains over straight Python – about 1%. However, when I added a few small changes to your code:

def test(long long value):
    cdef long long i
    cdef long long z
    for i in xrange(value):
        z = i**2
            print i
        if z < i:
            print "yes"

and compiled it, I got the following times:

  • Pure Python code: 20.4553578737 seconds
  • Cython code: 0.199339860234 seconds

That’s a 100× speed-up. Not too shabby.

Answered By: Justin Peel

A big point that seems to be missing: Cython is not a strict superset of Python. There are some features that Python supports, but Cython does not. Most notably, generators and lambdas (but they are coming).

Answered By: carl
  • Biggest reason Cython is not that popular is that it lacks standalone (without python) executables.

  • Lack of publicity. The developers seem to be academics more interested in Developing their Sage software, than a cutting edge language.

  • Pitfalls encountered during development. One I encountered is lack of true thread support. Everything is wrapped up in a global interpreter lock, making it threadsafe, but disabling concurrency!

Answered By: unixman83

For anyone who wishes that cython actually compiled and ran your program in one line, I created runcython ( runcython test.pyx will have the semantics that the OP intended

Answered By: RussellStewart

As many of comments suggesting the time you have mentioned with cython is just time taken in converting .py file into .c file. That .c file will be big interms of memory. The advantage is that you can compile that .c file using C compiler also

I will give you one example :
if you have a python file we need to store as .pyx file. By running “cython xyz.pyx” it will create .c and .so files

Lets check in picture files created by cython command

The size also increases in my case it changed for Bytes to Kb

There is some specific method to compile the python code
you can refer this links:

another thing i noticed is no where they mention the memory problems in cython
When i try to implement in my embedded project.I am facing lot of memory problems.

You can try pypy Where there is no need to put extra efforts to compile and the time also reduced in terms of compilation time

Refer the picture:
compare time between python and pypy

Thank you..

Answered By: Kusalkumar
Categories: questions Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.