Can I somehow speed up this simple code in python? That takes 0.221seconds for executing
for some_list in some_list_of_lists: i = 1 while i < len(some_list): a = some_list[i-1] b = some_list[i] i += 2
Same code in cython takes only 0.004seconds, my goal is achieve 0.02seconds or less in python(without using cython)
result =  for some_list in some_list_of_lists: t1 = 1 t2 = 1 i = 1 while i < len(some_list): a = some_list[i-1] #int b = some_list[i] #int t1 *= a t2 *= b i += 2 if t1 > t2: result.append(some_list)
Threading and multiprocessing are both modules used to speed up functionality and get more done in less time. I’m not sure how much improvement you’ll see in this example but these modules are great to know about in general.
The main problem here is that your implementation is very inefficient.
from timeit import timeit from random import random import numpy as np # for a numpy solution from math import prod # for a better Python solution # just some random example data that is "some list of lists" (of numbers) example_data = [ [random() * 10 for _ in range(100)] for _ in range(100) ] def f(some_list_of_lists): # your code, in a function, so it's easy to time result =  for some_list in some_list_of_lists: t1 = 1 t2 = 1 i = 1 while i < len(some_list): a = some_list[i - 1] # int b = some_list[i] # int t1 *= a t2 *= b i += 2 if t1 > t2: result.append(some_list) return result # the same data again, but in a numpy array example_array = np.array(example_data) def numpy_f(some_array): # this function does exactly the same, it selects those 'lists' for which # the product of the even elements is greater than the product of the odd ones return some_array[ np.prod(some_array[:,::2], axis=1) > np.prod(some_array[:,1::2], axis=1) ] def better_python_f(some_list_of_lists): return [ xs for xs in some_list_of_lists if prod(xs[::2]) > prod(xs[1::2]) ] # showing that the results are the same simple_example = [[2, 1, 1, 1], [1, 2, 1, 1], [2, 1, .5, 1], [.5, 1, 1, 1], [1, .5, 1, 1]] print(f(simple_example)) print(numpy_f(np.array(simple_example))) print(better_python_f(simple_example)) # timing running each 10,000 times on the large example data set print('yours:', timeit(lambda: f(example_data), number=10000)) print('numpy:', timeit(lambda: numpy_f(example_array), number=10000)) print('mine :', timeit(lambda: better_python_f(example_data), number=10000))
[[2, 1, 1, 1], [1, 0.5, 1, 1]] [[2. 1. 1. 1. ] [1. 0.5 1. 1. ]] [[2, 1, 1, 1], [1, 0.5, 1, 1]] yours: 5.2597994999996445 numpy: 0.1987909999988915 mine : 0.7029579999998532
This shows that using just plain Python, without external libraries, you can easily get almost an order of magnitude faster. I’m sure someone can come up with an even faster method just in Python.
numpy is perfect for this, and even the trivial solution I wrote is more than 25x faster than yours – and again I think someone more experienced in the application of
numpy can probably write an even faster solution using
The key takeaway here shouldn’t be "I need to use numpy because it’s fast" though. The key thing is that you should think about what problem you’re actually solving and then decide what the most efficient way is to do exactly that.
In your case, what the code actually does is "for a series of number series, decide if the product of numbers on even indices is greater than the product of numbers on odd incides for each number series, and keep only those where this is the case". To me, that quickly points at thinking about slicing, efficient ways to compute a product, and how to select and return the desired result.
Not only does the
better_python_f do exactly what it needs to, the code almost reads like a description of the problem:
[ xs for xs in some_list_of_lists if prod(xs[::2]) > prod(xs[1::2]) ]
"A list of all
xs in the original list, where the product of even elements is greater than the product of odd elements."
Of course, it does require that you know about the Python concepts ‘list comprehension’ (
x for x in .. if <condition on x>) and ‘slicing’ (e.g.
xs[::2]), and that
math has a fairly efficient
prod() function to compute the product of elements of an iterable like a list.