failed to convert a for loop into loop comprehension expression. why is my list comprehension slower than my for loop?

Question

While building a project that uses sliding window, I have the following code, which works fine:

#aby is a one dimension list like, eg:[0.010,0.012,0.008,......,0.009]
adj_qth_amp = 100
scan_window_size = 10000
wanted_time_stamps = []
window_sum = sum(aby[:scan_window_size])
for i in range(len(aby)-scan_window_size):
    window_sum = window_sum - aby[i] + aby[scan_window_size+i]
    if window_sum > adj_qth_amp:
        wanted_time_stamps.append(i)

#runtime is 4 minutes
#returns a wanted_time_stamps list that has valid elements after execution

then I remember people have a cool way of building a list like:

my_list = [i for i in range(10)]              #basic list
my_list_2 = [i+1 for i in my_list]            #calculation
my_list_3 = [i+1 for i in my_list_2 if i>3]   #adding a condition

hence I wrote the following for practice:

wanted_time_stamps = [(window_sum - aby[i] + aby[scan_window_size+i]) for i in range(len(aby)-scan_window_size) if (window_sum - aby[i] + aby[scan_window_size+i]) > adj_qth_amp]

#runtime is 8 minutes
#returns a wanted_time_stamps that is empty (eg.[]) after execution

I understand that this can be too long of a blob to make this a good way of making the list. but it was for practice.

Question:

what did I do wrong for the second method to return an empty list?
why is the runtime for the second method aproximately twice as long? isn’t list comprehension suppose to be faster?

3.what is this way of declaring a list called? I am having a hard time searching about it, and thus naming my question here. edit: it is called list comprehension.

Asked By: bentrification

||

Source

Answer 1

You update window_sum each time in the loop of the first way, but you do not do so in the second way.
In the second way, you calculate (window_sum - aby[i] + aby[scan_window_size+i]) twice each time in the loop.
list comprehension

The correct way by using list comprehension:

[i for i in range(len(aby) - scan_window_size) if (window_sum := window_sum - aby[i] + aby[scan_window_size+i]) > adj_qth_amp]

Answered By: Mechanic Pig

Answer 2

Your code with some sample data:

from math import sin

aby = [sin(i/900) for i in range(11000)]

adj_qth_amp = 100
scan_window_size = 10000
wanted_time_stamps = []
window_sum = sum(aby[:scan_window_size])
for i in range(len(aby)-scan_window_size):
    window_sum = window_sum - aby[i] + aby[scan_window_size+i]
    if window_sum > adj_qth_amp:
        wanted_time_stamps.append(i)

print(len(wanted_time_stamps))

window_sum = sum(aby[:scan_window_size])  # reset this variable to its initial value
wanted_time_stamps_comprehension = [
    i
    for i in range(len(aby)-scan_window_size)
    if (window_sum := window_sum - aby[i] + aby[scan_window_size+i]) > adj_qth_amp
]

print(wanted_time_stamps_comprehension == wanted_time_stamps)

Note that you update window_sum in every iteration – a way to do that in the list comprehension would be to update it with a walrus operator as shown.

Before the list comprehension, window_sum is reset to its initial value. The resulting list is identical – if it’s actually the list you want though, I’m not sure. But if your original code was correct, so is the comprehension.

Output:

580
True

i.e. for 580 out of a possible 1,000 positions, the condition is true – and the two lists are identical.

Note that I named the new variable wanted_time_stamps_comprehension, but of course by the time the statement completes, it’s resolved to a list just like the other.

Answered By: Grismar

failed to convert a for loop into loop comprehension expression. why is my list comprehension slower than my for loop?

Question:

Answers: