Why does a list of generators only return the elements of the last generator?

Question

I’m given an arbitrary list of objects (e.g. ['foo', 'bar']). My goal is to produce a list of equal size where every element in the result list is a generator that repeats the respective input element 5 times.

This is a big simplification of what I actually want to do, and I am aware that there are many ways in which to solve this task.
However, I have stumbled upon some weird behavior in how I wanted to solve this that I can not explain.

Here is my solution to the above task:

my_iterators = [
    (element for _ in range(5))
    for element in ["foo", "bar"]
]
for my_iterator in my_iterators:
    print(list(my_iterator))

I now expected the output to be:

['foo', 'foo', 'foo', 'foo', 'foo']
['bar', 'bar', 'bar', 'bar', 'bar']

However, to my surprise it was instead:

['bar', 'bar', 'bar', 'bar', 'bar']
['bar', 'bar', 'bar', 'bar', 'bar']

Why is it that (element for _ in range(5)) seems to be an iterator over the last element in the input_list irregarding of what it actually is in the context of for element in ["foo", "bar"]?
How would I need to adapt my code to produce what my original goal was?

Asked By: Lukas Schmelzeisen

||

Source

Answer 1

As pointed out by jonsharpe and David Buck in the comments, what’s actually causing the non-intuitive behavior here is Python’s late binding.

There seem to be multiple ways of fixing the posted code, one would be:

def make_iterator(element):
    return (element for _ in range(5))

my_iterators = [
    make_iterator(element)
    for element in ["foo", "bar"]
]
for my_iterator in my_iterators:
    print(list(my_iterator))

Answered By: Lukas Schmelzeisen

Why does a list of generators only return the elements of the last generator?

Question:

Answers: