Why can a python generator only be used once?
Question:
Once I use a generator once, it can’t be used again. Why is this?
Consider the following code:
def generator(n):
a = 1
for _ in range(n):
yield a
a += 1
def run_generator(generator):
for a in generator:
print(a, end = " ")
If I were to execute this:
count_generator = generator(10)
run_generator(count_generator)
run_generator(count_generator)
It would only print:
1 2 3 4 5 6 7 8 9 10
Basically, the generator just dies after a single execution.
I know that the fact that a generator can only be used once is something built into Python, but why is it like this? Is there a specific reason to only allowing a generator object to be executed once?
Answers:
A generator works like ticker tape. When you call next on it, it gives you the next number, but then it forgets it, unlike a list. This is where most of the efficiency comes from. Since it doesn’t have to remember what its previous values were, there’s a much smaller memory footprint (especially when not all of its values will eventually be needed!)
Some generators can perhaps be reset to be able to run again, but that’s by no means guaranteed, and some generators will outright fail if you try to do that. Python is not a pure language, and so you might have generators that modify state while they produce values. For instance, a generator such as:
def gimme_randoms():
while True:
yield random.random()
I can call this a bunch, but the state behind the PRNG in random will change every time I do.
rs = gimme_randoms()
a = next(rs)
b = next(rs)
c = next(rs) # some numbers
What would it mean to reset this state? Well you’d expect:
rs2 = gimme_randoms()
x = next(rs2)
y = next(rs2)
z = next(rs2)
assert a == x and b == y and c == z # Nonsense!
To make this hold, well, you’d have to keep track of the initial state of the PRNG, then have a way to set its seed back to the initial state. That’s certainly doable, but it’s not the generator’s job to know how its underlying implementation has modified state.
Once I use a generator once, it can’t be used again. Why is this?
Consider the following code:
def generator(n):
a = 1
for _ in range(n):
yield a
a += 1
def run_generator(generator):
for a in generator:
print(a, end = " ")
If I were to execute this:
count_generator = generator(10)
run_generator(count_generator)
run_generator(count_generator)
It would only print:
1 2 3 4 5 6 7 8 9 10
Basically, the generator just dies after a single execution.
I know that the fact that a generator can only be used once is something built into Python, but why is it like this? Is there a specific reason to only allowing a generator object to be executed once?
A generator works like ticker tape. When you call next on it, it gives you the next number, but then it forgets it, unlike a list. This is where most of the efficiency comes from. Since it doesn’t have to remember what its previous values were, there’s a much smaller memory footprint (especially when not all of its values will eventually be needed!)
Some generators can perhaps be reset to be able to run again, but that’s by no means guaranteed, and some generators will outright fail if you try to do that. Python is not a pure language, and so you might have generators that modify state while they produce values. For instance, a generator such as:
def gimme_randoms():
while True:
yield random.random()
I can call this a bunch, but the state behind the PRNG in random will change every time I do.
rs = gimme_randoms()
a = next(rs)
b = next(rs)
c = next(rs) # some numbers
What would it mean to reset this state? Well you’d expect:
rs2 = gimme_randoms()
x = next(rs2)
y = next(rs2)
z = next(rs2)
assert a == x and b == y and c == z # Nonsense!
To make this hold, well, you’d have to keep track of the initial state of the PRNG, then have a way to set its seed back to the initial state. That’s certainly doable, but it’s not the generator’s job to know how its underlying implementation has modified state.