Does next() eliminate values from a generator?

Question

I’ve written a generator that does nothing more or less than store a range from 0 to 10:

result = (num for num in range(11))

When I want to print values, I can use next():

print(next(result))
[Out]: 0
print(next(result))
[Out]: 1
print(next(result))
[Out]: 2
print(next(result))
[Out]: 3
print(next(result))
[Out]: 4

If I then run a for loop on the generator, it runs on the values that I have not called next() on:

for value in result:
    print(value)
[Out]: 5
6
7
8
9
10

Has the generator eliminated the other values by acting on them with a next() function? I’ve tried to find some documentation on the functionality of next() and generators but haven’t been successful.

Asked By: Yehuda

||

Source

Answer 1

Actually this is can be implicitly deduced from next‘s docs and by understanding the iterator protocol/contract:

next(iterator[, default])
Retrieve the next item from the iterator by
calling its next() method. If default is given, it is returned if
the iterator is exhausted, otherwise StopIteration is raised.

Yes. Using a generator’s __next__ method retrieves and removes the next value from the generator.

Answered By: DeepSpace

Answer 2

tldr; yes

An iterator is essentially a value producer that yields successive values from its associated iterable object. The built-in function next() is used to obtain the next value from in iterator.
Here is an example using the same list as above:

>>> l = ['Sarah', 'Roark']
>>> itr = iter(l)
>>> itr
<list_iterator object at 0x100ba8950>
>>> next(itr)
'Sarah'
>>> next(itr)
'Roark'
>>> next(itr)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

In this example, l is an iterable list and itr is the associated iterator, obtained with iter(). Each next(itr) call obtains the next value from itr.

Notice how an iterator retains its state internally. It knows which values have been obtained already, so when you call next(), it knows what value to return next.

If all the values from an iterator have been returned already, a subsequent next() call raises a StopIteration exception. Any further attempts to obtain values from the iterator will fail.

We can only obtain values from an iterator in one direction. We can’t go backward. There is no prev() function. But we can define two independent iterators on the same iterable object:

>>> l
['Sarah', 'Roark', 30]
>>> itr1 = iter(l)
>>> itr2 = iter(l)
>>> next(itr1)
'Sarah'
>>> next(itr1)
'Roark'
>>> next(itr1)
30
>>> next(itr2)
'Sarah'

Answered By: Rachit Tayal

Answer 3

Yes, a for loop in Python just returns the next item from the iterator, the same way that next() does.

https://docs.python.org/3/reference/compound_stmts.html#the-for-statement

The suite is then executed once for each item provided by the iterator, in the order returned by the iterator.

So you can think of a for loop like this:

for x in container:
    statement()

As (almost) equivalent to a while loop:

iterator = iter(container)
while True:
    x = next(iterator)
    if x is None:
        break
    statement()

If container is already an iterator, then iter(container) is container.

Note: Technically, a for loop is more like this:

iterator = iter(container)
while True:
    try:
        x = iterator.__next__()
    except StopIteration:
        break
    statement()

Answered By: Dietrich Epp

Does next() eliminate values from a generator?

Question:

Answers: