How to loop through a generator

Question

How can one loop through a generator? I thought about this way:

gen = function_that_returns_a_generator(param1, param2)
if gen: # in case the generator is null
    while True:
        try:
            print gen.next()
        except StopIteration:
            break

Is there a more pythonic way?

Asked By: iTayb

||

Source

Answer 1

for item in function_that_returns_a_generator(param1, param2):
    print item

You don’t need to worry about the test to see if there is anything being returned by your function as if there’s nothing returned you won’t enter the loop.

Answered By: Christian Witts

Answer 2

Simply

for x in gen:
    # whatever

will do the trick. Note that if gen always returns True.

Answered By: Sven Marnach

Answer 3

Just treat it like any other iterable:

for val in function_that_returns_a_generator(p1, p2):
    print val

Note that if gen: will always be True, so it’s a false test

Answered By: Jon Clements

Answer 4

If you want to manually move through the generator (i.e., to work with each loop manually) then you could do something like this:

    from pdb import set_trace

    for x in gen:
        set_trace()
        #do whatever you want with x at the command prompt
        #use pdb commands to step through each loop of the generator e.g., >>c #continue

Answered By: rysqui

Answer 5

In case you don’t need the output of the generator because you care only about its side effects, you can use the following one-liner:

for _ in gen: pass

Follow up

Following the comment by aiven I made some performance tests, and while it seems that list(gen) is slightly faster than for _ in gen: pass, it comes out that tuple(gen) is even faster. However, as Erik Aronesty correctly points out, tuple(gen) and list(gen) store the results, so my final advice is to use

tuple(gen)

but only if the generator is not going to loop billions of times soaking up too much memory.

Answered By: mmj

Answer 6

You can simply loop through it:

>>> gen = (i for i in range(1, 4))
>>> for i in gen: print i
1
2
3

But be aware, that you can only loop one time. Next time generator will be empty:

>>> for i in gen: print i
>>>

Answered By: Wojciech Kałuski

Answer 7

The other answers are good for complicated scenarios. If you simply want to stream the items into a list:

x = list(generator)

For simple preprocessing, use list comprehensions:

x = [tup[0] for tup in generator]

If you just want to execute the generator without saving the results, you can skip variable assignment:

# no var assignment b/c we don't need what print() returns
[print(_) for _ in gen]

Don’t do this if your generator is infinite (say, streaming items from the internet). The list construction is a blocking op that won’t stop until the generator is empty.

Answered By: crypdick

How to loop through a generator

Question:

Answers:

Follow up