Is there an elegant way to cycle through a list N times via iteration (like itertools.cycle but limit the cycles)?
Question:
I’d like to cycle through a list repeatedly (N times) via an iterator, so as not to actually store N copies of the list in memory. Is there a built-in or elegant way to do this without writing my own generator?
Ideally, itertools.cycle(my_list) would have a second argument to limit how many times it cycles… alas, no such luck.
Answers:
itertools.chain.from_iterable(iter(L) for x in range(N))
import itertools
it = itertools.chain.from_iterable(itertools.repeat([1, 2, 3], 5))
print(list(it))
# [1, 2, 3, 1, 2, 3, 1, 2, 3, 1, 2, 3, 1, 2, 3]
Itertools is a wonderful library. 🙂
As pointed out in the comments, if you are using iterators (instead of lists or similar collections), iterators are exhausted after the first iteration, and so you have to use a slightly different approach to get the expected result. Note that infinite or excessively long iterators can’t be reasonably handled, as you would have to cache the result (requiring infinite or excessive amounts of memory).
This may not be the most efficient implementation (tee stores N copies of the iterable’s contents):
import itertools
it = itertools.chain(*itertools.tee(iter([1, 2, 3]), 5))
print(list(it))
# [1, 2, 3, 1, 2, 3, 1, 2, 3, 1, 2, 3, 1, 2, 3]
For the special case where you need to iterate over a list several times, this is not too bad.
It does create a list of n references to my_list, so if n is very large it is better to use Darthfelt’s answer
>>> import itertools as it
>>> it.chain(*[my_list]*n)
All the other answers are excellent. Another solution would be to use islice. This allows you to interrupt the cycle at any point:
>>> from itertools import islice, cycle
>>> l = [1, 2, 3]
>>> list(islice(cycle(l), len(l) * 3))
[1, 2, 3, 1, 2, 3, 1, 2, 3]
>>> list(islice(cycle(l), 7))
[1, 2, 3, 1, 2, 3, 1]
You said that you don’t want to write your own generator, but a generator expression would probably be the easiest and most efficient way to accomplish what you’re after. It doesn’t require any function calls or importing of any modules. itertools is a great module, but maybe not necessary in this case?
some_list = [1, 2, 3]
cycles = 3
gen_expr = (elem for _ in xrange(cycles) for elem in some_list)
or just
(elem for _ in xrange(3) for elem in [1, 2, 3])
or
for elem in (e for _ in xrange(3) for e in [1, 2, 3]):
print "hoo-ray, {}!".format(elem)
@Darthfett’s answer is documented as an itertools recipes:
from itertools import chain, repeat
def ncycles(iterable, n):
"Returns the sequence elements n times"
return chain.from_iterable(repeat(tuple(iterable), n))
list(ncycles(["a", "b"], 3))
# ['a', 'b', 'a', 'b', 'a', 'b']
For convenience, I add that the more_itertools library implements this recipe (and many others) for you:
import more_itertools as mit
list(mit.ncycles(["a", "b"], 3))
# ['a', 'b', 'a', 'b', 'a', 'b']
I’d like to cycle through a list repeatedly (N times) via an iterator, so as not to actually store N copies of the list in memory. Is there a built-in or elegant way to do this without writing my own generator?
Ideally, itertools.cycle(my_list) would have a second argument to limit how many times it cycles… alas, no such luck.
itertools.chain.from_iterable(iter(L) for x in range(N))
import itertools
it = itertools.chain.from_iterable(itertools.repeat([1, 2, 3], 5))
print(list(it))
# [1, 2, 3, 1, 2, 3, 1, 2, 3, 1, 2, 3, 1, 2, 3]
Itertools is a wonderful library. 🙂
As pointed out in the comments, if you are using iterators (instead of lists or similar collections), iterators are exhausted after the first iteration, and so you have to use a slightly different approach to get the expected result. Note that infinite or excessively long iterators can’t be reasonably handled, as you would have to cache the result (requiring infinite or excessive amounts of memory).
This may not be the most efficient implementation (tee stores N copies of the iterable’s contents):
import itertools
it = itertools.chain(*itertools.tee(iter([1, 2, 3]), 5))
print(list(it))
# [1, 2, 3, 1, 2, 3, 1, 2, 3, 1, 2, 3, 1, 2, 3]
For the special case where you need to iterate over a list several times, this is not too bad.
It does create a list of n references to my_list, so if n is very large it is better to use Darthfelt’s answer
>>> import itertools as it
>>> it.chain(*[my_list]*n)
All the other answers are excellent. Another solution would be to use islice. This allows you to interrupt the cycle at any point:
>>> from itertools import islice, cycle
>>> l = [1, 2, 3]
>>> list(islice(cycle(l), len(l) * 3))
[1, 2, 3, 1, 2, 3, 1, 2, 3]
>>> list(islice(cycle(l), 7))
[1, 2, 3, 1, 2, 3, 1]
You said that you don’t want to write your own generator, but a generator expression would probably be the easiest and most efficient way to accomplish what you’re after. It doesn’t require any function calls or importing of any modules. itertools is a great module, but maybe not necessary in this case?
some_list = [1, 2, 3]
cycles = 3
gen_expr = (elem for _ in xrange(cycles) for elem in some_list)
or just
(elem for _ in xrange(3) for elem in [1, 2, 3])
or
for elem in (e for _ in xrange(3) for e in [1, 2, 3]):
print "hoo-ray, {}!".format(elem)
@Darthfett’s answer is documented as an itertools recipes:
from itertools import chain, repeat
def ncycles(iterable, n):
"Returns the sequence elements n times"
return chain.from_iterable(repeat(tuple(iterable), n))
list(ncycles(["a", "b"], 3))
# ['a', 'b', 'a', 'b', 'a', 'b']
For convenience, I add that the more_itertools library implements this recipe (and many others) for you:
import more_itertools as mit
list(mit.ncycles(["a", "b"], 3))
# ['a', 'b', 'a', 'b', 'a', 'b']