Alternative way to split a list into groups of n
Question:
Let’s say I have a list of arbitrary length, L:
L = list(range(1000))
What is the best way to split that list into groups of n? This is the best structure that I have been able to come up with, and for some reason it does not feel like it is the best way of accomplishing the task:
n = 25
for i in range(0, len(L), n):
chunk = L[i:i+25]
Is there a built-in to do this I’m missing?
Edit: Early answers are reworking my for loop into a listcomp, which is not the idea; you’re basically giving me my exact answer back in a different form. I’m seeing if there’s an alternate means to accomplish this, like a hypothetical .split on lists or something. I also do use this as a generator in some code that I wrote last night:
def split_list(L, n):
assert type(L) is list, "L is not a list"
for i in range(0, len(L), n):
yield L[i:i+n]
Answers:
How about:
>>> n = 2
>>> l = [1,2,3,4,5,6,7,8,9]
>>> [l[i:i+n] for i in range(0, len(l), n)]
[[1, 2], [3, 4], [5, 6], [7, 8], [9]]
n = 25
list_of_lists = [L[i:i+n] for i in range(0, len(L), n)]
it gives you the list of lists [[0..24], [25..49], ..]
If len(L) % n isn’t 0, the last element’s (list_of_lists[-1]) lenght will be len(L) % n.
A Python recipe (In Python 2.6, use itertools.izip_longest):
def grouper(n, iterable, fillvalue=None):
"grouper(3, 'ABCDEFG', 'x') --> ABC DEF Gxx"
args = [iter(iterable)] * n
return itertools.zip_longest(*args, fillvalue=fillvalue)
Example usage:
>>> list(grouper(3, range(9)))
[(0, 1, 2), (3, 4, 5), (6, 7, 8)]
>>> list(grouper(3, range(10)))
[(0, 1, 2), (3, 4, 5), (6, 7, 8), (9, None, None)]
If you want the last group to be shorter than the others instead of padded with fillvalue, then you could e.g. change the code like this:
>>> def mygrouper(n, iterable):
... args = [iter(iterable)] * n
... return ([e for e in t if e != None] for t in itertools.zip_longest(*args))
...
>>> list(mygrouper(3, range(9)))
[[0, 1, 2], [3, 4, 5], [6, 7, 8]]
>>> list(mygrouper(3, range(10)))
[[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]
Here you go:
list_of_groups = zip(*(iter(the_list),) * group_size)
Example:
print zip(*(iter(range(10)),) * 3)
[(0, 1, 2), (3, 4, 5), (6, 7, 8)]
If the number of elements is not divisible by N but you still want to include them you can use izip_longest but it is only available since python 2.6
izip_longest(*(iter(range(10)),) * 3)
The result is a generator so you need to convert it into a list if you want to print it.
Finally, if you don’t have python 2.6 and stuck with an older version but you still want to have the same result you can use map:
print map(None, *(iter(range(10)),) * 3)
[(0, 1, 2), (3, 4, 5), (6, 7, 8), (9, None, None)]
I’d like to add some speed comparison between the different methods presented so far:
python -m timeit -s 'from itertools import izip_longest; L = range(1000)' 'list(izip_longest(*(iter(L),) * 3))'
10000 loops, best of 3: 47.1 usec per loop
python -m timeit -s 'L = range(1000)' 'zip(*(iter(L),) * 3)'
10000 loops, best of 3: 50.1 usec per loop
python -m timeit -s 'L = range(1000)' 'map(None, *(iter(L),) * 3)'
10000 loops, best of 3: 50.7 usec per loop
python -m timeit -s 'L = range(1000)' '[L[i:i+3] for i in range(0, len(L), 3)]'
10000 loops, best of 3: 157 usec per loop
python -m timeit -s 'import itertools; L = range(1000)' '[list(group) for key, group in itertools.groupby(L, lambda k: k//3)]'
1000 loops, best of 3: 1.41 msec per loop
The list comprehension and the group by methods are clearly slower than zip, izip_longest and map
Itertools.groupby is a fine tool, here is a way to split a list of integers simply by using integer division:
>>> for key, group in itertools.groupby(range(10), lambda k: k//3):
... print key, list(group)
...
0 [0, 1, 2]
1 [3, 4, 5]
2 [6, 7, 8]
3 [9]
(The list has to start with 0 to begin with a full group.)
Here is recursion version. It is inefficient because Python has recursion limits, but this version illustrates that every task can be solved through recursion.
def split_to_groups(l, n):
assert (len(l) / n) < 998, "Can't split to {} groups".format(len(l) / n)
if l == []:
return []
else:
f = [l[:n]]
f.extend(split_to_groups(l[n:], n))
return f
Let’s say I have a list of arbitrary length, L:
L = list(range(1000))
What is the best way to split that list into groups of n? This is the best structure that I have been able to come up with, and for some reason it does not feel like it is the best way of accomplishing the task:
n = 25
for i in range(0, len(L), n):
chunk = L[i:i+25]
Is there a built-in to do this I’m missing?
Edit: Early answers are reworking my for loop into a listcomp, which is not the idea; you’re basically giving me my exact answer back in a different form. I’m seeing if there’s an alternate means to accomplish this, like a hypothetical .split on lists or something. I also do use this as a generator in some code that I wrote last night:
def split_list(L, n):
assert type(L) is list, "L is not a list"
for i in range(0, len(L), n):
yield L[i:i+n]
How about:
>>> n = 2
>>> l = [1,2,3,4,5,6,7,8,9]
>>> [l[i:i+n] for i in range(0, len(l), n)]
[[1, 2], [3, 4], [5, 6], [7, 8], [9]]
n = 25
list_of_lists = [L[i:i+n] for i in range(0, len(L), n)]
it gives you the list of lists [[0..24], [25..49], ..]
If len(L) % n isn’t 0, the last element’s (list_of_lists[-1]) lenght will be len(L) % n.
A Python recipe (In Python 2.6, use itertools.izip_longest):
def grouper(n, iterable, fillvalue=None):
"grouper(3, 'ABCDEFG', 'x') --> ABC DEF Gxx"
args = [iter(iterable)] * n
return itertools.zip_longest(*args, fillvalue=fillvalue)
Example usage:
>>> list(grouper(3, range(9)))
[(0, 1, 2), (3, 4, 5), (6, 7, 8)]
>>> list(grouper(3, range(10)))
[(0, 1, 2), (3, 4, 5), (6, 7, 8), (9, None, None)]
If you want the last group to be shorter than the others instead of padded with fillvalue, then you could e.g. change the code like this:
>>> def mygrouper(n, iterable):
... args = [iter(iterable)] * n
... return ([e for e in t if e != None] for t in itertools.zip_longest(*args))
...
>>> list(mygrouper(3, range(9)))
[[0, 1, 2], [3, 4, 5], [6, 7, 8]]
>>> list(mygrouper(3, range(10)))
[[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]
Here you go:
list_of_groups = zip(*(iter(the_list),) * group_size)
Example:
print zip(*(iter(range(10)),) * 3)
[(0, 1, 2), (3, 4, 5), (6, 7, 8)]
If the number of elements is not divisible by N but you still want to include them you can use izip_longest but it is only available since python 2.6
izip_longest(*(iter(range(10)),) * 3)
The result is a generator so you need to convert it into a list if you want to print it.
Finally, if you don’t have python 2.6 and stuck with an older version but you still want to have the same result you can use map:
print map(None, *(iter(range(10)),) * 3)
[(0, 1, 2), (3, 4, 5), (6, 7, 8), (9, None, None)]
I’d like to add some speed comparison between the different methods presented so far:
python -m timeit -s 'from itertools import izip_longest; L = range(1000)' 'list(izip_longest(*(iter(L),) * 3))'
10000 loops, best of 3: 47.1 usec per loop
python -m timeit -s 'L = range(1000)' 'zip(*(iter(L),) * 3)'
10000 loops, best of 3: 50.1 usec per loop
python -m timeit -s 'L = range(1000)' 'map(None, *(iter(L),) * 3)'
10000 loops, best of 3: 50.7 usec per loop
python -m timeit -s 'L = range(1000)' '[L[i:i+3] for i in range(0, len(L), 3)]'
10000 loops, best of 3: 157 usec per loop
python -m timeit -s 'import itertools; L = range(1000)' '[list(group) for key, group in itertools.groupby(L, lambda k: k//3)]'
1000 loops, best of 3: 1.41 msec per loop
The list comprehension and the group by methods are clearly slower than zip, izip_longest and map
Itertools.groupby is a fine tool, here is a way to split a list of integers simply by using integer division:
>>> for key, group in itertools.groupby(range(10), lambda k: k//3):
... print key, list(group)
...
0 [0, 1, 2]
1 [3, 4, 5]
2 [6, 7, 8]
3 [9]
(The list has to start with 0 to begin with a full group.)
Here is recursion version. It is inefficient because Python has recursion limits, but this version illustrates that every task can be solved through recursion.
def split_to_groups(l, n):
assert (len(l) / n) < 998, "Can't split to {} groups".format(len(l) / n)
if l == []:
return []
else:
f = [l[:n]]
f.extend(split_to_groups(l[n:], n))
return f