Creating sublists
Question:
The opposite of list flattening.
Given a list and a length n return a list of sub lists of length n.
def sublist(lst, n):
sub=[] ; result=[]
for i in lst:
sub+=[i]
if len(sub)==n: result+=[sub] ; sub=[]
if sub: result+=[sub]
return result
An example:
If the list is:
[1,2,3,4,5,6,7,8]
And n is:
3
Return:
[[1, 2, 3], [4, 5, 6], [7, 8]]
Is there a more eloquent / concise way?
An aside, what is preferred when appending lists to lists (in the context above):
list1+=[list2]
Or:
list1.append(list2)
Given that (according to Summerfeild’s ‘Programming in Python 3’) they are the same?
Thanks.
Answers:
I think this split function does what you’re looking for (though it works with any iterator rather than just lists):
from itertools import islice
def take(n, it):
"Return first n items of the iterable as a list"
return list(islice(it, n))
def split(it, size):
it = iter(it)
size = int(size)
ret = take(size, it)
while ret:
yield ret
ret = take(size, it)
Edit: Regarding your asside, I always use list.append(blah), as it feels more idiomatic to me, but I believe they are functionally equivalent.
Such a list of lists could be constructed using a list comprehension:
In [17]: seq=[1,2,3,4,5,6,7,8]
In [18]: [seq[i:i+3] for i in range(0,len(seq),3)]
Out[18]: [[1, 2, 3], [4, 5, 6], [7, 8]]
There is also the grouper idiom:
In [19]: import itertools
In [20]: list(itertools.izip_longest(*[iter(seq)]*3))
Out[20]: [(1, 2, 3), (4, 5, 6), (7, 8, None)]
but note that missing elements are filled with the value None. izip_longest can take a fillvalue parameter as well if something other than None is desired.
list1+=[list2] — noting the brackets this time — is equivalent to list1.append(list2). My highest priority when writing code is readability,
not speed. For this reason, I would go with list1.append(list2). Readability is subjective, however, and probably is influenced greatly by what idioms you’re familiar with.
Happily, in this case, readability and speed seem to coincide:
In [41]: %timeit list1=[1,2,3]; list1.append(list2)
1000000 loops, best of 3: 612 ns per loop
In [42]: %timeit list1=[1,2,3]; list1+=[list2]
1000000 loops, best of 3: 847 ns per loop
How about the following (where x is your list):
[x[i:i+3] for i in range(0, len(x), 3)]
This is trivial to generalize for n!=3.
As to your second question, they’re equivalent so I think it’s a matter of style. However, do make sure you’re not confusing append with extend.
This function can take any kind of iterable (not only sequences of known length):
import itertools
def grouper(n, it):
"grouper(3, 'ABCDEFG') --> ABC DEF G"
it = iter(it)
return iter(lambda: list(itertools.islice(it, n)), [])
print(list(grouper(3, [1,2,3,4,5,6,7,8,9,10])))
# [[1, 2, 3], [4, 5, 6], [7, 8, 9], [10]]
I know, it looks like a brainfuck, but is works:
>>> a = [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]
>>> n = 3
>>> [i for j in [[a[t:t+n] for x in a[:1:t+1] if (t%n)==False] for t in range(len(a))] for i in j]
[[1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11, 12], [13, 14, 15]]
>>> n = 4
>>> [i for j in [[a[t:t+n] for x in a[:1:t+1] if (t%n)==False] for t in range(len(a))] for i in j]
[[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15]]
For some specific cases, it might be useful to use the numpy package. In this package you have a reshape routine:
import numpy as np
x = np.array([1,2,3,4,5,6])
np.reshape(x, (-1,3))
However, this solution won’t pad your list, if it’s not a multiply of n.
Have you heard of boltons?
Boltons is a set of pure-Python utilities in the same spirit as — and yet conspicuously missing from — the the standard library
It has what you want, built-in, called chunked
from boltons import iterutils
iterutils.chunked([1,2,3,4,5,6,7,8], 3)
Output:
[[1, 2, 3], [4, 5, 6], [7, 8]]
And whats more attractive in boltons is that it has chunked as an iterator, called chunked_iter, so you don’t need to store the whole thing in memory. Neat, right?
The opposite of list flattening.
Given a list and a length n return a list of sub lists of length n.
def sublist(lst, n):
sub=[] ; result=[]
for i in lst:
sub+=[i]
if len(sub)==n: result+=[sub] ; sub=[]
if sub: result+=[sub]
return result
An example:
If the list is:
[1,2,3,4,5,6,7,8]
And n is:
3
Return:
[[1, 2, 3], [4, 5, 6], [7, 8]]
Is there a more eloquent / concise way?
An aside, what is preferred when appending lists to lists (in the context above):
list1+=[list2]
Or:
list1.append(list2)
Given that (according to Summerfeild’s ‘Programming in Python 3’) they are the same?
Thanks.
I think this split function does what you’re looking for (though it works with any iterator rather than just lists):
from itertools import islice
def take(n, it):
"Return first n items of the iterable as a list"
return list(islice(it, n))
def split(it, size):
it = iter(it)
size = int(size)
ret = take(size, it)
while ret:
yield ret
ret = take(size, it)
Edit: Regarding your asside, I always use list.append(blah), as it feels more idiomatic to me, but I believe they are functionally equivalent.
Such a list of lists could be constructed using a list comprehension:
In [17]: seq=[1,2,3,4,5,6,7,8]
In [18]: [seq[i:i+3] for i in range(0,len(seq),3)]
Out[18]: [[1, 2, 3], [4, 5, 6], [7, 8]]
There is also the grouper idiom:
In [19]: import itertools
In [20]: list(itertools.izip_longest(*[iter(seq)]*3))
Out[20]: [(1, 2, 3), (4, 5, 6), (7, 8, None)]
but note that missing elements are filled with the value None. izip_longest can take a fillvalue parameter as well if something other than None is desired.
list1+=[list2] — noting the brackets this time — is equivalent to list1.append(list2). My highest priority when writing code is readability,
not speed. For this reason, I would go with list1.append(list2). Readability is subjective, however, and probably is influenced greatly by what idioms you’re familiar with.
Happily, in this case, readability and speed seem to coincide:
In [41]: %timeit list1=[1,2,3]; list1.append(list2)
1000000 loops, best of 3: 612 ns per loop
In [42]: %timeit list1=[1,2,3]; list1+=[list2]
1000000 loops, best of 3: 847 ns per loop
How about the following (where x is your list):
[x[i:i+3] for i in range(0, len(x), 3)]
This is trivial to generalize for n!=3.
As to your second question, they’re equivalent so I think it’s a matter of style. However, do make sure you’re not confusing append with extend.
This function can take any kind of iterable (not only sequences of known length):
import itertools
def grouper(n, it):
"grouper(3, 'ABCDEFG') --> ABC DEF G"
it = iter(it)
return iter(lambda: list(itertools.islice(it, n)), [])
print(list(grouper(3, [1,2,3,4,5,6,7,8,9,10])))
# [[1, 2, 3], [4, 5, 6], [7, 8, 9], [10]]
I know, it looks like a brainfuck, but is works:
>>> a = [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]
>>> n = 3
>>> [i for j in [[a[t:t+n] for x in a[:1:t+1] if (t%n)==False] for t in range(len(a))] for i in j]
[[1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11, 12], [13, 14, 15]]
>>> n = 4
>>> [i for j in [[a[t:t+n] for x in a[:1:t+1] if (t%n)==False] for t in range(len(a))] for i in j]
[[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15]]
For some specific cases, it might be useful to use the numpy package. In this package you have a reshape routine:
import numpy as np
x = np.array([1,2,3,4,5,6])
np.reshape(x, (-1,3))
However, this solution won’t pad your list, if it’s not a multiply of n.
Have you heard of boltons?
Boltonsis a set of pure-Python utilities in the same spirit as — and yet conspicuously missing from — the the standard library
It has what you want, built-in, called chunked
from boltons import iterutils
iterutils.chunked([1,2,3,4,5,6,7,8], 3)
Output:
[[1, 2, 3], [4, 5, 6], [7, 8]]
And whats more attractive in boltons is that it has chunked as an iterator, called chunked_iter, so you don’t need to store the whole thing in memory. Neat, right?