How can I use list comprehensions to process a nested list?
Question:
I have this nested list:
l = [['40', '20', '10', '30'], ['20', '20', '20', '20', '20', '30', '20'], ['30', '20', '30', '50', '10', '30', '20', '20', '20'], ['100', '100'], ['100', '100', '100', '100', '100'], ['100', '100', '100', '100']]
I want to convert each element in l to float. I have this code:
newList = []
for x in l:
for y in x:
newList.append(float(y))
How can I solve the problem with a nested list comprehension instead?
See also: How can I get a flat result from a list comprehension instead of a nested list?
Answers:
>>> l = [['40', '20', '10', '30'], ['20', '20', '20', '20', '20', '30', '20'], ['30', '20', '30', '50', '10', '30', '20', '20', '20'], ['100', '100'], ['100', '100', '100', '100', '100'], ['100', '100', '100', '100']]
>>> new_list = [float(x) for xs in l for x in xs]
>>> new_list
[40.0, 20.0, 10.0, 30.0, 20.0, 20.0, 20.0, 20.0, 20.0, 30.0, 20.0, 30.0, 20.0, 30.0, 50.0, 10.0, 30.0, 20.0, 20.0, 20.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0]
Here is how you would do this with a nested list comprehension:
[[float(y) for y in x] for x in l]
This would give you a list of lists, similar to what you started with except with floats instead of strings.
If you want one flat list, then you would use
[float(y) for x in l for y in x]
Note the loop order – for x in l comes first in this one.
Yes, you can do it with such a code:
l = [[float(y) for y in x] for x in l]
If you don’t like nested list comprehensions, you can make use of the map function as well,
>>> from pprint import pprint
>>> l = l = [['40', '20', '10', '30'], ['20', '20', '20', '20', '20', '30', '20'], ['30', '20', '30', '50', '10', '30', '20', '20', '20'], ['100', '100'], ['100', '100', '100', '100', '100'], ['100', '100', '100', '100']]
>>> pprint(l)
[['40', '20', '10', '30'],
['20', '20', '20', '20', '20', '30', '20'],
['30', '20', '30', '50', '10', '30', '20', '20', '20'],
['100', '100'],
['100', '100', '100', '100', '100'],
['100', '100', '100', '100']]
>>> float_l = [map(float, nested_list) for nested_list in l]
>>> pprint(float_l)
[[40.0, 20.0, 10.0, 30.0],
[20.0, 20.0, 20.0, 20.0, 20.0, 30.0, 20.0],
[30.0, 20.0, 30.0, 50.0, 10.0, 30.0, 20.0, 20.0, 20.0],
[100.0, 100.0],
[100.0, 100.0, 100.0, 100.0, 100.0],
[100.0, 100.0, 100.0, 100.0]]
The best way to do this in my opinion is to use python’s itertools package.
>>>import itertools
>>>l1 = [1,2,3]
>>>l2 = [10,20,30]
>>>[l*2 for l in itertools.chain(*[l1,l2])]
[2, 4, 6, 20, 40, 60]
Not sure what your desired output is, but if you’re using list comprehension, the order follows the order of nested loops, which you have backwards. So I got the what I think you want with:
[float(y) for x in l for y in x]
The principle is: use the same order you’d use in writing it out as nested for loops.
This Problem can be solved without using for loop.Single line code will be sufficient for this. Using Nested Map with lambda function will also works here.
l = [['40', '20', '10', '30'], ['20', '20', '20', '20', '20', '30', '20'], ['30', '20', '30', '50', '10', '30', '20', '20', '20'], ['100', '100'], ['100', '100', '100', '100', '100'], ['100', '100', '100', '100']]
map(lambda x:map(lambda y:float(y),x),l)
And Output List would be as follows:
[[40.0, 20.0, 10.0, 30.0], [20.0, 20.0, 20.0, 20.0, 20.0, 30.0, 20.0], [30.0, 20.0, 30.0, 50.0, 10.0, 30.0, 20.0, 20.0, 20.0], [100.0, 100.0], [100.0, 100.0, 100.0, 100.0, 100.0], [100.0, 100.0, 100.0, 100.0]]
Here is how to convert nested for loop to nested list comprehension:
Here is how nested list comprehension works:
l a b c d e f
↓ ↓ ↓ ↓ ↓ ↓ ↓
In [1]: l = [ [ [ [ [ [ 1 ] ] ] ] ] ]
In [2]: for a in l:
...: for b in a:
...: for c in b:
...: for d in c:
...: for e in d:
...: for f in e:
...: print(float(f))
...:
1.0
In [3]: [float(f)
for a in l
...: for b in a
...: for c in b
...: for d in c
...: for e in d
...: for f in e]
Out[3]: [1.0]
For your case, if you want a flat list, it will be something like this.
In [4]: new_list = [float(y) for x in l for y in x]
I wanted to share how the list comprehension actually works, especially for nested list comprehensions:
new_list= [float(x) for x in l]
is actually the same as:
new_list=[]
for x in l:
new_list.append(float(x))
And now for the nested list comprehension:
[[float(y) for y in x] for x in l]
is the same as:
new_list=[]
for x in l:
sub_list=[]
for y in x:
sub_list.append(float(y))
new_list.append(sub_list)
print(new_list)
output:
[[40.0, 20.0, 10.0, 30.0], [20.0, 20.0, 20.0, 20.0, 20.0, 30.0, 20.0], [30.0, 20.0, 30.0, 50.0, 10.0, 30.0, 20.0, 20.0, 20.0], [100.0, 100.0], [100.0, 100.0, 100.0, 100.0, 100.0], [100.0, 100.0, 100.0, 100.0]]
I had a similar problem to solve so I came across this question. I did a performance comparison of Andrew Clark’s and narayan’s answer which I would like to share.
The primary difference between two answers is how they iterate over inner lists. One of them uses builtin map, while other is using list comprehension. Map function has slight performance advantage to its equivalent list comprehension if it doesn’t require the use lambdas. So in context of this question map should perform slightly better than list comprehension.
Lets do a performance benchmark to see if it is actually true. I used python version 3.5.0 to perform all these tests. In first set of tests I would like to keep elements per list to be 10 and vary number of lists from 10-100,000
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,10))]*10]"
>>> 100000 loops, best of 3: 15.2 usec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,10))]*10]"
>>> 10000 loops, best of 3: 19.6 usec per loop
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,10))]*100]"
>>> 100000 loops, best of 3: 15.2 usec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,10))]*100]"
>>> 10000 loops, best of 3: 19.6 usec per loop
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,10))]*1000]"
>>> 1000 loops, best of 3: 1.43 msec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,10))]*1000]"
>>> 100 loops, best of 3: 1.91 msec per loop
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,10))]*10000]"
>>> 100 loops, best of 3: 13.6 msec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,10))]*10000]"
>>> 10 loops, best of 3: 19.1 msec per loop
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,10))]*100000]"
>>> 10 loops, best of 3: 164 msec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,10))]*100000]"
>>> 10 loops, best of 3: 216 msec per loop
In the next set of tests I would like to raise number of elements per lists to 100.
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,100))]*10]"
>>> 10000 loops, best of 3: 110 usec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,100))]*10]"
>>> 10000 loops, best of 3: 151 usec per loop
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,100))]*100]"
>>> 1000 loops, best of 3: 1.11 msec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,100))]*100]"
>>> 1000 loops, best of 3: 1.5 msec per loop
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,100))]*1000]"
>>> 100 loops, best of 3: 11.2 msec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,100))]*1000]"
>>> 100 loops, best of 3: 16.7 msec per loop
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,100))]*10000]"
>>> 10 loops, best of 3: 134 msec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,100))]*10000]"
>>> 10 loops, best of 3: 171 msec per loop
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,100))]*100000]"
>>> 10 loops, best of 3: 1.32 sec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,100))]*100000]"
>>> 10 loops, best of 3: 1.7 sec per loop
Lets take a brave step and modify the number of elements in lists to be 1000
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,1000))]*10]"
>>> 1000 loops, best of 3: 800 usec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,1000))]*10]"
>>> 1000 loops, best of 3: 1.16 msec per loop
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,1000))]*100]"
>>> 100 loops, best of 3: 8.26 msec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,1000))]*100]"
>>> 100 loops, best of 3: 11.7 msec per loop
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,1000))]*1000]"
>>> 10 loops, best of 3: 83.8 msec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,1000))]*1000]"
>>> 10 loops, best of 3: 118 msec per loop
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,1000))]*10000]"
>>> 10 loops, best of 3: 868 msec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,1000))]*10000]"
>>> 10 loops, best of 3: 1.23 sec per loop
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,1000))]*100000]"
>>> 10 loops, best of 3: 9.2 sec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,1000))]*100000]"
>>> 10 loops, best of 3: 12.7 sec per loop
From these test we can conclude that map has a performance benefit over list comprehension in this case. This is also applicable if you are trying to cast to either int or str. For small number of lists with less elements per list, the difference is negligible. For larger lists with more elements per list one might like to use map instead of list comprehension, but it totally depends on application needs.
However I personally find list comprehension to be more readable and idiomatic than map. It is a de-facto standard in python. Usually people are more proficient and comfortable(specially beginner) in using list comprehension than map.
deck = []
for rank in ranks:
for suit in suits:
deck.append(('%s%s')%(rank, suit))
This can be achieved using list comprehension:
[deck.append((rank,suit)) for suit in suits for rank in ranks ]
Yes you can do the following.
[[float(y) for y in x] for x in l]
In case a flattened list is needed:
[y for x in l for y in x]
In case a nested list (list in list) is needed:
[[float(y) for y in x] for x in l]
I have this nested list:
l = [['40', '20', '10', '30'], ['20', '20', '20', '20', '20', '30', '20'], ['30', '20', '30', '50', '10', '30', '20', '20', '20'], ['100', '100'], ['100', '100', '100', '100', '100'], ['100', '100', '100', '100']]
I want to convert each element in l to float. I have this code:
newList = []
for x in l:
for y in x:
newList.append(float(y))
How can I solve the problem with a nested list comprehension instead?
See also: How can I get a flat result from a list comprehension instead of a nested list?
>>> l = [['40', '20', '10', '30'], ['20', '20', '20', '20', '20', '30', '20'], ['30', '20', '30', '50', '10', '30', '20', '20', '20'], ['100', '100'], ['100', '100', '100', '100', '100'], ['100', '100', '100', '100']]
>>> new_list = [float(x) for xs in l for x in xs]
>>> new_list
[40.0, 20.0, 10.0, 30.0, 20.0, 20.0, 20.0, 20.0, 20.0, 30.0, 20.0, 30.0, 20.0, 30.0, 50.0, 10.0, 30.0, 20.0, 20.0, 20.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0]
Here is how you would do this with a nested list comprehension:
[[float(y) for y in x] for x in l]
This would give you a list of lists, similar to what you started with except with floats instead of strings.
If you want one flat list, then you would use
[float(y) for x in l for y in x]
Note the loop order – for x in l comes first in this one.
Yes, you can do it with such a code:
l = [[float(y) for y in x] for x in l]
If you don’t like nested list comprehensions, you can make use of the map function as well,
>>> from pprint import pprint
>>> l = l = [['40', '20', '10', '30'], ['20', '20', '20', '20', '20', '30', '20'], ['30', '20', '30', '50', '10', '30', '20', '20', '20'], ['100', '100'], ['100', '100', '100', '100', '100'], ['100', '100', '100', '100']]
>>> pprint(l)
[['40', '20', '10', '30'],
['20', '20', '20', '20', '20', '30', '20'],
['30', '20', '30', '50', '10', '30', '20', '20', '20'],
['100', '100'],
['100', '100', '100', '100', '100'],
['100', '100', '100', '100']]
>>> float_l = [map(float, nested_list) for nested_list in l]
>>> pprint(float_l)
[[40.0, 20.0, 10.0, 30.0],
[20.0, 20.0, 20.0, 20.0, 20.0, 30.0, 20.0],
[30.0, 20.0, 30.0, 50.0, 10.0, 30.0, 20.0, 20.0, 20.0],
[100.0, 100.0],
[100.0, 100.0, 100.0, 100.0, 100.0],
[100.0, 100.0, 100.0, 100.0]]
The best way to do this in my opinion is to use python’s itertools package.
>>>import itertools
>>>l1 = [1,2,3]
>>>l2 = [10,20,30]
>>>[l*2 for l in itertools.chain(*[l1,l2])]
[2, 4, 6, 20, 40, 60]
Not sure what your desired output is, but if you’re using list comprehension, the order follows the order of nested loops, which you have backwards. So I got the what I think you want with:
[float(y) for x in l for y in x]
The principle is: use the same order you’d use in writing it out as nested for loops.
This Problem can be solved without using for loop.Single line code will be sufficient for this. Using Nested Map with lambda function will also works here.
l = [['40', '20', '10', '30'], ['20', '20', '20', '20', '20', '30', '20'], ['30', '20', '30', '50', '10', '30', '20', '20', '20'], ['100', '100'], ['100', '100', '100', '100', '100'], ['100', '100', '100', '100']]
map(lambda x:map(lambda y:float(y),x),l)
And Output List would be as follows:
[[40.0, 20.0, 10.0, 30.0], [20.0, 20.0, 20.0, 20.0, 20.0, 30.0, 20.0], [30.0, 20.0, 30.0, 50.0, 10.0, 30.0, 20.0, 20.0, 20.0], [100.0, 100.0], [100.0, 100.0, 100.0, 100.0, 100.0], [100.0, 100.0, 100.0, 100.0]]
Here is how to convert nested for loop to nested list comprehension:
Here is how nested list comprehension works:
l a b c d e f
↓ ↓ ↓ ↓ ↓ ↓ ↓
In [1]: l = [ [ [ [ [ [ 1 ] ] ] ] ] ]
In [2]: for a in l:
...: for b in a:
...: for c in b:
...: for d in c:
...: for e in d:
...: for f in e:
...: print(float(f))
...:
1.0
In [3]: [float(f)
for a in l
...: for b in a
...: for c in b
...: for d in c
...: for e in d
...: for f in e]
Out[3]: [1.0]
For your case, if you want a flat list, it will be something like this.
In [4]: new_list = [float(y) for x in l for y in x]
I wanted to share how the list comprehension actually works, especially for nested list comprehensions:
new_list= [float(x) for x in l]
is actually the same as:
new_list=[]
for x in l:
new_list.append(float(x))
And now for the nested list comprehension:
[[float(y) for y in x] for x in l]
is the same as:
new_list=[]
for x in l:
sub_list=[]
for y in x:
sub_list.append(float(y))
new_list.append(sub_list)
print(new_list)
output:
[[40.0, 20.0, 10.0, 30.0], [20.0, 20.0, 20.0, 20.0, 20.0, 30.0, 20.0], [30.0, 20.0, 30.0, 50.0, 10.0, 30.0, 20.0, 20.0, 20.0], [100.0, 100.0], [100.0, 100.0, 100.0, 100.0, 100.0], [100.0, 100.0, 100.0, 100.0]]
I had a similar problem to solve so I came across this question. I did a performance comparison of Andrew Clark’s and narayan’s answer which I would like to share.
The primary difference between two answers is how they iterate over inner lists. One of them uses builtin map, while other is using list comprehension. Map function has slight performance advantage to its equivalent list comprehension if it doesn’t require the use lambdas. So in context of this question map should perform slightly better than list comprehension.
Lets do a performance benchmark to see if it is actually true. I used python version 3.5.0 to perform all these tests. In first set of tests I would like to keep elements per list to be 10 and vary number of lists from 10-100,000
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,10))]*10]"
>>> 100000 loops, best of 3: 15.2 usec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,10))]*10]"
>>> 10000 loops, best of 3: 19.6 usec per loop
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,10))]*100]"
>>> 100000 loops, best of 3: 15.2 usec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,10))]*100]"
>>> 10000 loops, best of 3: 19.6 usec per loop
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,10))]*1000]"
>>> 1000 loops, best of 3: 1.43 msec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,10))]*1000]"
>>> 100 loops, best of 3: 1.91 msec per loop
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,10))]*10000]"
>>> 100 loops, best of 3: 13.6 msec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,10))]*10000]"
>>> 10 loops, best of 3: 19.1 msec per loop
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,10))]*100000]"
>>> 10 loops, best of 3: 164 msec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,10))]*100000]"
>>> 10 loops, best of 3: 216 msec per loop
In the next set of tests I would like to raise number of elements per lists to 100.
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,100))]*10]"
>>> 10000 loops, best of 3: 110 usec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,100))]*10]"
>>> 10000 loops, best of 3: 151 usec per loop
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,100))]*100]"
>>> 1000 loops, best of 3: 1.11 msec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,100))]*100]"
>>> 1000 loops, best of 3: 1.5 msec per loop
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,100))]*1000]"
>>> 100 loops, best of 3: 11.2 msec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,100))]*1000]"
>>> 100 loops, best of 3: 16.7 msec per loop
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,100))]*10000]"
>>> 10 loops, best of 3: 134 msec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,100))]*10000]"
>>> 10 loops, best of 3: 171 msec per loop
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,100))]*100000]"
>>> 10 loops, best of 3: 1.32 sec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,100))]*100000]"
>>> 10 loops, best of 3: 1.7 sec per loop
Lets take a brave step and modify the number of elements in lists to be 1000
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,1000))]*10]"
>>> 1000 loops, best of 3: 800 usec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,1000))]*10]"
>>> 1000 loops, best of 3: 1.16 msec per loop
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,1000))]*100]"
>>> 100 loops, best of 3: 8.26 msec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,1000))]*100]"
>>> 100 loops, best of 3: 11.7 msec per loop
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,1000))]*1000]"
>>> 10 loops, best of 3: 83.8 msec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,1000))]*1000]"
>>> 10 loops, best of 3: 118 msec per loop
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,1000))]*10000]"
>>> 10 loops, best of 3: 868 msec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,1000))]*10000]"
>>> 10 loops, best of 3: 1.23 sec per loop
>>> python -m timeit "[list(map(float,k)) for k in [list(range(0,1000))]*100000]"
>>> 10 loops, best of 3: 9.2 sec per loop
>>> python -m timeit "[[float(y) for y in x] for x in [list(range(0,1000))]*100000]"
>>> 10 loops, best of 3: 12.7 sec per loop
From these test we can conclude that map has a performance benefit over list comprehension in this case. This is also applicable if you are trying to cast to either int or str. For small number of lists with less elements per list, the difference is negligible. For larger lists with more elements per list one might like to use map instead of list comprehension, but it totally depends on application needs.
However I personally find list comprehension to be more readable and idiomatic than map. It is a de-facto standard in python. Usually people are more proficient and comfortable(specially beginner) in using list comprehension than map.
deck = []
for rank in ranks:
for suit in suits:
deck.append(('%s%s')%(rank, suit))
This can be achieved using list comprehension:
[deck.append((rank,suit)) for suit in suits for rank in ranks ]
Yes you can do the following.
[[float(y) for y in x] for x in l]
In case a flattened list is needed:
[y for x in l for y in x]
In case a nested list (list in list) is needed:
[[float(y) for y in x] for x in l]