Why are arbitrary target expressions allowed in for-loops?
Question:
I accidentally wrote some code like this:
foo = [42]
k = {'c': 'd'}
for k['z'] in foo: # Huh??
print k
But to my surprise, this was not a syntax error. Instead, it prints {'c': 'd', 'z': 42}.
My guess is that the code is translated literally to something like:
i = iter(foo)
while True:
try:
k['z'] = i.next() # literally translated to assignment; modifies k!
print k
except StopIteration:
break
But… why is this allowed by the language? I would expect only single identifiers and tuples of identifiers should be allowed in the for-stmt’s target expression. Is there any situation in which this is actually useful, not just a weird gotcha?
Answers:
The for loop follows the standard rules of assignment so what works on the LHS of a vanilla assignment should work with the for:
Each item in turn is assigned to the target list using the standard
rules for assignments
The for construct simply summons the underlying mechanism for assigning to the target which in the case of your sample code is STORE_SUBSCR:
>>> foo = [42]
>>> k = {'c': 'd'}
>>> dis.dis('for k["e"] in foo: pass')
1 0 SETUP_LOOP 16 (to 18)
2 LOAD_NAME 0 (foo)
4 GET_ITER
>> 6 FOR_ITER 8 (to 16)
8 LOAD_NAME 1 (k)
10 LOAD_CONST 0 ('e')
12 STORE_SUBSCR <--------------------
14 JUMP_ABSOLUTE 6
>> 16 POP_BLOCK
>> 18 LOAD_CONST 1 (None)
20 RETURN_VALUE
But to my surprise, this was not a syntax error
Apparently, whatever works in a regular assignment such as the following:
full slice assignment:
>>> for [][:] in []:
... pass
...
>>>
list subscription
>>> for [2][0] in [42]:
... pass
...
>>>
dictionary subscription etc. would be valid candidate targets, with the lone exception being a chained assignment; although, I secretly think one can cook up some dirty syntax to perform the chaining.
I would expect only single identifiers and tuples of identifiers
I can’t think of a good use case for a dictionary key as a target. Besides, it is more readable to do the dictionary key assignment in the loop body, than use it as a target in the for clause.
However, extended unpacking (Python 3) which is very useful in regular assignments also comes equally handy in a for loop:
>>> lst = [[1, '', '', 3], [3, '', '', 6]]
>>> for x, *y, z in lst:
... print(x,y,z)
...
1 ['', ''] 3
3 ['', ''] 6
The corresponding mechanism for assigning to the different targets here is also summoned; multiple STORE_NAMEs:
>>> dis.dis('for x, *y, z in lst: pass')
1 0 SETUP_LOOP 20 (to 22)
2 LOAD_NAME 0 (lst)
4 GET_ITER
>> 6 FOR_ITER 12 (to 20)
8 EXTENDED_ARG 1
10 UNPACK_EX 257
12 STORE_NAME 1 (x) <-----
14 STORE_NAME 2 (y) <-----
16 STORE_NAME 3 (z) <-----
18 JUMP_ABSOLUTE 6
>> 20 POP_BLOCK
>> 22 LOAD_CONST 0 (None)
24 RETURN_VALUE
Goes to show that a for is barely simple assignment statements executed successively.
Is there any situation in which this is actually useful?
Indeed. Ever wanted to get rid of itertools.combinations?
def combinations (pool, repeat):
def combinations_recurse (acc, pool, index = 0):
if index < len(acc):
for acc[index] in pool:
yield from combinations_recurse(acc, pool, index + 1)
else:
yield acc
yield from combinations_recurse([pool[0]] * repeat, pool)
for comb in combinations([0, 1], 3):
print(comb)
The following code would make sense, right?
foo = [42]
for x in foo:
print x
The for loop would iterate over the list foo and assign each object to the name x in the current namespace in turn. The result would be a single iteration and a single print of 42.
In place of x in your code, you have k['z']. k['z'] is a valid storage name. Like x in my example, it doesn’t yet exist. It is, in effect, k.z in the global namespace. The loop creates k.z or k['z'] and assigns the the values it finds in foo to it in the same way it would create x and assign the values to it in my example. If you had more values in foo…
foo = [42, 51, "bill", "ted"]
k = {'c': 'd'}
for k['z'] in foo:
print k
would result in:
{'c': 'd', 'z': 42}
{'c': 'd', 'z': 51}
{'c': 'd', 'z': 'bill'}
{'c': 'd', 'z': 'ted'}
You wrote perfectly valid accidental code. It’s not even strange code. You just usually don’t think of dictionary entries as variables.
Even if the code isn’t strange, how can allowing such an assignment be useful?
key_list = ['home', 'car', 'bike', 'locker']
loc_list = ['under couch', 'on counter', 'in garage', 'in locker']
chain = {}
for index, chain[key_list[index]] in enumerate(loc_list):
pass
Probably not the best way to do that, but puts two equal length lists together into a dictionary. I’m sure there are other things more experienced programmers have used dictionary key assignment in for loops for. Maybe…
Every name is just a dictionary key*.
for x in blah:
is precisely
for vars()['x'] in blah:
* (though that dictionary needn’t be implemented as an actual dict object, in case of some optimizations, such as in function scopes).
I accidentally wrote some code like this:
foo = [42]
k = {'c': 'd'}
for k['z'] in foo: # Huh??
print k
But to my surprise, this was not a syntax error. Instead, it prints {'c': 'd', 'z': 42}.
My guess is that the code is translated literally to something like:
i = iter(foo)
while True:
try:
k['z'] = i.next() # literally translated to assignment; modifies k!
print k
except StopIteration:
break
But… why is this allowed by the language? I would expect only single identifiers and tuples of identifiers should be allowed in the for-stmt’s target expression. Is there any situation in which this is actually useful, not just a weird gotcha?
The for loop follows the standard rules of assignment so what works on the LHS of a vanilla assignment should work with the for:
Each item in turn is assigned to the target list using the standard
rules for assignments
The for construct simply summons the underlying mechanism for assigning to the target which in the case of your sample code is STORE_SUBSCR:
>>> foo = [42]
>>> k = {'c': 'd'}
>>> dis.dis('for k["e"] in foo: pass')
1 0 SETUP_LOOP 16 (to 18)
2 LOAD_NAME 0 (foo)
4 GET_ITER
>> 6 FOR_ITER 8 (to 16)
8 LOAD_NAME 1 (k)
10 LOAD_CONST 0 ('e')
12 STORE_SUBSCR <--------------------
14 JUMP_ABSOLUTE 6
>> 16 POP_BLOCK
>> 18 LOAD_CONST 1 (None)
20 RETURN_VALUE
But to my surprise, this was not a syntax error
Apparently, whatever works in a regular assignment such as the following:
full slice assignment:
>>> for [][:] in []:
... pass
...
>>>
list subscription
>>> for [2][0] in [42]:
... pass
...
>>>
dictionary subscription etc. would be valid candidate targets, with the lone exception being a chained assignment; although, I secretly think one can cook up some dirty syntax to perform the chaining.
I would expect only single identifiers and tuples of identifiers
I can’t think of a good use case for a dictionary key as a target. Besides, it is more readable to do the dictionary key assignment in the loop body, than use it as a target in the for clause.
However, extended unpacking (Python 3) which is very useful in regular assignments also comes equally handy in a for loop:
>>> lst = [[1, '', '', 3], [3, '', '', 6]]
>>> for x, *y, z in lst:
... print(x,y,z)
...
1 ['', ''] 3
3 ['', ''] 6
The corresponding mechanism for assigning to the different targets here is also summoned; multiple STORE_NAMEs:
>>> dis.dis('for x, *y, z in lst: pass')
1 0 SETUP_LOOP 20 (to 22)
2 LOAD_NAME 0 (lst)
4 GET_ITER
>> 6 FOR_ITER 12 (to 20)
8 EXTENDED_ARG 1
10 UNPACK_EX 257
12 STORE_NAME 1 (x) <-----
14 STORE_NAME 2 (y) <-----
16 STORE_NAME 3 (z) <-----
18 JUMP_ABSOLUTE 6
>> 20 POP_BLOCK
>> 22 LOAD_CONST 0 (None)
24 RETURN_VALUE
Goes to show that a for is barely simple assignment statements executed successively.
Is there any situation in which this is actually useful?
Indeed. Ever wanted to get rid of itertools.combinations?
def combinations (pool, repeat):
def combinations_recurse (acc, pool, index = 0):
if index < len(acc):
for acc[index] in pool:
yield from combinations_recurse(acc, pool, index + 1)
else:
yield acc
yield from combinations_recurse([pool[0]] * repeat, pool)
for comb in combinations([0, 1], 3):
print(comb)
The following code would make sense, right?
foo = [42]
for x in foo:
print x
The for loop would iterate over the list foo and assign each object to the name x in the current namespace in turn. The result would be a single iteration and a single print of 42.
In place of x in your code, you have k['z']. k['z'] is a valid storage name. Like x in my example, it doesn’t yet exist. It is, in effect, k.z in the global namespace. The loop creates k.z or k['z'] and assigns the the values it finds in foo to it in the same way it would create x and assign the values to it in my example. If you had more values in foo…
foo = [42, 51, "bill", "ted"]
k = {'c': 'd'}
for k['z'] in foo:
print k
would result in:
{'c': 'd', 'z': 42}
{'c': 'd', 'z': 51}
{'c': 'd', 'z': 'bill'}
{'c': 'd', 'z': 'ted'}
You wrote perfectly valid accidental code. It’s not even strange code. You just usually don’t think of dictionary entries as variables.
Even if the code isn’t strange, how can allowing such an assignment be useful?
key_list = ['home', 'car', 'bike', 'locker']
loc_list = ['under couch', 'on counter', 'in garage', 'in locker']
chain = {}
for index, chain[key_list[index]] in enumerate(loc_list):
pass
Probably not the best way to do that, but puts two equal length lists together into a dictionary. I’m sure there are other things more experienced programmers have used dictionary key assignment in for loops for. Maybe…
Every name is just a dictionary key*.
for x in blah:
is precisely
for vars()['x'] in blah:
* (though that dictionary needn’t be implemented as an actual dict object, in case of some optimizations, such as in function scopes).