Understand Python swapping: why is a, b = b, a not always equivalent to b, a = a, b?
Question:
As we all know, the pythonic way to swap the values of two items a and b is
a, b = b, a
and it should be equivalent to
b, a = a, b
However, today when I was working on some code, I accidentally found that the following two swaps give different results:
nums = [1, 2, 4, 3]
i = 2
nums[i], nums[nums[i]-1] = nums[nums[i]-1], nums[i]
print(nums)
# [1, 2, 4, 3]
nums = [1, 2, 4, 3]
i = 2
nums[nums[i]-1], nums[i] = nums[i], nums[nums[i]-1]
print(nums)
# [1, 2, 3, 4]
This is mind-boggling to me. Can someone explain to me what happened here? I thought in a Python swap the two assignments happen simultaneously and independently.
Answers:
This is because evaluation — specifically at the left side of the = — happens from left to right:
nums[i], nums[nums[i]-1] =
First nums[i] gets assigned, and then that value is used to determine the index in the assignment to nums[nums[i]-1]
When doing the assignment like this:
nums[nums[i]-1], nums[i] =
… the index of nums[nums[i]-1] is dependent on the old value of nums[i], since the assignment to nums[i] still follows later…
From python.org
Assignment of an object to a target list, optionally enclosed in parentheses or square brackets, is recursively defined as follows.
…
- Else: The object must be an iterable with the same number of items as there are targets in the target list, and the items are assigned, from left to right, to the corresponding targets.
So I interpret that to mean that your assignment
nums[i], nums[nums[i]-1] = nums[nums[i]-1], nums[i]
is roughly equivalent to
tmp = nums[nums[i]-1], nums[i]
nums[i] = tmp[0]
nums[nums[i] - 1] = tmp[1]
(with better error-checking, of course)
whereas the other
nums[nums[i]-1], nums[i] = nums[i], nums[nums[i]-1]
is like
tmp = nums[i], nums[nums[i]-1]
nums[nums[i] - 1] = tmp[0]
nums[i] = tmp[1]
So the right-hand side is evaluated first in both cases. But then the two pieces of the left-hand side are evaluated in order, and the assignments are done immediately after evaluation. Crucially, this means that the second term on the left-hand side is only evaluated after the first assignment is already done. So if you update nums[i] first, then the nums[nums[i] - 1] refers to a different index than if you update nums[i] second.
This happens according to the rules:
- The right hand side is evaluated first
- Then, each value of the left hand side gets its new value, from left to right.
So, with nums = [1, 2, 4, 3], your code in the first case
nums[2], nums[nums[2]-1] = nums[nums[2]-1], nums[2]
is equivalent to:
nums[2], nums[nums[2]-1] = nums[nums[2]-1], nums[2]
nums[2], nums[nums[2]-1] = nums[3], nums[2]
nums[2], nums[nums[2]-1] = 3, 4
and as the right hand side is now evaluated, the assignments are equivalent to:
nums[2] = 3
nums[nums[2]-1] = 4
nums[2] = 3
nums[3-1] = 4
nums[2] = 3
nums[2] = 4
which gives:
print(nums)
# [1, 2, 4, 3]
In the second case, we get:
nums[nums[2]-1], nums[2] = nums[2], nums[nums[2]-1]
nums[nums[2]-1], nums[2] = nums[2], nums[3]
nums[nums[2]-1], nums[2] = 4, 3
nums[nums[2]-1] = 4
nums[2] = 3
nums[4-1] = 4
nums[2] = 3
nums[3] = 4
nums[2] = 3
print(nums)
# [1, 2, 3, 4]
On the left hand side of your expression you are both reading and writing nums[i], I dunno if python gaurantees processing of unpacking operations in left to right order, but lets assume it does, your first example would be equivilent to.
t = nums[nums[i]-1], nums[i] # t = (3,4)
nums[i] = t[0] # nums = [1,2,3,3]
n = nums[i]-1 # n = 2
nums[n] = t[1] # nums = [1,2,4,3]
While your second example would be equivilent to
t = nums[i], nums[nums[i]-1] # t = (4,3)
n = nums[i]-1 # n = 3
nums[n] = t[0] # nums = [1,2,4,4]
nums[i] = t[0] # nums = [1,2,3,4]
Which is consistent with what you got.
To understand the order of evaluation I made a ‘Variable’ class that prints when sets and gets occur to it’s ‘value’.
class Variable:
def __init__(self, name, value):
self._name = name
self._value = value
@property
def value(self):
print(self._name, 'get', self._value)
return self._value
@value.setter
def value(self):
print(self._name, 'set', self._value)
self._value = value
a = Variable('a', 1)
b = Variable('b', 2)
a.value, b.value = b.value, a.value
When run results in:
b get 2
a get 1
a set 2
b set 1
This shows that the right side is evaluated first (from left to right) and then the left side is evaluated (again, from left to right).
Regarding the OP’s example:
Right side will evaluate to same values in both cases. Left side first term is set and this impacts the evaluation of the second term. It was never simultaneous and independently evaluated, it’s just most times you see this used, the terms don’t depended on each other. Setting a value in a list and then taking a value from the that list to use as an index in the same list is usually not a thing and if that’s hard to understand, you understand it. Like changing the length of a list in a for loop is bad, this has the same smell. (Stimulating question though, as you may have guessed from me running to a scratch pad)
One way to analyze code snippets in CPython is to disassemble its bytecode for its simulated stack machine.
>>> import dis
>>> dis.dis("nums[i], nums[nums[i]-1] = nums[nums[i]-1], nums[i]")
1 0 LOAD_NAME 0 (nums)
2 LOAD_NAME 0 (nums)
4 LOAD_NAME 1 (i)
6 BINARY_SUBSCR
8 LOAD_CONST 0 (1)
10 BINARY_SUBTRACT
12 BINARY_SUBSCR
14 LOAD_NAME 0 (nums)
16 LOAD_NAME 1 (i)
18 BINARY_SUBSCR
20 ROT_TWO
22 LOAD_NAME 0 (nums)
24 LOAD_NAME 1 (i)
26 STORE_SUBSCR
28 LOAD_NAME 0 (nums)
30 LOAD_NAME 0 (nums)
32 LOAD_NAME 1 (i)
34 BINARY_SUBSCR
36 LOAD_CONST 0 (1)
38 BINARY_SUBTRACT
40 STORE_SUBSCR
42 LOAD_CONST 1 (None)
44 RETURN_VALUE
I added the blank lines to make reading this easier. The two fetch expressions are calculated in bytes 0-13 and 14-19. BINARY_SUBSCR replaces the top two values on the stack, an object and subscript, with the value fetched from the object. The two fetched values are swapped so that the first calculated is the first bound. The two store operations are done in bytes 22-27 and 28-41. STORE_SUBSCR uses and removes the top three values on the stack, a value to be stored, an object, and a subscript. (The return None part is apparently always added at the end.) The important part for the question is that the calculations for the stores are done sequentially in separate and independent batches.
The closest description in Python of the CPython calculation requires introduction of a stack variable
stack = []
stack.append(nums[nums[i]-1])
stack.append(nums[i])
stack.reverse()
nums[i] = stack.pop()
nums[nums[i]-1] = stack.pop()
Here is the disassembly for the reversed statement
>>> dis.dis("nums[nums[i]-1], nums[i] = nums[i], nums[nums[i]-1]")
1 0 LOAD_NAME 0 (nums)
2 LOAD_NAME 1 (i)
4 BINARY_SUBSCR
6 LOAD_NAME 0 (nums)
8 LOAD_NAME 0 (nums)
10 LOAD_NAME 1 (i)
12 BINARY_SUBSCR
14 LOAD_CONST 0 (1)
16 BINARY_SUBTRACT
18 BINARY_SUBSCR
20 ROT_TWO
22 LOAD_NAME 0 (nums)
24 LOAD_NAME 0 (nums)
26 LOAD_NAME 1 (i)
28 BINARY_SUBSCR
30 LOAD_CONST 0 (1)
32 BINARY_SUBTRACT
34 STORE_SUBSCR
36 LOAD_NAME 0 (nums)
38 LOAD_NAME 1 (i)
40 STORE_SUBSCR
42 LOAD_CONST 1 (None)
44 RETURN_VALUE
It seems to me that this would only happen when the contents of the list is in the range of list indexes for the list. If for example:
nums = [10, 20, 40, 30]
The code will fail with:
>>> nums[i], nums[nums[i]-1] = nums[nums[i]-1], nums[i]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IndexError: list index out of range
So definitely, a gotcha. Never Ever Use the Contents of a list as an index into that list.
Thierry did give a good answer, let me be more clear. Be aware that if nums = [1, 2, 4, 3],
in this code:
nums[nums[i]-1], nums[i]
- i is 2,
- nums[nums[i]-1] is nums[4-1], so nums[3], (value is 3)
- nums[i] is nums[2], (value is 4)
- the result is: (3, 4)
in this code:
nums[i], nums[nums[i]-1]
- nums[i] is nums[2] becomes 3, (=>[1, 2, 3, 3])
- but nums[nums[i]-1] is not nums[4-1] but nums[3-1], so nums[2] too, becomes (back to) 4 (=>[1, 2, 4, 3])
Perhaps the good question concerning a swap, was using:
nums[i], nums[i-1] = nums[i-1], nums[i] ?
Try it:
>>> print(nums)
>>> [1, 2, 4, 3]
>>> nums[i], nums[i-1] = nums[i-1], nums[i]
>>> print(nums)
>>> [1, 4, 2, 3]
ChD
As we all know, the pythonic way to swap the values of two items a and b is
a, b = b, a
and it should be equivalent to
b, a = a, b
However, today when I was working on some code, I accidentally found that the following two swaps give different results:
nums = [1, 2, 4, 3]
i = 2
nums[i], nums[nums[i]-1] = nums[nums[i]-1], nums[i]
print(nums)
# [1, 2, 4, 3]
nums = [1, 2, 4, 3]
i = 2
nums[nums[i]-1], nums[i] = nums[i], nums[nums[i]-1]
print(nums)
# [1, 2, 3, 4]
This is mind-boggling to me. Can someone explain to me what happened here? I thought in a Python swap the two assignments happen simultaneously and independently.
This is because evaluation — specifically at the left side of the = — happens from left to right:
nums[i], nums[nums[i]-1] =
First nums[i] gets assigned, and then that value is used to determine the index in the assignment to nums[nums[i]-1]
When doing the assignment like this:
nums[nums[i]-1], nums[i] =
… the index of nums[nums[i]-1] is dependent on the old value of nums[i], since the assignment to nums[i] still follows later…
From python.org
Assignment of an object to a target list, optionally enclosed in parentheses or square brackets, is recursively defined as follows.
…
- Else: The object must be an iterable with the same number of items as there are targets in the target list, and the items are assigned, from left to right, to the corresponding targets.
So I interpret that to mean that your assignment
nums[i], nums[nums[i]-1] = nums[nums[i]-1], nums[i]
is roughly equivalent to
tmp = nums[nums[i]-1], nums[i]
nums[i] = tmp[0]
nums[nums[i] - 1] = tmp[1]
(with better error-checking, of course)
whereas the other
nums[nums[i]-1], nums[i] = nums[i], nums[nums[i]-1]
is like
tmp = nums[i], nums[nums[i]-1]
nums[nums[i] - 1] = tmp[0]
nums[i] = tmp[1]
So the right-hand side is evaluated first in both cases. But then the two pieces of the left-hand side are evaluated in order, and the assignments are done immediately after evaluation. Crucially, this means that the second term on the left-hand side is only evaluated after the first assignment is already done. So if you update nums[i] first, then the nums[nums[i] - 1] refers to a different index than if you update nums[i] second.
This happens according to the rules:
- The right hand side is evaluated first
- Then, each value of the left hand side gets its new value, from left to right.
So, with nums = [1, 2, 4, 3], your code in the first case
nums[2], nums[nums[2]-1] = nums[nums[2]-1], nums[2]
is equivalent to:
nums[2], nums[nums[2]-1] = nums[nums[2]-1], nums[2]
nums[2], nums[nums[2]-1] = nums[3], nums[2]
nums[2], nums[nums[2]-1] = 3, 4
and as the right hand side is now evaluated, the assignments are equivalent to:
nums[2] = 3
nums[nums[2]-1] = 4
nums[2] = 3
nums[3-1] = 4
nums[2] = 3
nums[2] = 4
which gives:
print(nums)
# [1, 2, 4, 3]
In the second case, we get:
nums[nums[2]-1], nums[2] = nums[2], nums[nums[2]-1]
nums[nums[2]-1], nums[2] = nums[2], nums[3]
nums[nums[2]-1], nums[2] = 4, 3
nums[nums[2]-1] = 4
nums[2] = 3
nums[4-1] = 4
nums[2] = 3
nums[3] = 4
nums[2] = 3
print(nums)
# [1, 2, 3, 4]
On the left hand side of your expression you are both reading and writing nums[i], I dunno if python gaurantees processing of unpacking operations in left to right order, but lets assume it does, your first example would be equivilent to.
t = nums[nums[i]-1], nums[i] # t = (3,4)
nums[i] = t[0] # nums = [1,2,3,3]
n = nums[i]-1 # n = 2
nums[n] = t[1] # nums = [1,2,4,3]
While your second example would be equivilent to
t = nums[i], nums[nums[i]-1] # t = (4,3)
n = nums[i]-1 # n = 3
nums[n] = t[0] # nums = [1,2,4,4]
nums[i] = t[0] # nums = [1,2,3,4]
Which is consistent with what you got.
To understand the order of evaluation I made a ‘Variable’ class that prints when sets and gets occur to it’s ‘value’.
class Variable:
def __init__(self, name, value):
self._name = name
self._value = value
@property
def value(self):
print(self._name, 'get', self._value)
return self._value
@value.setter
def value(self):
print(self._name, 'set', self._value)
self._value = value
a = Variable('a', 1)
b = Variable('b', 2)
a.value, b.value = b.value, a.value
When run results in:
b get 2
a get 1
a set 2
b set 1
This shows that the right side is evaluated first (from left to right) and then the left side is evaluated (again, from left to right).
Regarding the OP’s example:
Right side will evaluate to same values in both cases. Left side first term is set and this impacts the evaluation of the second term. It was never simultaneous and independently evaluated, it’s just most times you see this used, the terms don’t depended on each other. Setting a value in a list and then taking a value from the that list to use as an index in the same list is usually not a thing and if that’s hard to understand, you understand it. Like changing the length of a list in a for loop is bad, this has the same smell. (Stimulating question though, as you may have guessed from me running to a scratch pad)
One way to analyze code snippets in CPython is to disassemble its bytecode for its simulated stack machine.
>>> import dis
>>> dis.dis("nums[i], nums[nums[i]-1] = nums[nums[i]-1], nums[i]")
1 0 LOAD_NAME 0 (nums)
2 LOAD_NAME 0 (nums)
4 LOAD_NAME 1 (i)
6 BINARY_SUBSCR
8 LOAD_CONST 0 (1)
10 BINARY_SUBTRACT
12 BINARY_SUBSCR
14 LOAD_NAME 0 (nums)
16 LOAD_NAME 1 (i)
18 BINARY_SUBSCR
20 ROT_TWO
22 LOAD_NAME 0 (nums)
24 LOAD_NAME 1 (i)
26 STORE_SUBSCR
28 LOAD_NAME 0 (nums)
30 LOAD_NAME 0 (nums)
32 LOAD_NAME 1 (i)
34 BINARY_SUBSCR
36 LOAD_CONST 0 (1)
38 BINARY_SUBTRACT
40 STORE_SUBSCR
42 LOAD_CONST 1 (None)
44 RETURN_VALUE
I added the blank lines to make reading this easier. The two fetch expressions are calculated in bytes 0-13 and 14-19. BINARY_SUBSCR replaces the top two values on the stack, an object and subscript, with the value fetched from the object. The two fetched values are swapped so that the first calculated is the first bound. The two store operations are done in bytes 22-27 and 28-41. STORE_SUBSCR uses and removes the top three values on the stack, a value to be stored, an object, and a subscript. (The return None part is apparently always added at the end.) The important part for the question is that the calculations for the stores are done sequentially in separate and independent batches.
The closest description in Python of the CPython calculation requires introduction of a stack variable
stack = []
stack.append(nums[nums[i]-1])
stack.append(nums[i])
stack.reverse()
nums[i] = stack.pop()
nums[nums[i]-1] = stack.pop()
Here is the disassembly for the reversed statement
>>> dis.dis("nums[nums[i]-1], nums[i] = nums[i], nums[nums[i]-1]")
1 0 LOAD_NAME 0 (nums)
2 LOAD_NAME 1 (i)
4 BINARY_SUBSCR
6 LOAD_NAME 0 (nums)
8 LOAD_NAME 0 (nums)
10 LOAD_NAME 1 (i)
12 BINARY_SUBSCR
14 LOAD_CONST 0 (1)
16 BINARY_SUBTRACT
18 BINARY_SUBSCR
20 ROT_TWO
22 LOAD_NAME 0 (nums)
24 LOAD_NAME 0 (nums)
26 LOAD_NAME 1 (i)
28 BINARY_SUBSCR
30 LOAD_CONST 0 (1)
32 BINARY_SUBTRACT
34 STORE_SUBSCR
36 LOAD_NAME 0 (nums)
38 LOAD_NAME 1 (i)
40 STORE_SUBSCR
42 LOAD_CONST 1 (None)
44 RETURN_VALUE
It seems to me that this would only happen when the contents of the list is in the range of list indexes for the list. If for example:
nums = [10, 20, 40, 30]
The code will fail with:
>>> nums[i], nums[nums[i]-1] = nums[nums[i]-1], nums[i]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IndexError: list index out of range
So definitely, a gotcha. Never Ever Use the Contents of a list as an index into that list.
Thierry did give a good answer, let me be more clear. Be aware that if nums = [1, 2, 4, 3],
in this code:
nums[nums[i]-1], nums[i]
- i is 2,
- nums[nums[i]-1] is nums[4-1], so nums[3], (value is 3)
- nums[i] is nums[2], (value is 4)
- the result is: (3, 4)
in this code:
nums[i], nums[nums[i]-1]
- nums[i] is nums[2] becomes 3, (=>[1, 2, 3, 3])
- but nums[nums[i]-1] is not nums[4-1] but nums[3-1], so nums[2] too, becomes (back to) 4 (=>[1, 2, 4, 3])
Perhaps the good question concerning a swap, was using:
nums[i], nums[i-1] = nums[i-1], nums[i] ?
Try it:
>>> print(nums)
>>> [1, 2, 4, 3]
>>> nums[i], nums[i-1] = nums[i-1], nums[i]
>>> print(nums)
>>> [1, 4, 2, 3]
ChD