a += b not the same as a = a + b
Question:
Possible Duplicate:
Why does += behave unexpectedly on lists?
I found an interesting “feature” of the python language today, that gave me much grief.
>>> a = [1, 2, 3]
>>> b = "lol"
>>> a = a + b
TypeError: can only concatenate list (not "str") to list
>>> a += b
>>> a
[1, 2, 3, 'l', 'o', 'l']
How is that? I thought the two were meant to be equivalent! Even worse, this is the code that I had a hell of a time debugging
>>> a = [1, 2, 3]
>>> b = {'omg': 'noob', 'wtf' : 'bbq'}
>>> a = a + b
TypeError: can only concatenate list (not "dict") to list
>>> a += b
>>> a
[1, 2, 3, 'omg', 'wtf']
WTF! I had lists and dicts within my code, and was wondering how the hell I ended up appending the keys of my dict onto a list without ever calling .keys(). As it turns out, this is how.
I thought the two statements were meant to be equivalent. Even ignoring that, I can kind of understand the way you append strings onto lists (since strings are just character arrays) but dictionaries? Maybe if it appended a list of (key, value) tuples, but grabbing only the keys to add to the list seems completely arbitrary.
Does anyone know the logic behind this?
Answers:
This is and always has been a problem with mutability in general, and operator overloading specifically. C++ is no better.
The expression a + b computes a new list from the objects bound to a and b, which are not modified. When you assign this back to a, you change the binding of one variable to point to the new value. It is expected that + is symmetrical, so you can’t add a dict and a list.
The statement a += b modifies the existing list bound to a. Since it does not change the object identity, the changes are visible to all bindings to the object represented by a. The operator += is obviously not symmetrical, it is equivalent to list.extend, which iterates over the second operand. For dictionaries, this means listing the keys.
Discussion:
If an object doesn’t implement +=, then Python will translate it into an equivalent statement using + and =. So the two are sometimes equivalent, depending on the type of the objects involved.
The benefit of a += that mutates the referand (as opposed to the operand value, which is a reference) is that the implementation can be more efficient without a corresponding increase in implementation complexity.
In other languages, you might use more obvious notation. For example, in a hypothetical version of Python with no operator overloading, you might see:
a = concat(a, b)
versus
a.extend(a, b)
The operator notation is really just shorthand for these.
Bonus:
Try it with other iterables too.
>>> a = [1,2,3]
>>> b = "abc"
>>> a + b
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: can only concatenate list (not "str") to list
>>> a += b
>>> a
[1, 2, 3, 'a', 'b', 'c']
It’s useful to be able to do this, because you can append a generator to a list with += and get the generator contents. It’s unfortunate that it breaks compatibility with +, but oh well.
The reason behind this is because the python lists (a in your case) implement the __iadd__ method, which in turns calls the __iter__ method on the passed parameter.
The following code snippet illustrates this better:
class MyDict(dict):
def __iter__(self):
print "__iter__ was called"
return super(MyDict, self).__iter__()
class MyList(list):
def __iadd__(self, other):
print "__iadd__ was called"
return super(MyList, self).__iadd__(other)
a = MyList(['a', 'b', 'c'])
b = MyDict((('d1', 1), ('d2', 2), ('d3', 3)))
a += b
print a
The result is:
__iadd__ was called
__iter__ was called
['a', 'b', 'c', 'd2', 'd3', 'd1']
The python interpreter checks if an object implements the __iadd__ operation (+=) and only if it doesn’t it will emulate it by doing a add operation followed by an assignment.
Possible Duplicate:
Why does += behave unexpectedly on lists?
I found an interesting “feature” of the python language today, that gave me much grief.
>>> a = [1, 2, 3]
>>> b = "lol"
>>> a = a + b
TypeError: can only concatenate list (not "str") to list
>>> a += b
>>> a
[1, 2, 3, 'l', 'o', 'l']
How is that? I thought the two were meant to be equivalent! Even worse, this is the code that I had a hell of a time debugging
>>> a = [1, 2, 3]
>>> b = {'omg': 'noob', 'wtf' : 'bbq'}
>>> a = a + b
TypeError: can only concatenate list (not "dict") to list
>>> a += b
>>> a
[1, 2, 3, 'omg', 'wtf']
WTF! I had lists and dicts within my code, and was wondering how the hell I ended up appending the keys of my dict onto a list without ever calling .keys(). As it turns out, this is how.
I thought the two statements were meant to be equivalent. Even ignoring that, I can kind of understand the way you append strings onto lists (since strings are just character arrays) but dictionaries? Maybe if it appended a list of (key, value) tuples, but grabbing only the keys to add to the list seems completely arbitrary.
Does anyone know the logic behind this?
This is and always has been a problem with mutability in general, and operator overloading specifically. C++ is no better.
The expression a + b computes a new list from the objects bound to a and b, which are not modified. When you assign this back to a, you change the binding of one variable to point to the new value. It is expected that + is symmetrical, so you can’t add a dict and a list.
The statement a += b modifies the existing list bound to a. Since it does not change the object identity, the changes are visible to all bindings to the object represented by a. The operator += is obviously not symmetrical, it is equivalent to list.extend, which iterates over the second operand. For dictionaries, this means listing the keys.
Discussion:
If an object doesn’t implement +=, then Python will translate it into an equivalent statement using + and =. So the two are sometimes equivalent, depending on the type of the objects involved.
The benefit of a += that mutates the referand (as opposed to the operand value, which is a reference) is that the implementation can be more efficient without a corresponding increase in implementation complexity.
In other languages, you might use more obvious notation. For example, in a hypothetical version of Python with no operator overloading, you might see:
a = concat(a, b)
versus
a.extend(a, b)
The operator notation is really just shorthand for these.
Bonus:
Try it with other iterables too.
>>> a = [1,2,3]
>>> b = "abc"
>>> a + b
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: can only concatenate list (not "str") to list
>>> a += b
>>> a
[1, 2, 3, 'a', 'b', 'c']
It’s useful to be able to do this, because you can append a generator to a list with += and get the generator contents. It’s unfortunate that it breaks compatibility with +, but oh well.
The reason behind this is because the python lists (a in your case) implement the __iadd__ method, which in turns calls the __iter__ method on the passed parameter.
The following code snippet illustrates this better:
class MyDict(dict):
def __iter__(self):
print "__iter__ was called"
return super(MyDict, self).__iter__()
class MyList(list):
def __iadd__(self, other):
print "__iadd__ was called"
return super(MyList, self).__iadd__(other)
a = MyList(['a', 'b', 'c'])
b = MyDict((('d1', 1), ('d2', 2), ('d3', 3)))
a += b
print a
The result is:
__iadd__ was called
__iter__ was called
['a', 'b', 'c', 'd2', 'd3', 'd1']
The python interpreter checks if an object implements the __iadd__ operation (+=) and only if it doesn’t it will emulate it by doing a add operation followed by an assignment.