Why can tuples contain mutable items?
Question:
If a tuple is immutable then why can it contain mutable items?
It is seemingly a contradiction that when a mutable item such as a list does get modified, the tuple it belongs to maintains being immutable.
Answers:
That’s an excellent question.
The key insight is that tuples have no way of knowing whether the objects inside them are mutable. The only thing that makes an object mutable is to have a method that alters its data. In general, there is no way to detect this.
Another insight is that Python’s containers don’t actually contain anything. Instead, they keep references to other objects. Likewise, Python’s variables aren’t like variables in compiled languages; instead the variable names are just keys in a namespace dictionary where they are associated with a corresponding object. Ned Batchhelder explains this nicely in his blog post. Either way, objects only know their reference count; they don’t know what those references are (variables, containers, or the Python internals).
Together, these two insights explain your mystery (why an immutable tuple “containing” a list seems to change when the underlying list changes). In fact, the tuple did not change (it still has the same references to other objects that it did before). The tuple could not change (because it did not have mutating methods). When the list changed, the tuple didn’t get notified of the change (the list doesn’t know whether it is referred to by a variable, a tuple, or another list).
While we’re on the topic, here are a few other thoughts to help complete your mental model of what tuples are, how they work, and their intended use:
-
Tuples are characterized less by their immutability and more by their intended purpose.
Tuples are Python’s way of collecting heterogeneous pieces of information under one roof. For example,
s = ('www.python.org', 80)
brings together a string and a number so that the host/port pair can be passed around as a socket, a composite object. Viewed in that light, it is perfectly reasonable to have mutable components.
-
Immutability goes hand-in-hand with another property, hashability. But hashability isn’t an absolute property. If one of the tuple’s components isn’t hashable, then the overall tuple isn’t hashable either. For example, t = ('red', [10, 20, 30]) isn’t hashable.
The last example shows a 2-tuple that contains a string and a list. The tuple itself isn’t mutable (i.e. it doesn’t have any methods that for changing its contents). Likewise, the string is immutable because strings don’t have any mutating methods. The list object does have mutating methods, so it can be changed. This shows that mutability is a property of an object type — some objects have mutating methods and some don’t. This doesn’t change just because the objects are nested.
Remember two things. First, immutability is not magic — it is merely the absence of mutating methods. Second, objects don’t know what variables or containers refer to them — they only know the reference count.
Hope, this was useful to you 🙂
I’ll go out on a limb here and say that the relevant part here is that while you can change the contents of a list, or the state of an object, contained within a tuple, what you can’t change is that the object or list is there. If you had something that depended on thing[3] being a list, even if empty, then I could see this being useful.
A tuple is immutable in the sense that the tuple itself can not expand or shrink, not that all the items contained themselves are immutable. Otherwise tuples are dull.
That’s because tuples don’t contain lists, strings or numbers. They contain references to other objects.1 The inability to change the sequence of references a tuple contains doesn’t mean that you can’t mutate the objects associated with those references.2
1. Objects, values and types (see: second to last paragraph)
2. The standard type hierarchy (see: “Immutable sequences”)
You cannot change the id of its items. So it will always contain the same items.
$ python
>>> t = (1, [2, 3])
>>> id(t[1])
12371368
>>> t[1].append(4)
>>> id(t[1])
12371368
First of all, the word "immutable" can mean many different things to different people. I particularly like how Eric Lippert categorized immutability in his blog post [archive 2012-03-12]. There, he lists these kinds of immutability:
- Realio-trulio immutability
- Write-once immutability
- Popsicle immutability
- Shallow vs deep immutability
- Immutable facades
- Observational immutability
These can be combined in various ways to make even more kinds of immutability, and I’m sure more exist. The kind of immutability you seems interested in deep (also known as transitive) immutability, in which immutable objects can only contain other immutable objects.
The key point of this is that deep immutability is only one of many, many kinds of immutability. You can adopt whichever kind you prefer, as long as you are aware that your notion of "immutable" probably differs from someone else’s notion of "immutable".
As I understand it, this question needs to be rephrased as a question about design decisions: Why did the designers of Python choose to create an immutable sequence type that can contain mutable objects?
To answer this question, we have to think about the purpose tuples serve: they serve as fast, general-purpose sequences. With that in mind, it becomes quite obvious why tuples are immutable but can contain mutable objects. To wit:
-
Tuples are fast and memory efficient: Tuples are faster to create than lists because they are immutable. Immutability means that tuples can be created as constants and loaded as such, using constant folding. It also means they’re faster and more memory efficient to create because there’s no need for overallocation, etc. They’re a bit slower than lists for random item access, but faster again for unpacking (at least on my machine). If tuples were mutable, then they wouldn’t be as fast for purposes such as these.
-
Tuples are general-purpose: Tuples need to be able to contain any kind of object. They’re used to (quickly) do things like variable-length argument lists (via the * operator in function definitions). If tuples couldn’t hold mutable objects, they would be useless for things like this. Python would have to use lists, which would probably slow things down, and would certainly be less memory efficient.
So you see, in order to fulfill their purpose, tuples must be immutable, but also must be able to contain mutable objects. If the designers of Python wanted to create an immutable object that guarantees that all the objects it “contains” are also immutable, they would have to create a third sequence type. The gain is not worth the extra complexity.
One reason is that there is no general way in Python to convert a mutable type into an immutable one (see the rejected PEP 351, and the linked discussion for why it was rejected). Thus, it would be impossible to put various types of objects in tuples if it had this restriction, including just about any user-created non-hashable object.
The only reason that dictionaries and sets have this restriction is that they require the objects to be hashable, since they are internally implemented as hash tables. But note that, ironically, dictionaries and sets themselves are not immutable (or hashable). Tuples do not use an object’s hash, so its mutability does not matter.
If a tuple is immutable then why can it contain mutable items?
It is seemingly a contradiction that when a mutable item such as a list does get modified, the tuple it belongs to maintains being immutable.
That’s an excellent question.
The key insight is that tuples have no way of knowing whether the objects inside them are mutable. The only thing that makes an object mutable is to have a method that alters its data. In general, there is no way to detect this.
Another insight is that Python’s containers don’t actually contain anything. Instead, they keep references to other objects. Likewise, Python’s variables aren’t like variables in compiled languages; instead the variable names are just keys in a namespace dictionary where they are associated with a corresponding object. Ned Batchhelder explains this nicely in his blog post. Either way, objects only know their reference count; they don’t know what those references are (variables, containers, or the Python internals).
Together, these two insights explain your mystery (why an immutable tuple “containing” a list seems to change when the underlying list changes). In fact, the tuple did not change (it still has the same references to other objects that it did before). The tuple could not change (because it did not have mutating methods). When the list changed, the tuple didn’t get notified of the change (the list doesn’t know whether it is referred to by a variable, a tuple, or another list).
While we’re on the topic, here are a few other thoughts to help complete your mental model of what tuples are, how they work, and their intended use:
-
Tuples are characterized less by their immutability and more by their intended purpose.
Tuples are Python’s way of collecting heterogeneous pieces of information under one roof. For example,
s = ('www.python.org', 80)
brings together a string and a number so that the host/port pair can be passed around as a socket, a composite object. Viewed in that light, it is perfectly reasonable to have mutable components. -
Immutability goes hand-in-hand with another property, hashability. But hashability isn’t an absolute property. If one of the tuple’s components isn’t hashable, then the overall tuple isn’t hashable either. For example,
t = ('red', [10, 20, 30])isn’t hashable.
The last example shows a 2-tuple that contains a string and a list. The tuple itself isn’t mutable (i.e. it doesn’t have any methods that for changing its contents). Likewise, the string is immutable because strings don’t have any mutating methods. The list object does have mutating methods, so it can be changed. This shows that mutability is a property of an object type — some objects have mutating methods and some don’t. This doesn’t change just because the objects are nested.
Remember two things. First, immutability is not magic — it is merely the absence of mutating methods. Second, objects don’t know what variables or containers refer to them — they only know the reference count.
Hope, this was useful to you 🙂
I’ll go out on a limb here and say that the relevant part here is that while you can change the contents of a list, or the state of an object, contained within a tuple, what you can’t change is that the object or list is there. If you had something that depended on thing[3] being a list, even if empty, then I could see this being useful.
A tuple is immutable in the sense that the tuple itself can not expand or shrink, not that all the items contained themselves are immutable. Otherwise tuples are dull.
That’s because tuples don’t contain lists, strings or numbers. They contain references to other objects.1 The inability to change the sequence of references a tuple contains doesn’t mean that you can’t mutate the objects associated with those references.2
1. Objects, values and types (see: second to last paragraph)
2. The standard type hierarchy (see: “Immutable sequences”)
You cannot change the id of its items. So it will always contain the same items.
$ python
>>> t = (1, [2, 3])
>>> id(t[1])
12371368
>>> t[1].append(4)
>>> id(t[1])
12371368
First of all, the word "immutable" can mean many different things to different people. I particularly like how Eric Lippert categorized immutability in his blog post [archive 2012-03-12]. There, he lists these kinds of immutability:
- Realio-trulio immutability
- Write-once immutability
- Popsicle immutability
- Shallow vs deep immutability
- Immutable facades
- Observational immutability
These can be combined in various ways to make even more kinds of immutability, and I’m sure more exist. The kind of immutability you seems interested in deep (also known as transitive) immutability, in which immutable objects can only contain other immutable objects.
The key point of this is that deep immutability is only one of many, many kinds of immutability. You can adopt whichever kind you prefer, as long as you are aware that your notion of "immutable" probably differs from someone else’s notion of "immutable".
As I understand it, this question needs to be rephrased as a question about design decisions: Why did the designers of Python choose to create an immutable sequence type that can contain mutable objects?
To answer this question, we have to think about the purpose tuples serve: they serve as fast, general-purpose sequences. With that in mind, it becomes quite obvious why tuples are immutable but can contain mutable objects. To wit:
-
Tuples are fast and memory efficient: Tuples are faster to create than lists because they are immutable. Immutability means that tuples can be created as constants and loaded as such, using constant folding. It also means they’re faster and more memory efficient to create because there’s no need for overallocation, etc. They’re a bit slower than lists for random item access, but faster again for unpacking (at least on my machine). If tuples were mutable, then they wouldn’t be as fast for purposes such as these.
-
Tuples are general-purpose: Tuples need to be able to contain any kind of object. They’re used to (quickly) do things like variable-length argument lists (via the
*operator in function definitions). If tuples couldn’t hold mutable objects, they would be useless for things like this. Python would have to use lists, which would probably slow things down, and would certainly be less memory efficient.
So you see, in order to fulfill their purpose, tuples must be immutable, but also must be able to contain mutable objects. If the designers of Python wanted to create an immutable object that guarantees that all the objects it “contains” are also immutable, they would have to create a third sequence type. The gain is not worth the extra complexity.
One reason is that there is no general way in Python to convert a mutable type into an immutable one (see the rejected PEP 351, and the linked discussion for why it was rejected). Thus, it would be impossible to put various types of objects in tuples if it had this restriction, including just about any user-created non-hashable object.
The only reason that dictionaries and sets have this restriction is that they require the objects to be hashable, since they are internally implemented as hash tables. But note that, ironically, dictionaries and sets themselves are not immutable (or hashable). Tuples do not use an object’s hash, so its mutability does not matter.