Why are Python strings immutable? Best practices for using them
Question:
- What are the design reasons of making Python strings immutable? How does it make programming easier?
- I’m used to mutable strings, like the ones in C. How am I supposed to program without mutable strings? Are there any best practices?
Answers:
Immutable strings makes programming much easier, which is why C# and Java use them too.
Had strings been mutable, you would not be able to trust any externally-provided string, since a malicious caller could change it underneath you.
It would also make multi-threading much more difficult.
Most languages have immutable strings. This includes Java, Python, and C#. Usually when concatenating strings, the language allocates an entirely new string and copies the content of the two strings into the new string.
Immutability does tend to make programming easier. Especially when dealing with a multi-threaded environment.
- Immutable objects are automatically threadsafe.
- You will find that using Python strings is trivially easy in comparison to the extreme pain associated with strings in C.
When you receive a string, you’ll be sure that it stays the same. Suppose that you’d construct a Foo as below with a string argument, and would then modify the string; then the Foo‘s name would suddenly change:
class Foo(object):
def __init__(self, name):
self.name = name
name = "Hello"
foo = Foo(name)
name[0] = "J"
With mutable strings, you’d have to make copies all the time to prevent bad things from happening.
It also allows the convenience that a single character is no different from a string of length one, so all string operators apply to characters as well.
And lastly, if strings weren’t immutable, you couldn’t reliably use them as keys in a dict, since their hash value might suddenly change.
As for programming with immutable strings, just get used to treating them the same way you treat numbers: as values, not as objects. Changing the first letter of name would be
name = "J" + name[1:]
Immutable strings greatly simplify memory allocation when compared with C strings: you don’t guess at a length and over-allocate hoping you over-allocated enough.
They’re more secure: you can never have a buffer overrun the way you can in C.
There is only one mutable string use case.
- replacing a substring or a single character
All other string use cases (concatenation, searching, etc., etc.) the mutability does not matter. In all other cases, mutability does not matter.
If you want to replace a character or a substring in Python, you simply create a new string
x = x[:place] + replacement + x[place+1:]
That’s the only code that novel or distinctive.
For reasons I fail to understand, it appears important to add the following.
“There are other ways to avoid a string buffer overflow than immutable strings.”
For the purposes of this question (about Python, specifically) immutable strings have a pleasant consequence of no buffer overflows. For other languages, other principles, rules and nuances apply.
Immutable strings can be keys in dictionaries and similar data structures, without the need to copy the strings. It is easier to make a mutable wrapper around an immutable string than the other way around.
Immutable strings are much more dangerous than mutable strings in certain contexts. A best practice list should include never temporarily storing passwords or keys as strings in python.
While it is great that immutable strings are safer in string operations and offer other consistency advantages, storing (even temporarily) a password or goodness forbid a key: This puts that value in memory for the life of that program. This may end up inside a core dump file (that is improperly permissioned). If a virtual machine running the program is paused, v-motioned to a different physical machine… this allows the data in memory in this program to leak to people outside the group of users and administrators of the solution.
Due to our massively convenient/complex virtualization stack we’ve built, you need to do your best to avoid accepting passwords or storing keys in memory. There is precious little information on how to do this, but it’s best to follow solutions that use OS native mechanisms designed to perform these operations properly including openssl, ssh, keyring, etc.
- What are the design reasons of making Python strings immutable? How does it make programming easier?
- I’m used to mutable strings, like the ones in C. How am I supposed to program without mutable strings? Are there any best practices?
Immutable strings makes programming much easier, which is why C# and Java use them too.
Had strings been mutable, you would not be able to trust any externally-provided string, since a malicious caller could change it underneath you.
It would also make multi-threading much more difficult.
Most languages have immutable strings. This includes Java, Python, and C#. Usually when concatenating strings, the language allocates an entirely new string and copies the content of the two strings into the new string.
Immutability does tend to make programming easier. Especially when dealing with a multi-threaded environment.
- Immutable objects are automatically threadsafe.
- You will find that using Python strings is trivially easy in comparison to the extreme pain associated with strings in C.
When you receive a string, you’ll be sure that it stays the same. Suppose that you’d construct a Foo as below with a string argument, and would then modify the string; then the Foo‘s name would suddenly change:
class Foo(object):
def __init__(self, name):
self.name = name
name = "Hello"
foo = Foo(name)
name[0] = "J"
With mutable strings, you’d have to make copies all the time to prevent bad things from happening.
It also allows the convenience that a single character is no different from a string of length one, so all string operators apply to characters as well.
And lastly, if strings weren’t immutable, you couldn’t reliably use them as keys in a dict, since their hash value might suddenly change.
As for programming with immutable strings, just get used to treating them the same way you treat numbers: as values, not as objects. Changing the first letter of name would be
name = "J" + name[1:]
Immutable strings greatly simplify memory allocation when compared with C strings: you don’t guess at a length and over-allocate hoping you over-allocated enough.
They’re more secure: you can never have a buffer overrun the way you can in C.
There is only one mutable string use case.
- replacing a substring or a single character
All other string use cases (concatenation, searching, etc., etc.) the mutability does not matter. In all other cases, mutability does not matter.
If you want to replace a character or a substring in Python, you simply create a new string
x = x[:place] + replacement + x[place+1:]
That’s the only code that novel or distinctive.
For reasons I fail to understand, it appears important to add the following.
“There are other ways to avoid a string buffer overflow than immutable strings.”
For the purposes of this question (about Python, specifically) immutable strings have a pleasant consequence of no buffer overflows. For other languages, other principles, rules and nuances apply.
Immutable strings can be keys in dictionaries and similar data structures, without the need to copy the strings. It is easier to make a mutable wrapper around an immutable string than the other way around.
Immutable strings are much more dangerous than mutable strings in certain contexts. A best practice list should include never temporarily storing passwords or keys as strings in python.
While it is great that immutable strings are safer in string operations and offer other consistency advantages, storing (even temporarily) a password or goodness forbid a key: This puts that value in memory for the life of that program. This may end up inside a core dump file (that is improperly permissioned). If a virtual machine running the program is paused, v-motioned to a different physical machine… this allows the data in memory in this program to leak to people outside the group of users and administrators of the solution.
Due to our massively convenient/complex virtualization stack we’ve built, you need to do your best to avoid accepting passwords or storing keys in memory. There is precious little information on how to do this, but it’s best to follow solutions that use OS native mechanisms designed to perform these operations properly including openssl, ssh, keyring, etc.