r"string" b"string" u"string" Python 2 / 3 comparison
Question:
I already know r"string" in Python 2.7 often used for regex patterns. I also have seen u"string" for, I think, Unicode strings. Now with Python 3 we see b"string".
I have searched for these in different sources / questions, such as What does a b prefix before a python string mean?, but it’s difficult to see the big picture of all these strings with prefixes in Python, especially with Python 2 vs 3.
Question: would you have a rule of thumb to remember the different types of strings with prefixes in Python? (or maybe a table with a column for Python 2 and one for Python 3?)
NB: I have read a few questions+answers but I haven’t found an easy to remember comparison with all prefixes / Python 2+3
Answers:
From the python docs for literals: https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals
Bytes literals are always prefixed with ‘b’ or ‘B’; they produce an
instance of the bytes type instead of the str type. They may only
contain ASCII characters; bytes with a numeric value of 128 or greater
must be expressed with escapes.
Both string and bytes literals may optionally be prefixed with a
letter ‘r’ or ‘R’; such strings are called raw strings and treat
backslashes as literal characters. As a result, in string literals,
‘U’ and ‘u’ escapes in raw strings are not treated specially. Given
that Python 2.x’s raw unicode literals behave differently than Python
3.x’s the ‘ur’ syntax is not supported.
and
A string literal with ‘f’ or ‘F’ in its prefix is a formatted string
literal; see Formatted string literals. The ‘f’ may be combined with
‘r’, but not with ‘b’ or ‘u’, therefore raw formatted strings are
possible, but formatted bytes literals are not.
So:
- r means raw
- b means bytes
- u means unicode
- f means format
The r and b were already available in Python 2, as such in many other languages (they are very handy sometimes).
Since the strings literals were not unicode in Python 2, the u-strings were created to offer support for internationalization. As of Python 3, u-strings are the default strings, so “…” is semantically the same as u”…”.
Finally, from those, the f-string is the only one that isn’t supported in Python 2.
-
u-strings if for unicode in python 2. Most probably you should forget this, if you’re working with modern applications — default strings in python 3 is all unicode, and if you’re migrating from python 2, you’ll most probably use from __future__ import unicode_literals, which makes [almost] the same for python 2
-
b-strings is for raw bytes — have no idea of text, rather just stream of bytes. Rarely used as input for your source, most often as result of network or low-level code — reading data in binary format, unpacking archives, working with encryption libraries.
Moving from/to b-string to str done via
# python 3
>>> 'hēllö'.encode('utf-8')
b'hxc4x93llxc3xb6'
>>> b'hxc4x93llxc3xb6'.decode()
'hēllö'
# python 2 without __future__
>>> u'hēllö'.encode('utf-8')
'hxc4x93llxc3xb6'
>>> 'hxc4x93llxc3xb6'.decode('utf-8')
u'hu0113llxf6' # this is correct representation
-
r-strings is not specifically for regex, this is "raw" string. Unlike regular string literals, r-string doesn’t give any special meaning for escape characters. I.e. normal string 'abcn' is 4 characters long, last char is "newline" special character. To provide it in literal, we’re using escaping with r'abcn' is 5-length string, last two characters is literally n. Two places to see raw strings often:
-
regex patterns — to not mess escaping with actual special characters in patters
-
file path notations for windows systems, as windows family uses 'C:\dir\file', or '\\share\dir', while raw would be nicer: r'C:dirfile' and r'\sharedir' respectively
- One more notable is f-strings, which came to life with python 3.6 as simple and powerful way of formatting strings:
f'a equals {a} and b is {b}' will substitute variables a and b in runtime.
There are really only two types of string (or string-like object) in Python.
The first is ‘Unicode’ strings, which are a sequence of characters.
The second is bytes (or ‘bytestrings’), which are a sequence of bytes.
The first is a series of letter characters found in the Unicode specification.
The second is a series of integers between 0 and 255 that are usually rendered to text using some assumed encoding such as ASCII or UTF-8 (which is a specification for encoding Unicode characters in a bytestream).
In Python 2, the default "my string" is a bytestring.
The prefix ‘u’ indicates a ‘Unicode’ string, e.g. u"my string".
In Python 3, ‘Unicode’ strings became the default, and thus "my string" is equivalent to u"my string".
To get the old Python 2 bytestrings, you use the prefix b"my string" (not in the oldest versions of Python 3).
There are two further prefixes, but they do not affect the type of string object, just the way it is interpreted.
The first is ‘raw’ strings which do not interpret escape characters such as n or t. For example, the raw string r"my_stringn" contains the literal backslash and ‘n’ character, while "my_stringn" contains a linebreak at the end of the line.
The second was introduced in the newest versions of Python 3: formatted strings with the prefix ‘f’. In these, curly braces are used to show expressions to be interpreted. For example, the string in:
my_object = 'avocado'
f"my {0.5 + 1.0, my_object} string"
will be interpreted to "my (1.5, avocado) string" (where the comma created a tuple). This interpretation happens immediately when the code is read; there is nothing special subsequently about the string.
And finally, you can use the multiline string notation:
"""this is my
multiline
string"""
with ‘r’ or ‘f’ specifiers as you wish.
In Python 2, if you have used no prefix or only an ‘r’ prefix, it is a bytestring, and if you have used a ‘u’ prefix it is a Unicode string.
In Python 3, if you have used no prefix or only a combination of ‘r’, ‘f’ and ‘u’, it is a Unicode string. If you have used a ‘b’ prefix it is a bytestring. Using both ‘b’ and ‘u’ is obviously not allowed.
This is what I observed (seems confirmed by other answers):
Python 2 Python 3
-----------------------------------------------
"hello" b"hello"
b"hello" <=>
<type 'str'> <class 'bytes'>
-----------------------------------------------
u"hello" <=> "hello"
u"hello"
<type 'unicode'> <class 'str'>
I already know r"string" in Python 2.7 often used for regex patterns. I also have seen u"string" for, I think, Unicode strings. Now with Python 3 we see b"string".
I have searched for these in different sources / questions, such as What does a b prefix before a python string mean?, but it’s difficult to see the big picture of all these strings with prefixes in Python, especially with Python 2 vs 3.
Question: would you have a rule of thumb to remember the different types of strings with prefixes in Python? (or maybe a table with a column for Python 2 and one for Python 3?)
NB: I have read a few questions+answers but I haven’t found an easy to remember comparison with all prefixes / Python 2+3
From the python docs for literals: https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals
Bytes literals are always prefixed with ‘b’ or ‘B’; they produce an
instance of the bytes type instead of the str type. They may only
contain ASCII characters; bytes with a numeric value of 128 or greater
must be expressed with escapes.Both string and bytes literals may optionally be prefixed with a
letter ‘r’ or ‘R’; such strings are called raw strings and treat
backslashes as literal characters. As a result, in string literals,
‘U’ and ‘u’ escapes in raw strings are not treated specially. Given
that Python 2.x’s raw unicode literals behave differently than Python
3.x’s the ‘ur’ syntax is not supported.
and
A string literal with ‘f’ or ‘F’ in its prefix is a formatted string
literal; see Formatted string literals. The ‘f’ may be combined with
‘r’, but not with ‘b’ or ‘u’, therefore raw formatted strings are
possible, but formatted bytes literals are not.
So:
- r means raw
- b means bytes
- u means unicode
- f means format
The r and b were already available in Python 2, as such in many other languages (they are very handy sometimes).
Since the strings literals were not unicode in Python 2, the u-strings were created to offer support for internationalization. As of Python 3, u-strings are the default strings, so “…” is semantically the same as u”…”.
Finally, from those, the f-string is the only one that isn’t supported in Python 2.
-
u-strings if for unicode in python 2. Most probably you should forget this, if you’re working with modern applications — default strings in python 3 is all unicode, and if you’re migrating from python 2, you’ll most probably use
from __future__ import unicode_literals, which makes [almost] the same for python 2 -
b-strings is for raw bytes — have no idea of text, rather just stream of bytes. Rarely used as input for your source, most often as result of network or low-level code — reading data in binary format, unpacking archives, working with encryption libraries.
Moving from/to b-string to
strdone via# python 3 >>> 'hēllö'.encode('utf-8') b'hxc4x93llxc3xb6' >>> b'hxc4x93llxc3xb6'.decode() 'hēllö' # python 2 without __future__ >>> u'hēllö'.encode('utf-8') 'hxc4x93llxc3xb6' >>> 'hxc4x93llxc3xb6'.decode('utf-8') u'hu0113llxf6' # this is correct representation -
r-strings is not specifically for regex, this is "raw" string. Unlike regular string literals, r-string doesn’t give any special meaning for escape characters. I.e. normal string
'abcn'is 4 characters long, last char is "newline" special character. To provide it in literal, we’re using escaping with. For raw strings,r'abcn'is 5-length string, last two characters is literallyandn. Two places to see raw strings often:
-
regex patterns — to not mess escaping with actual special characters in patters
-
file path notations for windows systems, as windows family uses
as delimeter, normal string literals will look like'C:\dir\file', or'\\share\dir', while raw would be nicer:r'C:dirfile'andr'\sharedir'respectively
- One more notable is f-strings, which came to life with python 3.6 as simple and powerful way of formatting strings:
f'a equals {a} and b is {b}'will substitute variablesaandbin runtime.
There are really only two types of string (or string-like object) in Python.
The first is ‘Unicode’ strings, which are a sequence of characters.
The second is bytes (or ‘bytestrings’), which are a sequence of bytes.
The first is a series of letter characters found in the Unicode specification.
The second is a series of integers between 0 and 255 that are usually rendered to text using some assumed encoding such as ASCII or UTF-8 (which is a specification for encoding Unicode characters in a bytestream).
In Python 2, the default "my string" is a bytestring.
The prefix ‘u’ indicates a ‘Unicode’ string, e.g. u"my string".
In Python 3, ‘Unicode’ strings became the default, and thus "my string" is equivalent to u"my string".
To get the old Python 2 bytestrings, you use the prefix b"my string" (not in the oldest versions of Python 3).
There are two further prefixes, but they do not affect the type of string object, just the way it is interpreted.
The first is ‘raw’ strings which do not interpret escape characters such as n or t. For example, the raw string r"my_stringn" contains the literal backslash and ‘n’ character, while "my_stringn" contains a linebreak at the end of the line.
The second was introduced in the newest versions of Python 3: formatted strings with the prefix ‘f’. In these, curly braces are used to show expressions to be interpreted. For example, the string in:
my_object = 'avocado'
f"my {0.5 + 1.0, my_object} string"
will be interpreted to "my (1.5, avocado) string" (where the comma created a tuple). This interpretation happens immediately when the code is read; there is nothing special subsequently about the string.
And finally, you can use the multiline string notation:
"""this is my
multiline
string"""
with ‘r’ or ‘f’ specifiers as you wish.
In Python 2, if you have used no prefix or only an ‘r’ prefix, it is a bytestring, and if you have used a ‘u’ prefix it is a Unicode string.
In Python 3, if you have used no prefix or only a combination of ‘r’, ‘f’ and ‘u’, it is a Unicode string. If you have used a ‘b’ prefix it is a bytestring. Using both ‘b’ and ‘u’ is obviously not allowed.
This is what I observed (seems confirmed by other answers):
Python 2 Python 3
-----------------------------------------------
"hello" b"hello"
b"hello" <=>
<type 'str'> <class 'bytes'>
-----------------------------------------------
u"hello" <=> "hello"
u"hello"
<type 'unicode'> <class 'str'>