When to use utf8 as a header in py files
Question:
Some source files, from downloaded code, have the following header
# -*- coding: utf-8 -*-
I have an idea what utf-8 encoding is but why would it be needed as a header in a python source file?
Answers:
wherever you need to use in your code chars that aren’t from ascii, like:
ă
interpreter will complain that he doesn’t understand that char.
Usually this happens when you define constants.
Example:
Add into x.py
print 'ă'
then start a python console
import x
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "x.py", line 1
SyntaxError: Non-ASCII character 'xc4' in file x.py on line 1,
but no encoding declared;
see http://www.python.org/peps/pep-0263.html for details
When you use non-ascii characters. For instance when I comment my source in norwegian if charachters ØÆÅ occur in the .py it will complain and not “compile”.
Whenever text is read or written, encodings come in play. Always. A python interpreter has to read your file as text, to understand it. The only situation where you could get away without having to deal with encodings is when you only use characters in the ASCII range. The interpreter can in this case use virtually any encoding in the world, and get it right because almost all encodings encode these characters to same bytes.
You should not use coding: utf-8 just because you have characters beyond ascii in your file, it can even be harmful. It is a hint for the python interpreter, to tell it what encoding your file is in. Unless you have configured your text editor, the text editor will most likely not save your files in utf-8. So now the hint you gave to the python interpreter, is wrong.
So you should use it when your file is encoded in utf-8. If it’s encoded in windows-1252, you should use coding: windows-1252 and so on.
Always use UTF-8 and make sure your editor also uses UTF-8.
Start your Python script like this if you use Python 27:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
This is a good blog post from Nick Johnson about Python and UTF-8:
http://blog.notdot.net/2010/07/Getting-unicode-right-in-Python
By the way, this post was written before he could use:
from __future__ import unicode_literals
A more direct answer:
In Python 3+: you don’t need to declare any encoding.
UTF-8 is the default. Make sure the file is encoded in UTF-8 without BOM. Some Windows editors don’t have it by default. It won’t hurt to declare it, and some editors may use it.
In Python 2: always.
The default is OS dependent. As first or second line line of your files put this comment:
# -*- coding: utf-8 -*-
And remember: this is just about your source code files. Now in the 3rd millennium the string type does not exist anymore. You must take care of the type text, that is a sequence of bytes and an encoding. You’ll still have to define your encoding in all input and output operation. These operations will still be dependent on your environment, so it’s still better to follow the rule: Explicit is better than implicit.
Some source files, from downloaded code, have the following header
# -*- coding: utf-8 -*-
I have an idea what utf-8 encoding is but why would it be needed as a header in a python source file?
wherever you need to use in your code chars that aren’t from ascii, like:
ă
interpreter will complain that he doesn’t understand that char.
Usually this happens when you define constants.
Example:
Add into x.py
print 'ă'
then start a python console
import x
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "x.py", line 1
SyntaxError: Non-ASCII character 'xc4' in file x.py on line 1,
but no encoding declared;
see http://www.python.org/peps/pep-0263.html for details
When you use non-ascii characters. For instance when I comment my source in norwegian if charachters ØÆÅ occur in the .py it will complain and not “compile”.
Whenever text is read or written, encodings come in play. Always. A python interpreter has to read your file as text, to understand it. The only situation where you could get away without having to deal with encodings is when you only use characters in the ASCII range. The interpreter can in this case use virtually any encoding in the world, and get it right because almost all encodings encode these characters to same bytes.
You should not use coding: utf-8 just because you have characters beyond ascii in your file, it can even be harmful. It is a hint for the python interpreter, to tell it what encoding your file is in. Unless you have configured your text editor, the text editor will most likely not save your files in utf-8. So now the hint you gave to the python interpreter, is wrong.
So you should use it when your file is encoded in utf-8. If it’s encoded in windows-1252, you should use coding: windows-1252 and so on.
Always use UTF-8 and make sure your editor also uses UTF-8.
Start your Python script like this if you use Python 27:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
This is a good blog post from Nick Johnson about Python and UTF-8:
http://blog.notdot.net/2010/07/Getting-unicode-right-in-Python
By the way, this post was written before he could use:
from __future__ import unicode_literals
A more direct answer:
In Python 3+: you don’t need to declare any encoding.
UTF-8 is the default. Make sure the file is encoded in UTF-8 without BOM. Some Windows editors don’t have it by default. It won’t hurt to declare it, and some editors may use it.
In Python 2: always.
The default is OS dependent. As first or second line line of your files put this comment:
# -*- coding: utf-8 -*-
And remember: this is just about your source code files. Now in the 3rd millennium the string type does not exist anymore. You must take care of the type text, that is a sequence of bytes and an encoding. You’ll still have to define your encoding in all input and output operation. These operations will still be dependent on your environment, so it’s still better to follow the rule: Explicit is better than implicit.