UnicodeDecodeError : 'ascii' codec can't decode byte 0xe0 in position 0: ordinal not in range(128)
Question:
On one of my machines I have the error when I am working with google apps engine or django.
For example:
-
app.yaml
application: demas1252c
version: 1
runtime: python
api_version: 1
handlers:
- url: /images
static_dir: images
- url: /css
static_dir: css
- url: /js
static_dir: js
- url: /.*
script: demas1252c.py
-
demas1252c.py
import cgi
import wsgiref.handlers
from google.appengine.ext.webapp import template
from google.appengine.ext import webapp
class MainPage(webapp.RequestHandler):
def get(self):
values = {'id' : 10}
self.response.out.write(template.render('foto.html', values))
application = webapp.WSGIApplication([('/', MainPage)], debug = True)
wsgiref.handlers.CGIHandler().run(application)
-
foto.html
<!DOCTYPE html>
<html lang="en">
<head></head>
<body>some</body>
</html>
error message:
C:artefactsdevproject>"c:Program FilesGooglegoogle_appenginedev_appserver.py" foto-hosting
Traceback (most recent call last):
File "c:Program FilesGooglegoogle_appenginedev_appserver.py", line 69, in <module>
run_file(__file__, globals())
File "c:Program FilesGooglegoogle_appenginedev_appserver.py", line 65, in run_file
execfile(script_path, globals_)
File "c:Program FilesGooglegoogle_appenginegoogleappenginetoolsdev_appserver_main.py", line 92, in <module>
from google.appengine.tools import dev_appserver
File "c:Program FilesGooglegoogle_appenginegoogleappenginetoolsdev_appserver.py", line 140, in <module>
mimetypes.add_type(mime_type, '.' + ext)
File "C:Python27libmimetypes.py", line 344, in add_type
init()
File "C:Python27libmimetypes.py", line 355, in init
db.read_windows_registry()
File "C:Python27libmimetypes.py", line 260, in read_windows_registry
for ctype in enum_types(mimedb):
File "C:Python27libmimetypes.py", line 250, in enum_types
ctype = ctype.encode(default_encoding) # omit in 3.x!
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe0 in position 0: ordinal not in range(128)
When I am working with static files in django (without gae) I have very similar error (with different stack).
I tried to find the reason of error and added code to mimetypes.py:
print '====='
print ctype
ctype = ctype.encode(default_encoding) # omit in 3.x!
Then I get next messages in my console:
=====
video/x-ms-wvx
=====
video/x-msvideo
=====
рєфшю/AMR
Traceback (most recent call last):
In the registry HKCR/Mime/Database/ContentType/ I have five keys with russian (cyrilic) letters. But how can I fix this error?
Answers:
This is a bug in mimetypes, triggered by bad data in the registry. (рєфшю/AMR is not at all a valid MIME media type.)
ctype is a registry key name returned by _winreg.EnumKey, which mimetypes is expecting to be a Unicode string, but it isn’t. Unlike _winreg.QueryValueEx, EnumKey returns a byte string (direct from the ANSI version of the Windows API; _winreg in Python 2 doesn’t use the Unicode interfaces even though it returns Unicode strings, so it’ll never read non-ANSI characters correctly).
So the attempt to .encode it fails with a UnicodeDecodeError trying to get a Unicode string before encoding it back to ASCII!
try:
ctype = ctype.encode(default_encoding) # omit in 3.x!
except UnicodeEncodeError:
pass
These lines in mimetypes should simply be removed.
ETA: added to bug tracker.
its a python bug with latin MIME tipes in registry
start regedit and inspect “HKEY_CLASSES_ROOTMIMEDatabaseContent Type” for non-latin names.
By the way, the main culpit of the problem is QuickTime which adds non-ascii mime types to the windows registry. The easiest way to fix it is to manually find and remove from the registry the subsections of the HKCR/Mime/Database/ContentType/ starting with аудио/ and видео/.
There is a patch:
http://bugs.python.org/file18143/9291.patch
Works great for me.
Just replace UnicodeEncodeError to be UnicodeError
An alternative solution from python issue9291 by Alexandr Zarubkin (me21):
add file named sitecustomize.py in Libsite-packages folder.
import sys
sys.setdefaultencoding("cp1251")
On one of my machines I have the error when I am working with google apps engine or django.
For example:
-
app.yaml
application: demas1252c version: 1 runtime: python api_version: 1 handlers: - url: /images static_dir: images - url: /css static_dir: css - url: /js static_dir: js - url: /.* script: demas1252c.py -
demas1252c.py
import cgi import wsgiref.handlers from google.appengine.ext.webapp import template from google.appengine.ext import webapp class MainPage(webapp.RequestHandler): def get(self): values = {'id' : 10} self.response.out.write(template.render('foto.html', values)) application = webapp.WSGIApplication([('/', MainPage)], debug = True) wsgiref.handlers.CGIHandler().run(application) -
foto.html
<!DOCTYPE html> <html lang="en"> <head></head> <body>some</body> </html>
error message:
C:artefactsdevproject>"c:Program FilesGooglegoogle_appenginedev_appserver.py" foto-hosting
Traceback (most recent call last):
File "c:Program FilesGooglegoogle_appenginedev_appserver.py", line 69, in <module>
run_file(__file__, globals())
File "c:Program FilesGooglegoogle_appenginedev_appserver.py", line 65, in run_file
execfile(script_path, globals_)
File "c:Program FilesGooglegoogle_appenginegoogleappenginetoolsdev_appserver_main.py", line 92, in <module>
from google.appengine.tools import dev_appserver
File "c:Program FilesGooglegoogle_appenginegoogleappenginetoolsdev_appserver.py", line 140, in <module>
mimetypes.add_type(mime_type, '.' + ext)
File "C:Python27libmimetypes.py", line 344, in add_type
init()
File "C:Python27libmimetypes.py", line 355, in init
db.read_windows_registry()
File "C:Python27libmimetypes.py", line 260, in read_windows_registry
for ctype in enum_types(mimedb):
File "C:Python27libmimetypes.py", line 250, in enum_types
ctype = ctype.encode(default_encoding) # omit in 3.x!
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe0 in position 0: ordinal not in range(128)
When I am working with static files in django (without gae) I have very similar error (with different stack).
I tried to find the reason of error and added code to mimetypes.py:
print '====='
print ctype
ctype = ctype.encode(default_encoding) # omit in 3.x!
Then I get next messages in my console:
=====
video/x-ms-wvx
=====
video/x-msvideo
=====
рєфшю/AMR
Traceback (most recent call last):
In the registry HKCR/Mime/Database/ContentType/ I have five keys with russian (cyrilic) letters. But how can I fix this error?
This is a bug in mimetypes, triggered by bad data in the registry. (рєфшю/AMR is not at all a valid MIME media type.)
ctype is a registry key name returned by _winreg.EnumKey, which mimetypes is expecting to be a Unicode string, but it isn’t. Unlike _winreg.QueryValueEx, EnumKey returns a byte string (direct from the ANSI version of the Windows API; _winreg in Python 2 doesn’t use the Unicode interfaces even though it returns Unicode strings, so it’ll never read non-ANSI characters correctly).
So the attempt to .encode it fails with a UnicodeDecodeError trying to get a Unicode string before encoding it back to ASCII!
try:
ctype = ctype.encode(default_encoding) # omit in 3.x!
except UnicodeEncodeError:
pass
These lines in mimetypes should simply be removed.
ETA: added to bug tracker.
its a python bug with latin MIME tipes in registry
start regedit and inspect “HKEY_CLASSES_ROOTMIMEDatabaseContent Type” for non-latin names.
By the way, the main culpit of the problem is QuickTime which adds non-ascii mime types to the windows registry. The easiest way to fix it is to manually find and remove from the registry the subsections of the HKCR/Mime/Database/ContentType/ starting with аудио/ and видео/.
There is a patch:
http://bugs.python.org/file18143/9291.patch
Works great for me.
Just replace UnicodeEncodeError to be UnicodeError
An alternative solution from python issue9291 by Alexandr Zarubkin (me21):
add file named sitecustomize.py in Libsite-packages folder.
import sys
sys.setdefaultencoding("cp1251")