python jsonify dictionary in utf-8
Question:
I want to get json data into utf-8
I have a list my_list = []
and then many appends unicode values to the list like this
my_list.append(u'ტესტ')
return jsonify(result=my_list)
and it gets
{
"result": [
"u10e2u10d4u10e1u10e2",
"u10e2u10ddu10dbu10d0u10e8u10d5u10d8u10dau10d8"
]
}
Answers:
Use the standard-library json
module instead, and set the ensure_ascii
keyword parameter to False when encoding, or do the same with flask.json.dumps()
:
>>> data = u'u10e2u10d4u10e1u10e2'
>>> import json
>>> json.dumps(data)
'"\u10e2\u10d4\u10e1\u10e2"'
>>> json.dumps(data, ensure_ascii=False)
u'"u10e2u10d4u10e1u10e2"'
>>> print json.dumps(data, ensure_ascii=False)
"ტესტ"
>>> json.dumps(data, ensure_ascii=False).encode('utf8')
'"xe1x83xa2xe1x83x94xe1x83xa1xe1x83xa2"'
Note that you still need to explicitly encode the result to UTF8 because the dumps()
function returns a unicode
object in that case.
You can make this the default (and use jsonify()
again) by setting JSON_AS_ASCII
to False in your Flask app config.
WARNING: do not include untrusted data in JSON that is not ASCII-safe, and then interpolate into a HTML template or use in a JSONP API, as you can cause syntax errors or open a cross-site scripting vulnerability this way. That’s because JSON is not a strict subset of Javascript, and when disabling ASCII-safe encoding the U+2028 and U+2029 separators will not be escaped to u2028
and u2029
sequences.
Use the following config to add UTF-8 support:
app.config['JSON_AS_ASCII'] = False
If you still want to user flask’s json and ensure the utf-8 encoding then you can do something like this:
from flask import json,Response
@app.route("/")
def hello():
my_list = []
my_list.append(u'ტესტ')
data = { "result" : my_list}
json_string = json.dumps(data,ensure_ascii = False)
#creating a Response object to set the content type and the encoding
response = Response(json_string,content_type="application/json; charset=utf-8" )
return response
#I hope this helps
In my case the above solution was not sufficient. (Running flask on the GCP App Engine flexible environment). I ended up doing:
json_str = json.dumps(myDict, ensure_ascii = False, indent=4, sort_keys=True)
encoding = chardet.detect(json_str)['encoding']
json_unicode = json_str.decode(encoding)
json_utf8 = json_unicode.encode('utf-8')
response = make_response(json_utf8)
response.headers['Content-Type'] = 'application/json; charset=utf-8'
response.headers['mimetype'] = 'application/json'
response.status_code = status
I want to get json data into utf-8
I have a list my_list = []
and then many appends unicode values to the list like this
my_list.append(u'ტესტ')
return jsonify(result=my_list)
and it gets
{
"result": [
"u10e2u10d4u10e1u10e2",
"u10e2u10ddu10dbu10d0u10e8u10d5u10d8u10dau10d8"
]
}
Use the standard-library json
module instead, and set the ensure_ascii
keyword parameter to False when encoding, or do the same with flask.json.dumps()
:
>>> data = u'u10e2u10d4u10e1u10e2'
>>> import json
>>> json.dumps(data)
'"\u10e2\u10d4\u10e1\u10e2"'
>>> json.dumps(data, ensure_ascii=False)
u'"u10e2u10d4u10e1u10e2"'
>>> print json.dumps(data, ensure_ascii=False)
"ტესტ"
>>> json.dumps(data, ensure_ascii=False).encode('utf8')
'"xe1x83xa2xe1x83x94xe1x83xa1xe1x83xa2"'
Note that you still need to explicitly encode the result to UTF8 because the dumps()
function returns a unicode
object in that case.
You can make this the default (and use jsonify()
again) by setting JSON_AS_ASCII
to False in your Flask app config.
WARNING: do not include untrusted data in JSON that is not ASCII-safe, and then interpolate into a HTML template or use in a JSONP API, as you can cause syntax errors or open a cross-site scripting vulnerability this way. That’s because JSON is not a strict subset of Javascript, and when disabling ASCII-safe encoding the U+2028 and U+2029 separators will not be escaped to u2028
and u2029
sequences.
Use the following config to add UTF-8 support:
app.config['JSON_AS_ASCII'] = False
If you still want to user flask’s json and ensure the utf-8 encoding then you can do something like this:
from flask import json,Response
@app.route("/")
def hello():
my_list = []
my_list.append(u'ტესტ')
data = { "result" : my_list}
json_string = json.dumps(data,ensure_ascii = False)
#creating a Response object to set the content type and the encoding
response = Response(json_string,content_type="application/json; charset=utf-8" )
return response
#I hope this helps
In my case the above solution was not sufficient. (Running flask on the GCP App Engine flexible environment). I ended up doing:
json_str = json.dumps(myDict, ensure_ascii = False, indent=4, sort_keys=True)
encoding = chardet.detect(json_str)['encoding']
json_unicode = json_str.decode(encoding)
json_utf8 = json_unicode.encode('utf-8')
response = make_response(json_utf8)
response.headers['Content-Type'] = 'application/json; charset=utf-8'
response.headers['mimetype'] = 'application/json'
response.status_code = status