Single vs double quotes in JSON
Question:
My code:
import simplejson as json
s = "{'username':'dfdsfdsf'}" #1
#s = '{"username":"dfdsfdsf"}' #2
j = json.loads(s)
#1
definition is wrong
#2
definition is right
I heard that in Python that single and double quote can be interchangable. Can anyone explain this to me?
Answers:
JSON syntax is not Python syntax. JSON requires double quotes for its strings.
You can dump JSON with double quote by:
import json
# mixing single and double quotes
data = {'jsonKey': 'jsonValue',"title": "hello world"}
# get string with all double quotes
json_string = json.dumps(data)
you can use ast.literal_eval()
>>> import ast
>>> s = "{'username':'dfdsfdsf'}"
>>> ast.literal_eval(s)
{'username': 'dfdsfdsf'}
demjson is also a good package to solve the problem of bad json syntax:
pip install demjson
Usage:
from demjson import decode
bad_json = "{'username':'dfdsfdsf'}"
python_dict = decode(bad_json)
Edit:
demjson.decode
is a great tool for damaged json, but when you are dealing with big amourt of json data ast.literal_eval
is a better match and much faster.
As said, JSON is not Python syntax. You need to use double quotes in JSON. Its creator is (in-)famous for using strict subsets of allowable syntax to ease programmer cognitive overload.
Below can fail if one of the JSON strings itself contains a single quote as pointed out by @Jiaaro. DO NOT USE. Left here as an example of what does not work.
It is really useful to know that there are no single quotes in a JSON string. Say, you copied and pasted it from a browser console/whatever. Then, you can just type
a = json.loads('very_long_json_string_pasted_here')
This might otherwise break if it used single quotes, too.
I recently came up against a very similar problem, and believe my solution would work for you too. I had a text file which contained a list of items in the form:
["first item", 'the "Second" item', "thi'rd", 'some \"hellish\" 'quoted" item']
I wanted to parse the above into a python list but was not keen on eval() as I couldn’t trust the input. I tried first using JSON but it only accepts double quoted items, so I wrote my own very simple lexer for this specific case (just plug in your own “stringtoparse” and you will get as output list: ‘items’)
#This lexer takes a JSON-like 'array' string and converts single-quoted array items into escaped double-quoted items,
#then puts the 'array' into a python list
#Issues such as ["item 1", '","item 2 including those double quotes":"', "item 3"] are resolved with this lexer
items = [] #List of lexed items
item = "" #Current item container
dq = True #Double-quotes active (False->single quotes active)
bs = 0 #backslash counter
in_item = False #True if currently lexing an item within the quotes (False if outside the quotes; ie comma and whitespace)
for c in stringtoparse[1:-1]: #Assuming encasement by brackets
if c=="\": #if there are backslashes, count them! Odd numbers escape the quotes...
bs = bs + 1
continue
if (dq and c=='"') or (not dq and c=="'"): #quote matched at start/end of an item
if bs & 1==1: #if escaped quote, ignore as it must be part of the item
continue
else: #not escaped quote - toggle in_item
in_item = not in_item
if item!="": #if item not empty, we must be at the end
items += [item] #so add it to the list of items
item = "" #and reset for the next item
continue
if not in_item: #toggle of single/double quotes to enclose items
if dq and c=="'":
dq = False
in_item = True
elif not dq and c=='"':
dq = True
in_item = True
continue
if in_item: #character is part of an item, append it to the item
if not dq and c=='"': #if we are using single quotes
item += bs * "\" + """ #escape double quotes for JSON
else:
item += bs * "\" + c
bs = 0
continue
Hopefully it is useful to somebody. Enjoy!
import ast
answer = subprocess.check_output(PYTHON_ + command, shell=True).strip()
print(ast.literal_eval(answer.decode(UTF_)))
Works for me
You can fix it that way:
s = "{'username':'dfdsfdsf'}"
j = eval(s)
It truly solved my problem using eval function.
single_quoted_dict_in_string = "{'key':'value', 'key2': 'value2'}"
desired_double_quoted_dict = eval(single_quoted_dict_in_string)
# Go ahead, now you can convert it into json easily
print(desired_double_quoted_dict)
Two issues with answers given so far, if , for instance, one streams such non-standard JSON. Because then one might have to interpret an incoming string (not a python dictionary).
Issue 1 – demjson
:
With Python 3.7.+ and using conda I wasn’t able to install demjson since obviosly it does not support Python >3.5 currently. So I need a solution with simpler means, for instance ast
and/or json.dumps
.
Issue 2 – ast
& json.dumps
:
If a JSON is both single quoted and contains a string in at least one value, which in turn contains single quotes, the only simple yet practical solution I have found is applying both:
In the following example we assume line
is the incoming JSON string object :
>>> line = str({'abc':'008565','name':'xyz','description':'can control TV's and more'})
Step 1: convert the incoming string into a dictionary using ast.literal_eval()
Step 2: apply json.dumps
to it for the reliable conversion of keys and values, but without touching the contents of values:
>>> import ast
>>> import json
>>> print(json.dumps(ast.literal_eval(line)))
{"abc": "008565", "name": "xyz", "description": "can control TV's and more"}
json.dumps
alone would not do the job because it does not interpret the JSON, but only see the string. Similar for ast.literal_eval()
: although it interprets correctly the JSON (dictionary), it does not convert what we need.
You can use
json.dumps(your_json, separators=(",", ":"))
well you can fix the one with single quote with
import json
single_quote_dictioary = "{'username':'dfdsfdsf'}" #1
json.dumps(eval(single_quote_dictioary))
My code:
import simplejson as json
s = "{'username':'dfdsfdsf'}" #1
#s = '{"username":"dfdsfdsf"}' #2
j = json.loads(s)
#1
definition is wrong
#2
definition is right
I heard that in Python that single and double quote can be interchangable. Can anyone explain this to me?
JSON syntax is not Python syntax. JSON requires double quotes for its strings.
You can dump JSON with double quote by:
import json
# mixing single and double quotes
data = {'jsonKey': 'jsonValue',"title": "hello world"}
# get string with all double quotes
json_string = json.dumps(data)
you can use ast.literal_eval()
>>> import ast
>>> s = "{'username':'dfdsfdsf'}"
>>> ast.literal_eval(s)
{'username': 'dfdsfdsf'}
demjson is also a good package to solve the problem of bad json syntax:
pip install demjson
Usage:
from demjson import decode
bad_json = "{'username':'dfdsfdsf'}"
python_dict = decode(bad_json)
Edit:
demjson.decode
is a great tool for damaged json, but when you are dealing with big amourt of json dataast.literal_eval
is a better match and much faster.
As said, JSON is not Python syntax. You need to use double quotes in JSON. Its creator is (in-)famous for using strict subsets of allowable syntax to ease programmer cognitive overload.
Below can fail if one of the JSON strings itself contains a single quote as pointed out by @Jiaaro. DO NOT USE. Left here as an example of what does not work.
It is really useful to know that there are no single quotes in a JSON string. Say, you copied and pasted it from a browser console/whatever. Then, you can just type
a = json.loads('very_long_json_string_pasted_here')
This might otherwise break if it used single quotes, too.
I recently came up against a very similar problem, and believe my solution would work for you too. I had a text file which contained a list of items in the form:
["first item", 'the "Second" item', "thi'rd", 'some \"hellish\" 'quoted" item']
I wanted to parse the above into a python list but was not keen on eval() as I couldn’t trust the input. I tried first using JSON but it only accepts double quoted items, so I wrote my own very simple lexer for this specific case (just plug in your own “stringtoparse” and you will get as output list: ‘items’)
#This lexer takes a JSON-like 'array' string and converts single-quoted array items into escaped double-quoted items,
#then puts the 'array' into a python list
#Issues such as ["item 1", '","item 2 including those double quotes":"', "item 3"] are resolved with this lexer
items = [] #List of lexed items
item = "" #Current item container
dq = True #Double-quotes active (False->single quotes active)
bs = 0 #backslash counter
in_item = False #True if currently lexing an item within the quotes (False if outside the quotes; ie comma and whitespace)
for c in stringtoparse[1:-1]: #Assuming encasement by brackets
if c=="\": #if there are backslashes, count them! Odd numbers escape the quotes...
bs = bs + 1
continue
if (dq and c=='"') or (not dq and c=="'"): #quote matched at start/end of an item
if bs & 1==1: #if escaped quote, ignore as it must be part of the item
continue
else: #not escaped quote - toggle in_item
in_item = not in_item
if item!="": #if item not empty, we must be at the end
items += [item] #so add it to the list of items
item = "" #and reset for the next item
continue
if not in_item: #toggle of single/double quotes to enclose items
if dq and c=="'":
dq = False
in_item = True
elif not dq and c=='"':
dq = True
in_item = True
continue
if in_item: #character is part of an item, append it to the item
if not dq and c=='"': #if we are using single quotes
item += bs * "\" + """ #escape double quotes for JSON
else:
item += bs * "\" + c
bs = 0
continue
Hopefully it is useful to somebody. Enjoy!
import ast
answer = subprocess.check_output(PYTHON_ + command, shell=True).strip()
print(ast.literal_eval(answer.decode(UTF_)))
Works for me
You can fix it that way:
s = "{'username':'dfdsfdsf'}"
j = eval(s)
It truly solved my problem using eval function.
single_quoted_dict_in_string = "{'key':'value', 'key2': 'value2'}"
desired_double_quoted_dict = eval(single_quoted_dict_in_string)
# Go ahead, now you can convert it into json easily
print(desired_double_quoted_dict)
Two issues with answers given so far, if , for instance, one streams such non-standard JSON. Because then one might have to interpret an incoming string (not a python dictionary).
Issue 1 – demjson
:
With Python 3.7.+ and using conda I wasn’t able to install demjson since obviosly it does not support Python >3.5 currently. So I need a solution with simpler means, for instance ast
and/or json.dumps
.
Issue 2 – ast
& json.dumps
:
If a JSON is both single quoted and contains a string in at least one value, which in turn contains single quotes, the only simple yet practical solution I have found is applying both:
In the following example we assume line
is the incoming JSON string object :
>>> line = str({'abc':'008565','name':'xyz','description':'can control TV's and more'})
Step 1: convert the incoming string into a dictionary using ast.literal_eval()
Step 2: apply json.dumps
to it for the reliable conversion of keys and values, but without touching the contents of values:
>>> import ast
>>> import json
>>> print(json.dumps(ast.literal_eval(line)))
{"abc": "008565", "name": "xyz", "description": "can control TV's and more"}
json.dumps
alone would not do the job because it does not interpret the JSON, but only see the string. Similar for ast.literal_eval()
: although it interprets correctly the JSON (dictionary), it does not convert what we need.
You can use
json.dumps(your_json, separators=(",", ":"))
well you can fix the one with single quote with
import json
single_quote_dictioary = "{'username':'dfdsfdsf'}" #1
json.dumps(eval(single_quote_dictioary))