JSON ValueError: Expecting property name: line 1 column 2 (char 1)
Question:
I am having trouble using json.loads to convert to a dict object and I can’t figure out what I’m doing wrong.The exact error I get running this is
ValueError: Expecting property name: line 1 column 2 (char 1)
Here is my code:
from kafka.client import KafkaClient
from kafka.consumer import SimpleConsumer
from kafka.producer import SimpleProducer, KeyedProducer
import pymongo
from pymongo import MongoClient
import json
c = MongoClient("54.210.157.57")
db = c.test_database3
collection = db.tweet_col
kafka = KafkaClient("54.210.157.57:9092")
consumer = SimpleConsumer(kafka,"myconsumer","test")
for tweet in consumer:
print tweet.message.value
jsonTweet=json.loads(({u'favorited': False, u'contributors': None})
collection.insert(jsonTweet)
I’m pretty sure that the error is occuring at the 2nd to last line
jsonTweet=json.loads({u'favorited': False, u'contributors': None})
but I do not know what to do to fix it. Any advice would be appreciated.
Answers:
json.loads will load a json string into a python dict, json.dumps will dump a python dict to a json string, for example:
>>> json_string = '{"favorited": false, "contributors": null}'
'{"favorited": false, "contributors": null}'
>>> value = json.loads(json_string)
{u'favorited': False, u'contributors': None}
>>> json_dump = json.dumps(value)
'{"favorited": false, "contributors": null}'
So that line is incorrect since you are trying to load a python dict, and json.loads is expecting a valid json string which should have <type 'str'>.
So if you are trying to load the json, you should change what you are loading to look like the json_string above, or you should be dumping it. This is just my best guess from the given information. What is it that you are trying to accomplish?
Also you don’t need to specify the u before your strings, as @Cld mentioned in the comments.
I encountered another problem that returns the same error.
Single quote issue
I used a json string with single quotes :
{
'property': 1
}
But json.loads accepts only double quotes for json properties :
{
"property": 1
}
Final comma issue
json.loads doesn’t accept a final comma:
{
"property": "text",
"property2": "text2",
}
Solution: ast to solve single quote and final comma issues
You can use ast (part of standard library for both Python 2 and 3) for this processing. Here is an example :
import ast
# ast.literal_eval() return a dict object, we must use json.dumps to get JSON string
import json
# Single quote to double with ast.literal_eval()
json_data = "{'property': 'text'}"
json_data = ast.literal_eval(json_data)
print(json.dumps(json_data))
# Displays : {"property": "text"}
# ast.literal_eval() with double quotes
json_data = '{"property": "text"}'
json_data = ast.literal_eval(json_data)
print(json.dumps(json_data))
# Displays : {"property": "text"}
# ast.literal_eval() with final coma
json_data = "{'property': 'text', 'property2': 'text2',}"
json_data = ast.literal_eval(json_data)
print(json.dumps(json_data))
# Displays : {"property2": "text2", "property": "text"}
Using ast will prevent you from single quote and final comma issues by interpet the JSON like Python dictionnary (so you must follow the Python dictionnary syntax). It’s a pretty good and safely alternative of eval() function for literal structures.
Python documentation warned us of using large/complex string :
Warning It is possible to crash the Python interpreter with a
sufficiently large/complex string due to stack depth limitations in
Python’s AST compiler.
json.dumps with single quotes
To use json.dumps with single quotes easily you can use this code:
import ast
import json
data = json.dumps(ast.literal_eval(json_data_single_quote))
ast documentation
Tool
If you frequently edit JSON, you may use CodeBeautify. It helps you to fix syntax error and minify/beautify JSON.
I hope it helps.
- replace all single quotes with double quotes
- replace ‘u”‘ from your strings to ‘”‘ … so basically convert internal unicodes to strings before loading the string into json
>> strs = "{u'key':u'val'}"
>> strs = strs.replace("'",'"')
>> json.loads(strs.replace('u"','"'))
All other answers may answer your query, but I faced same issue which was due to stray , which I added at the end of my json string like this:
{
"key":"123sdf",
"bus_number":"asd234sdf",
}
I finally got it working when I removed extra , like this:
{
"key":"123sdf",
"bus_number":"asd234sdf"
}
Hope this help! cheers.
used ast,
example
In [15]: a = "[{'start_city': '1', 'end_city': 'aaa', 'number': 1},
...: {'start_city': '2', 'end_city': 'bbb', 'number': 1},
...: {'start_city': '3', 'end_city': 'ccc', 'number': 1}]"
In [16]: import ast
In [17]: ast.literal_eval(a)
Out[17]:
[{'end_city': 'aaa', 'number': 1, 'start_city': '1'},
{'end_city': 'bbb', 'number': 1, 'start_city': '2'},
{'end_city': 'ccc', 'number': 1, 'start_city': '3'}]
A different case in which I encountered this was when I was using echo to pipe the JSON into my python script and carelessly wrapped the JSON string in double quotes:
echo "{"thumbnailWidth": 640}" | myscript.py
Note that the JSON string itself has quotes and I should have done:
echo '{"thumbnailWidth": 640}' | myscript.py
As it was, this is what the python script received: {thumbnailWidth: 640}; the double quotes were effectively stripped.
I am having trouble using json.loads to convert to a dict object and I can’t figure out what I’m doing wrong.The exact error I get running this is
ValueError: Expecting property name: line 1 column 2 (char 1)
Here is my code:
from kafka.client import KafkaClient
from kafka.consumer import SimpleConsumer
from kafka.producer import SimpleProducer, KeyedProducer
import pymongo
from pymongo import MongoClient
import json
c = MongoClient("54.210.157.57")
db = c.test_database3
collection = db.tweet_col
kafka = KafkaClient("54.210.157.57:9092")
consumer = SimpleConsumer(kafka,"myconsumer","test")
for tweet in consumer:
print tweet.message.value
jsonTweet=json.loads(({u'favorited': False, u'contributors': None})
collection.insert(jsonTweet)
I’m pretty sure that the error is occuring at the 2nd to last line
jsonTweet=json.loads({u'favorited': False, u'contributors': None})
but I do not know what to do to fix it. Any advice would be appreciated.
json.loads will load a json string into a python dict, json.dumps will dump a python dict to a json string, for example:
>>> json_string = '{"favorited": false, "contributors": null}'
'{"favorited": false, "contributors": null}'
>>> value = json.loads(json_string)
{u'favorited': False, u'contributors': None}
>>> json_dump = json.dumps(value)
'{"favorited": false, "contributors": null}'
So that line is incorrect since you are trying to load a python dict, and json.loads is expecting a valid json string which should have <type 'str'>.
So if you are trying to load the json, you should change what you are loading to look like the json_string above, or you should be dumping it. This is just my best guess from the given information. What is it that you are trying to accomplish?
Also you don’t need to specify the u before your strings, as @Cld mentioned in the comments.
I encountered another problem that returns the same error.
Single quote issue
I used a json string with single quotes :
{
'property': 1
}
But json.loads accepts only double quotes for json properties :
{
"property": 1
}
Final comma issue
json.loads doesn’t accept a final comma:
{
"property": "text",
"property2": "text2",
}
Solution: ast to solve single quote and final comma issues
You can use ast (part of standard library for both Python 2 and 3) for this processing. Here is an example :
import ast
# ast.literal_eval() return a dict object, we must use json.dumps to get JSON string
import json
# Single quote to double with ast.literal_eval()
json_data = "{'property': 'text'}"
json_data = ast.literal_eval(json_data)
print(json.dumps(json_data))
# Displays : {"property": "text"}
# ast.literal_eval() with double quotes
json_data = '{"property": "text"}'
json_data = ast.literal_eval(json_data)
print(json.dumps(json_data))
# Displays : {"property": "text"}
# ast.literal_eval() with final coma
json_data = "{'property': 'text', 'property2': 'text2',}"
json_data = ast.literal_eval(json_data)
print(json.dumps(json_data))
# Displays : {"property2": "text2", "property": "text"}
Using ast will prevent you from single quote and final comma issues by interpet the JSON like Python dictionnary (so you must follow the Python dictionnary syntax). It’s a pretty good and safely alternative of eval() function for literal structures.
Python documentation warned us of using large/complex string :
Warning It is possible to crash the Python interpreter with a
sufficiently large/complex string due to stack depth limitations in
Python’s AST compiler.
json.dumps with single quotes
To use json.dumps with single quotes easily you can use this code:
import ast
import json
data = json.dumps(ast.literal_eval(json_data_single_quote))
ast documentation
Tool
If you frequently edit JSON, you may use CodeBeautify. It helps you to fix syntax error and minify/beautify JSON.
I hope it helps.
- replace all single quotes with double quotes
- replace ‘u”‘ from your strings to ‘”‘ … so basically convert internal unicodes to strings before loading the string into json
>> strs = "{u'key':u'val'}"
>> strs = strs.replace("'",'"')
>> json.loads(strs.replace('u"','"'))
All other answers may answer your query, but I faced same issue which was due to stray , which I added at the end of my json string like this:
{
"key":"123sdf",
"bus_number":"asd234sdf",
}
I finally got it working when I removed extra , like this:
{
"key":"123sdf",
"bus_number":"asd234sdf"
}
Hope this help! cheers.
used ast,
example
In [15]: a = "[{'start_city': '1', 'end_city': 'aaa', 'number': 1},
...: {'start_city': '2', 'end_city': 'bbb', 'number': 1},
...: {'start_city': '3', 'end_city': 'ccc', 'number': 1}]"
In [16]: import ast
In [17]: ast.literal_eval(a)
Out[17]:
[{'end_city': 'aaa', 'number': 1, 'start_city': '1'},
{'end_city': 'bbb', 'number': 1, 'start_city': '2'},
{'end_city': 'ccc', 'number': 1, 'start_city': '3'}]
A different case in which I encountered this was when I was using echo to pipe the JSON into my python script and carelessly wrapped the JSON string in double quotes:
echo "{"thumbnailWidth": 640}" | myscript.py
Note that the JSON string itself has quotes and I should have done:
echo '{"thumbnailWidth": 640}' | myscript.py
As it was, this is what the python script received: {thumbnailWidth: 640}; the double quotes were effectively stripped.