How to convert a pymongo.cursor.Cursor into a dict?
Question:
I am using pymongo to query for all items in a region (actually it is to query for all venues in a region on a map). I used db.command(SON()) before to search in a spherical region, which can return me a dictionary and in the dictionary there is a key called results which contains the venues. Now I need to search in a square area and I am suggested to use db.places.find, however, this returns me a pymongo.cursor.Cursor class and I have no idea how to extract the venue results from it.
Does anyone know whether I should convert the cursor into a dict and extract the results out, or use another method to query for items in a square region?
BTW, db is pymongo.database.Database class
The code is:
>>> import pymongo
>>> db = pymongo.MongoClient(host).PSRC
>>> resp = db.places.find({"loc": {"$within": {"$box": [[ll_lng,ll_lat], [ur_lng,ur_lat]]}}})
>>> for doc in resp:
>>> print(doc)
I have values of ll_lng, ll_lat, ur_lng and ur_lat, use these values but it prints nothing from this codes
Answers:
The MongoDB find method does not return a single result, but a list of results in the form of a Cursor. This latter is an iterator, so you can go through it with a for loop.
For your case, just use the findOne method instead of find. This will returns you a single document as a dictionary.
to_dict()
Convert a SON document to a normal Python dictionary instance.
This is trickier than just dict(…) because it needs to be recursive.
The find method returns a Cursor instance, which allows you to iterate over all matching documents.
To get the first document that matches the given criteria, you need to use find_one. The result of find_one is a dictionary.
You can always use the list constructor to return a list of all the documents in the collection but bear in mind that this will load all the data in memory and may not be what you want.
You should do that if you need to reuse the cursor and have a good reason not to use rewind()
Demo using find:
>>> import pymongo
>>> conn = pymongo.MongoClient()
>>> db = conn.test #test is my database
>>> col = db.spam #Here spam is my collection
>>> cur = col.find()
>>> cur
<pymongo.cursor.Cursor object at 0xb6d447ec>
>>> for doc in cur:
... print(doc) # or do something with the document
...
{'a': 1, '_id': ObjectId('54ff30faadd8f30feb90268f'), 'b': 2}
{'a': 1, 'c': 3, '_id': ObjectId('54ff32a2add8f30feb902690'), 'b': 2}
Demo using find_one:
>>> col.find_one()
{'a': 1, '_id': ObjectId('54ff30faadd8f30feb90268f'), 'b': 2}
I suggest create a list and append dictionary into it.
x = []
cur = db.dbname.find()
for i in cur:
x.append(i)
print(x)
Now x is a list of dictionary, you can manipulate the same in usual python way.
Easy
import pymongo
conn = pymongo.MongoClient()
db = conn.test #test is my database
col = db.spam #Here spam is my collection
array = list(col.find())
print(array)
There you go
Map function is fast way to convert big collection
from time import time
cursor = db.collection.find()
def f(x):
return x['name']
t1 = time()
blackset = set(map(f, cursor))
print(time() - t1)
I am using pymongo to query for all items in a region (actually it is to query for all venues in a region on a map). I used db.command(SON()) before to search in a spherical region, which can return me a dictionary and in the dictionary there is a key called results which contains the venues. Now I need to search in a square area and I am suggested to use db.places.find, however, this returns me a pymongo.cursor.Cursor class and I have no idea how to extract the venue results from it.
Does anyone know whether I should convert the cursor into a dict and extract the results out, or use another method to query for items in a square region?
BTW, db is pymongo.database.Database class
The code is:
>>> import pymongo
>>> db = pymongo.MongoClient(host).PSRC
>>> resp = db.places.find({"loc": {"$within": {"$box": [[ll_lng,ll_lat], [ur_lng,ur_lat]]}}})
>>> for doc in resp:
>>> print(doc)
I have values of ll_lng, ll_lat, ur_lng and ur_lat, use these values but it prints nothing from this codes
The MongoDB find method does not return a single result, but a list of results in the form of a Cursor. This latter is an iterator, so you can go through it with a for loop.
For your case, just use the findOne method instead of find. This will returns you a single document as a dictionary.
to_dict()
Convert a SON document to a normal Python dictionary instance.
This is trickier than just dict(…) because it needs to be recursive.
The find method returns a Cursor instance, which allows you to iterate over all matching documents.
To get the first document that matches the given criteria, you need to use find_one. The result of find_one is a dictionary.
You can always use the list constructor to return a list of all the documents in the collection but bear in mind that this will load all the data in memory and may not be what you want.
You should do that if you need to reuse the cursor and have a good reason not to use rewind()
Demo using find:
>>> import pymongo
>>> conn = pymongo.MongoClient()
>>> db = conn.test #test is my database
>>> col = db.spam #Here spam is my collection
>>> cur = col.find()
>>> cur
<pymongo.cursor.Cursor object at 0xb6d447ec>
>>> for doc in cur:
... print(doc) # or do something with the document
...
{'a': 1, '_id': ObjectId('54ff30faadd8f30feb90268f'), 'b': 2}
{'a': 1, 'c': 3, '_id': ObjectId('54ff32a2add8f30feb902690'), 'b': 2}
Demo using find_one:
>>> col.find_one()
{'a': 1, '_id': ObjectId('54ff30faadd8f30feb90268f'), 'b': 2}
I suggest create a list and append dictionary into it.
x = []
cur = db.dbname.find()
for i in cur:
x.append(i)
print(x)
Now x is a list of dictionary, you can manipulate the same in usual python way.
Easy
import pymongo
conn = pymongo.MongoClient()
db = conn.test #test is my database
col = db.spam #Here spam is my collection
array = list(col.find())
print(array)
There you go
Map function is fast way to convert big collection
from time import time
cursor = db.collection.find()
def f(x):
return x['name']
t1 = time()
blackset = set(map(f, cursor))
print(time() - t1)