How can I copy one collection from MongoDB using pymongo and paste to another empty collection?

Question

I want to copy full collection (e.g. name ‘home’).
Then do some changes in the ‘home’ collection or remove doc inside it (not a collection).
And then replace changed ‘home’ collection to its default state from item 1.

I do next:

db = client["database"]
home = db['home'].find()  # get collection.
db['home'].remove({})  # remove doc from home
for i in home:
      self.db['home'].insert(i)

But the collection is empty.

Asked By: Vadim Kovrizhkin

||

Source

Answer 1

The problem with your code example is that find() returns a database cursor to the collection, not all documents in the collection. So when you remove all documents from the home collection, the cursor will also point to an empty collection.

In order to copy a collection to another collection in the same server, you can utilise MongoDB Aggregation operator $match and $out

pipeline = [ {"$match": {}}, 
             {"$out": "destination_collection"},
]
db.source_collection.aggregate(pipeline)

Using your example code, now you can do

source = db["source_collection"]
destination = db["destination_collection"]

# Remove all documents, or make modifications. 
source.remove({}) 

# Restore documents from the source collection.  
for doc in destination: 
      source.insert(doc)
# or instead you can just use the same aggregation method above but reverse the collection name.

Note : db.collection.copyTo() has been deprecated since MongoDB v3.0.

If you would like to copy to another MongoDB server, you can utilise db.cloneCollection(). In PyMongo it would be a command such below:

db.command("cloneCollection", **{'collection': "databaseName.source_collection", 'from': "another_host:another_port"})

Depending on your overall goal, you may find MongoDB BackUp methods useful.

Answered By: Wan B.

Answer 2

This could be the easiest way to do that, I personally prefer it, so you can add as many filters as you like:

from pymongo import MongoClient

def CopyFromColl1ToColl2(database1,collection1,database2,collection2):
    db1 = MongoClient('mongodb://127.0.0.1:27017')[database1][collection1]
    db2 = MongoClient('mongodb://127.0.0.1:27017')[database2][collection2]
    #here you can put the filters you like.
    for a in db1.find():
        try:
            db2.insert(a)
            print(a)
        except:
            print('did not copy')

# You can choose the database name and the collection name
CopyFromColl1ToColl2('database1','collection1','database2','collection2')

Answered By: Karam Qusai

Answer 3

There should be better ways, but mongo>4.0 deprecated the copydb/copycollection commands.
What you can do as an alternative, you can for loop around collections, and for loop through docs and insert the db/collection you want.

    db1_collections = client.db1.list_collection_names()

    for collection in db1_collections:
        collections_cursor = client.db1[collection].find()
        clt = client.db2[collection]
        for document in collections_cursor:
            clt.with_options(write_concern=WriteConcern(w=0)).insert_one(document)

Answered By: Mehmet Burak Sayıcı

How can I copy one collection from MongoDB using pymongo and paste to another empty collection?

Question:

Answers: