Python move all files from multiple subdirectories to different corresponding subdirectories

Question

Im working on a python script taking all the files in sub folders inside a folder and move to another folder that has identically structured sub folders. (see picture)

picture

My script currently just grabs all the files from directories and move to another folder location. Is there an elegant way to do this? I’m dealing with 31 sub directories inside a folder so hard coding for all 31 would be tedious

thanks a lot

import shutil
import os
src = r'C:folderA'
dst = r'C:folderB'

for root, subdirs, files in os.walk(src):
    for file in files:
        path = os.path.join(root, file)
        shutil.move(path, dst)

Asked By: maple 401

||

Source

Answer 1

You can use os.path.relpath to obtain the relative path to src and then join the relative path with dst to obtain the new path name:

import shutil
import os
src = r'C:folderA'
dst = r'C:folderB'

for root, subdirs, files in os.walk(src):
    for file in files:
        path = os.path.join(root, file)
        shutil.move(path, os.path.join(dst, os.path.relpath(path, src)))

Answered By: blhsing

Answer 2

To address the multiple directories, you can build a dict of {'src':'dest'} pairs:

import shutil
import os
src_dst_map = {
   r'C:folderA' : r'C:folderB',
   r'C:folderC' : r'C:folderD'
} #ect

for src, dst in src_dst_map.items():
   for root, subdirs, files in os.walk(src):
      for file in files:
         path = os.path.join(root, file)
         shutil.move(path, dst)

Answered By: TheoretiCAL

Answer 3

First, if you’re in the mood for something hacky, IIRC, you can call copytree and pass move rather than copy2 as the copier, and it ends up moving all of the files to the right place, but on either Unix or Windows (I forget which) it leaves empty directories behind instead of nothing. But that’s trivial to clean up:

shutil.copytree(src, dst, function=shutil.move)
shutil.rmtree(src)

If you don’t want something hacky:

From your diagram, it seems like you just have 31 subdirectories all directly under the source directory, not nested underneath each other. So, trying to walk the whole hierarchy recursively with walk, and then trying to reassemble the paths properly, etc. is just making things more complicated, and probably even less efficient.

All you have to do is get those 31 directories, and move them. For example, using scandir:

for entry in os.scandir(src):
    if entry.is_dir():
        shutil.move(entry.path, dst)

Or, if you have files as well as directories at the top level, it’s even simpler:

for entry in os.scandir(src):
    shutil.move(entry.path, dst)

If you’re on an old version of Python that doesn’t have scandir, you’ll have to use listdir, but that’s only a tiny bit more difficult:

for name in os.listdir(src):
    shutil.move(os.path.join(src, name), dst)

Answered By: abarnert

Answer 4

Here is modified way of doing the duplicate check:

src = r'D:TestSourceFolder'
dst = r'D:TestDestFolder'


for root, subdirs, files in os.walk(src):
    for file in files:
        path = os.path.join(root, file)
        
        print("Found a file: "+path)
        print("Only name of the file: "+file)
        print("Will be moved to: "+os.path.join(dst, file))
        
        # If there is no duplicate file present, then move else don't move
        if not os.path.exists(os.path.join(dst, file)):  
            #shutil.move(path, os.path.join(dst, os.path.relpath(path, src)))
            shutil.move(path, os.path.join(dst, file) )
            print("1 File moved : "+file+"n")
        else:
            print("1 File not moved because duplicate file found at: "+path+"n")

Answered By: Rajesh Swarnkar

Python move all files from multiple subdirectories to different corresponding subdirectories

Question:

Answers: