Python execute all Python files in directory and subdirs

Question

I’m trying using this code for the city wise updated population data but the python file is not writing the output results in that folder

import os
import subprocess
import glob

root_dir = "path/to/root/directory"

for dirpath, dirnames, filenames in os.walk(root_dir):
    for text_file in glob.glob(os.path.join(dirpath, '*.txt')):
        os.remove(text_file)

    for filename in filenames:
        if filename.endswith(".py"):
            filepath = os.path.join(dirpath, filename)
            os.system("python" + filepath)
            
            or 

            subprocess.run(["python", filepath])

It is deleting the old text files but python file is not generating the updated data and in the sub process it showing in the command prompt but didn’t write the text files or even creating new text files

but when I manually go to the folder and run the Python file it works fine

Asked By: Sarfraz

||

Source

Answer 1

The issue with the line os.system("python" + filepath) is that there is no space between "python" and the file, so it will attempt to run something like pythontest.py, an invalid command. The second issue is that your program might run itself, causing an infinite loop. To fix this, check if the file is the current file with the __file__ variable before you run it. Try this code:

import os
import subprocess
import glob

root_dir = os.getcwd()
for dirpath, dirnames, filenames in os.walk(root_dir):
    for text_file in glob.glob(os.path.join(dirpath, '*.txt')):
        os.remove(text_file)

    for filename in filenames:
        if filename.endswith(".py") and os.path.join(root_dir, filename) != __file__:
            filepath = os.path.join(dirpath, filename)
            subprocess.run(["python", filepath])

Note that I also changed root_dir to automatically get the current directory. You can change this if you want.

Also, thanks to MrChadMWoods for helping catch an edge case in this current file detection.

Answered By: Michael M.

Answer 2

You need to change the current working directory for your script to work:

Try:

subprocess.run(["python", filepath], cwd=dirpath)

Answered By: Corralien

Python execute all Python files in directory and subdirs

Question:

Answers: