How to go through directories to get a specific file from each directory and open that file in each directory and do some process

Question

I have a path which have many directories. I need to go through each directory and get a specific file "file.log.gz" from it and read the file and do some process.

This is my current attempt:

import os
import sys
import gzip
infile = sys.argv[1]
directory = ("%s/NEW_FOLDER" % infile)

for root, dirs, files in os.walk(directory):
    for file in files:
        if "file.log.gz" in file:
            with gzip.open(os.path.join(root, file)) as fin:
                new = False
                for line in fin:
                    if "CODE" in line.decode('utf-8'):
                        print("string is present")
                        found = True
                        exit()
                    else:
                        print("string is not present")

what i need is to go through each directories inside NEW_FOLDER and get file.log.gz. and do the following process for file.log.gz in each directory.

with the current code i get file.log.gz inside each directory but i’m not able to do rest of the process that is opening file.log.gz in each directory and do the rest process.

Expected Output:

/NEW_FOLDER/dir1/file.log.gz
string is present
/NEW_FOLDER/dir2/file.log.gz
string is present
/NEW_FOLDER/dir3/file.log.gz
string is not present

Asked By: V_S

||

Source

Answer 1

Because you are using os.walk(). You need to merge the root directory with the filename. You will notice it if you print (file) and see what the values you are getting.

Try print this out. You suppose to pass the entire directory to open and not just the file name.

for file in files:
  print(os.path.join(root, file))

Answered By: Sin Han Jinn

How to go through directories to get a specific file from each directory and open that file in each directory and do some process

Question:

Answers: