os.walk without hidden folders

Question

I need to list all files with the containing directory path inside a folder. I tried to use os.walk, which obviously would be the perfect solution.

However, it also lists hidden folders and files. I’d like my application not to list any hidden folders or files. Is there any flag you can use to make it not yield any hidden files?

Cross-platform is not really important to me, it’s ok if it only works for linux (.* pattern)

Asked By: unddoch

||

Source

Answer 1

No, there is no option to os.walk() that’ll skip those. You’ll need to do so yourself (which is easy enough):

for root, dirs, files in os.walk(path):
    files = [f for f in files if not f[0] == '.']
    dirs[:] = [d for d in dirs if not d[0] == '.']
    # use files and dirs

Note the dirs[:] = slice assignment; os.walk recursively traverses the subdirectories listed in dirs. By replacing the elements of dirs with those that satisfy a criteria (e.g., directories whose names don’t begin with .), os.walk() will not visit directories that fail to meet the criteria.

This only works if you keep the topdown keyword argument to True, from the documentation of os.walk():

When topdown is True, the caller can modify the dirnames list in-place (perhaps using del or slice assignment), and walk() will only recurse into the subdirectories whose names remain in dirnames; this can be used to prune the search, impose a specific order of visiting, or even to inform walk() about directories the caller creates or renames before it resumes walk() again.

Answered By: Martijn Pieters

Answer 2

I realize it wasn’t asked in the question, but I had a similar problem where I wanted to exclude both hidden files and files beginning with __, specifically __pycache__ directories. I landed on this question because I was trying to figure out why my list comprehension was not doing what I expected. I was not modifying the list in place with dirnames[:].

I created a list of prefixes I wanted to exclude and modified the dirnames in place like so:

    exclude_prefixes = ('__', '.')  # exclusion prefixes
    for dirpath, dirnames, filenames in os.walk(node):
        # exclude all dirs starting with exclude_prefixes
        dirnames[:] = [dirname
                       for dirname in dirnames
                       if not dirname.startswith(exclude_prefixes)]

Answered By: dmmfll

Answer 3

My use-case was similar to that of OP, except I wanted to return a count of the total number of sub-directories inside a certain folder. In my case I wanted to omit any sub-directories named .git (as well as any folders that may be nested inside these .git folders).

In Python 3.6.7, I found that the accepted answer’s approach didn’t work — it counted all .git folder and their sub-folders. Here’s what did work for me:

num_local_subdir = 0
for root, dirs, files in os.walk(local_folder_path):
    if '.git' in dirs:
        dirs.remove('.git')
    num_local_subdir += (len(dirs))

Answered By: James Dellinger

Answer 4

Another solution that can allow you to skip those hidden folders using any and map functions.

for root, dirs, files in os.walk(path):
    if any(map(lambda p: p[0] == '.', dirs)):
        continue

Answered By: François-Xavier Babin

os.walk without hidden folders

Question:

Answers: