Count number of files with certain extension in Python
Question:
I am fairly new to Python and I am trying to figure out the most efficient way to count the number of .TIF files in a particular sub-directory.
Doing some searching, I found one example (I have not tested), which claimed to count all of the files in a directory:
file_count = sum((len(f) for _, _, f in os.walk(myPath)))
This is fine, but I need to only count TIF files. My directory will contain other files types, but I only want to count TIFs.
Currently I am using the following code:
tifCounter = 0
for root, dirs, files in os.walk(myPath):
for file in files:
if file.endswith('.tif'):
tifCounter += 1
It works fine, but the looping seems to be excessive/expensive to me. Any way to do this more efficiently?
Thanks.
Answers:
Your code is fine.
Yes, you’re going to need to loop over those files to filter out the .tif files, but looping over a small in-memory array is negligible compared to the work of scanning the file directory to find these files in the first place, which you have to do anyway.
I wouldn’t worry about optimizing this code.
Something has to iterate over all files in the directory, and look at every single file name – whether that’s your code or a library routine. So no matter what the specific solution, they will all have roughly the same cost.
If you think it’s too much code, and if you don’t actually need to search subdirectories recursively, you can use the glob module:
import glob
tifCounter = len(glob.glob1(myPath,"*.tif"))
If you do need to search recursively, or for some other reason don’t want to use the glob module, you could use
file_count = sum(len(f for f in fs if f.lower().endswith('.tif')) for _, _, fs in os.walk(myPath))
This is the “Pythonic” way to adapt the example you found for your purposes. But it’s not going to be significantly faster or more efficient than the loop you’ve been using; it’s just a really compact syntax for more or less the same thing.
For this particular use case, if you don’t want to recursively search in the subdirectory, you can use os.listdir:
len([f for f in os.listdir(myPath)
if f.endswith('.tif') and os.path.isfile(os.path.join(myPath, f))])
try using fnmatch
https://docs.python.org/2/library/fnmatch.html
import fnmatch,os
num_files = len(fnmatch.filter(os.listdir(your_dir),'*.tif'))
print(num_files)
I am fairly new to Python and I am trying to figure out the most efficient way to count the number of .TIF files in a particular sub-directory.
Doing some searching, I found one example (I have not tested), which claimed to count all of the files in a directory:
file_count = sum((len(f) for _, _, f in os.walk(myPath)))
This is fine, but I need to only count TIF files. My directory will contain other files types, but I only want to count TIFs.
Currently I am using the following code:
tifCounter = 0
for root, dirs, files in os.walk(myPath):
for file in files:
if file.endswith('.tif'):
tifCounter += 1
It works fine, but the looping seems to be excessive/expensive to me. Any way to do this more efficiently?
Thanks.
Your code is fine.
Yes, you’re going to need to loop over those files to filter out the .tif files, but looping over a small in-memory array is negligible compared to the work of scanning the file directory to find these files in the first place, which you have to do anyway.
I wouldn’t worry about optimizing this code.
Something has to iterate over all files in the directory, and look at every single file name – whether that’s your code or a library routine. So no matter what the specific solution, they will all have roughly the same cost.
If you think it’s too much code, and if you don’t actually need to search subdirectories recursively, you can use the glob module:
import glob
tifCounter = len(glob.glob1(myPath,"*.tif"))
If you do need to search recursively, or for some other reason don’t want to use the glob module, you could use
file_count = sum(len(f for f in fs if f.lower().endswith('.tif')) for _, _, fs in os.walk(myPath))
This is the “Pythonic” way to adapt the example you found for your purposes. But it’s not going to be significantly faster or more efficient than the loop you’ve been using; it’s just a really compact syntax for more or less the same thing.
For this particular use case, if you don’t want to recursively search in the subdirectory, you can use os.listdir:
len([f for f in os.listdir(myPath)
if f.endswith('.tif') and os.path.isfile(os.path.join(myPath, f))])
try using fnmatch
https://docs.python.org/2/library/fnmatch.html
import fnmatch,os
num_files = len(fnmatch.filter(os.listdir(your_dir),'*.tif'))
print(num_files)