Cannot find file "…\r" when using os.path.join and trying to open file in read mode

Question

In my current working directory, where I run my Python script, I have multiple subdirectories which all contain the file ‘genes.faa.genespercontig.csv’. I want to create a pandas dataframe of these files.

When I run my script, I get the error:

FileNotFoundError: [Errno 2] No such file or directory: 'C:\Users\KLIF\Documents\Linda\genes.faa.genespercontig.csv\r'

My script is as follows:

import os
import pandas as pd

for root, dirs, files in os.walk(os.getcwd()):
    with open(os.path.join(root, 'genes.faa.genespercontig.csv', 'r')) as f1:
        df1 = pd.read_csv('f1', header=None, delim_whitespace=True, names = ["contig", "genes"]) 
        print(df1)

I am sure that the file is in the subdirectories, but why can Python not find it?

Asked By: Gravel

||

Source

Answer 1

Unless “r” is a file within a directory called “genes.faa.genespercontig.csv”, simple syntax error. Your parentheses need adjusting slightly:

...
with open(os.path.join(root, 'genes.faa.genespercontig.csv'), 'r') as f1:
...

Additionally, given that you say the file is contained within subdirectories, you probably need to loop over the subdirectories within the dirs variable returned by the os.walk function. I notice that you use the string 'f1' to pass to pd.read_csv but presumably you want the open file object. Something like:

import os
import pandas as pd

for root, dirs, files in os.walk(os.getcwd()):
    for subdir in dirs:
        with open(os.path.join(root,subdir,'genes.faa.genespercontig.csv'),'r') as f1:
            df1 = pd.read_csv(f1, header=None, delim_whitespace=True, names = ["contig", "genes"])
            print(df1)

To only attempt to open a file which actually exists, use:

...
filepath = os.path.join(root,subdir,'genes.faa.genespercontig.csv')
if os.path.isfile(filepath):
    with open(filepath, 'r') as f1:
...

Answered By: Kind Stranger

Cannot find file "…\r" when using os.path.join and trying to open file in read mode

Question:

Answers: