using while loop to read files

Question

I have 200 files, from which I wanna choose the second column. I wanna store the second column of each file in a list called "colv". I wanna have colv[0]=[second column of the first file. colv[1] be the second column of the second file and so on.
I write this code, but it does not work, in this code colv[0] is the first number of the second column of the first file. Can anyone help me how to fix this issue:

colv = []
i = 1
colvar = "step7_1.colvar"
while os.path.isfile(colvar):
    with open(colvar, "r") as f_in:
        line = next(f_in)
        for line in f_in:
            a = line.split()[1]
            colv.append(a)
    i+=1
    colvar = "step7_%d.colvar" %i

Asked By: amirre

||

Source

Answer 1

Use a list comprehension to get the 2nd element of all the lines into a list.

colv = []
i = 1
colvar = "step7_1.colvar"
while os.path.isfile(colvar):
    with open(colvar, "r") as f_in:
        readline(f_in) # skip 1st line
        data = [line.split()[1] for line in f_in]
        colv.append(data)
    i+=1
    colvar = "step7_%d.colvar" %i

Answered By: Barmar

Answer 2

How about using Pandas’ read_csv() since you mention that the data has a table-like structure. In particular, you could use

import pandas as pd

colv = []
i = 1
colvar = "step7_1.colvar"
while os.path.isfile(colvar):
    df = pd.read_csv(colvar, sep=',')
    colv.append(list(df[df.columns[1]]))
    
    i+=1
    colvar = "step7_%d.colvar" %i

It returned

>colv
[[5, 6, 7, 8], [8, 9, 10, 11], [12, 13, 14, 15]]

for my vanilla sample files step7_%d.colvar.

You might need to adjust the separator character with sep.

Answered By: 7shoe

using while loop to read files

Question:

Answers: