read multiple .dat file and extract specific information

Question

I have multiple .dat files and all of them start with similar information at the beginning about the observation time, lat, long, and… .
After this text information, there are 16 columns of observed data. which looks like this:

#Occultation start (UTC): 2022-01-01 00:10:00
#Occultation stop  (UTC): 2022-01-01 00:12:04
#Occultation point latitude (degN):   -59.5
#Occultation point longitude (degE):  -54.6
#Center of curvature (m):     3264.126   -4599.850   27305.984
#Radius of curvature (m):  6382932.736
#Azimuth (degN):    177.809
#Undulation (m):     20.772
#Elevation (m):       0.000
#Processing method:  Wave optics (CT2)
#Background data:    ECMWF forecast 
#Receiver data type: L1caL2c
#Navbit correction:  extern
#Occultation type:   Rising 
#OL/CL mode:         OL->CL
#Alt OL/CL (m):    8821
#alt w.r.t. EGM96 
geoid|lat|lon|azimuth|ba_opt|impact_opt|refrac|dry_press|dry_temp|geopotential height|ba_raw|ba_L1|ba_L2|ba_model|snr_L1|snr_L2
      0.000  -99999000.000  -99999000.000  -99999000.000   -0.99999E+08  -99999000.000   -0.99999E+08  -99999000.000  -99999000.000  -99999000.000   -0.99999E+08   -0.99999E+08   -0.99999E+08   -0.99999E+08  -99999000.000  -99999000.000

what I want to do is :

iterating through the multiple .dat files and extracting the latitude and longitude data and plotting them. I can do this with one file but in the loop I got errors.
this code could give me the lat and long, but for some unknown reason after running it I receive an error:

f = open('/Users/file.dat','r')
data = f.read()
a = data.split
lat=a[14:15]

lon=a[19:20]
lat

TypeError                                 Traceback (most recent call last)
<ipython-input-1-4f169d6b0f6f> in <module>
      2 data = f.read()
      3 a = data.split
----> 4 lat=a[14:15]
      5 
      6 lon=a[19:20]

TypeError: 'builtin_function_or_method' object is not subscriptable

and this is my loop:

x=[]
y=[]
for file in pathlist:
    with open (file, 'r') as input:   
        dataset= file.read()
        a=dataset.split
        lat=a[14:15]
        lon=a[19:20]
        dlat=x.append[lat]
        dlon=y.append[lon]

and I receive this error:

AttributeError: 'str' object has no attribute 'read'

Asked By: bah

||

Source

Answer 1

You forgot the brackets to call the method. You can specify the splitting criterium by passing a value to the sep-argument (default is whitespace), see doc for details.

a = data.split()

then slices a[14:15] will make sense hence TypeError: 'builtin_function_or_method' object is not subscriptable should be fix.

To fix the AttributeError: 'str' object has no attribute 'read' you need to apply the read method to the file descriptor not the file name. [Do shadow built-in function’ names, input is a built-in!]

for file in pathlist:
    with open (file, 'r') as fd:   
        dataset = fd.read()

Answered By: cards

Answer 2

Use re,

import re

for file in pathlist:
    with open (file, 'r') as fp:
        lines = fp.readlines()
        latitude = re.match(r'.* ([-.d]+)', lines[2]).group(1)
        longitude = re.match(r'.* ([-.d]+)', lines[3]).group(1)
        print(latitude, longitude)

Result:

-59.5 -54.6

You can use re to find the latitude and longitude. split the entire file data and find the value based on position it’s might be a high chance of getting wrong values.

Answered By: Rahul K P

read multiple .dat file and extract specific information

Question:

Answers: