Parsing GPS receiver output via regex in Python

It’s simpler to use split than a regex.

>>> line="$GPRMC,092204.999,4250.5589,S,14718.5084,E,1,12,24.4,89.6,M,,,0000*1F "
>>> line.split(',')
['$GPRMC', '092204.999', '4250.5589', 'S', '14718.5084', 'E', '1', '12', '24.4', '89.6', 'M', '', '', '0000*1F ']
>>> 

You should also first check the checksum of the data. It is calculated by XORing the characters between the $ and the * (not including them) and comparing it to the hex value at the end.

Your pastebin looks like it has some corrupt lines in it. Here is a simple check, it assumes that the line starts with $ and has no CR/LF at the end. To build a more robust parser you need to search for the ‘$’ and work through the string until hitting the ‘*’.

def check_nmea0183(s):
    """
    Check a string to see if it is a valid NMEA 0183 sentence
    """
    if s[0] != '$':
        return False
    if s[-3] != '*':
        return False

    checksum = 0
    for c in s[1:-3]:
        checksum ^= ord(c)

    if int(s[-2:],16) != checksum:
        return False

    return True

Those are comma separated values, so using a csv library is the easiest solution.

I threw that sample data you have into /var/tmp/sampledata, then I did this:

>>> import csv
>>> for line in csv.reader(open('/var/tmp/sampledata')):
...   print line
['$GPRMC', '092204.999', '**4250.5589', 'S', '14718.5084', 'E**', '1', '12', '24.4', '**89.6**', 'M', '', '', '0000\*1F']
['$GPRMC', '093345.679', '**4234.7899', 'N', '11344.2567', 'W**', '3', '02', '24.5', '**1000.23**', 'M', '', '', '0000\*1F']
['$GPRMC', '044584.936', '**1276.5539', 'N', '88734.1543', 'E**', '2', '04', '33.5', '**600.323**', 'M', '', '', '\*00']
['$GPRMC', '199304.973', '**3248.7780', 'N', '11355.7832', 'W**', '1', '06', '02.2', '**25722.5**', 'M', '', '', '\*00']
['$GPRMC', '066487.954', '**4572.0089', 'S', '45572.3345', 'W**', '3', '09', '15.0', '**35000.00**', 'M', '', '', '\*1F']

You can then process the data however you wish. It looks a little odd with the ‘**’ at the start and end of some of the values, you might want to strip that stuff off, you can do:

>> eastwest = 'E**'
>> eastwest = eastwest.strip('*')
>> print eastwest
E

You will have to cast some values as floats. So for example, the 3rd value on the first line of sample data is:

>> data = '**4250.5589'
>> print float(data.strip('*'))
4250.5589

If you need to do some more extensive analysis of your GPS data streams, here is a pyparsing solution that breaks up your data into named data fields. I extracted your pastebin’ned data to a file gpsstream.txt, and parsed it with the following:

"""
 Parse NMEA 0183 codes for GPS data
 http://en.wikipedia.org/wiki/NMEA_0183

 (data formats from http://www.gpsinformation.org/dale/nmea.htm)
"""
from pyparsing import *

lead = "$"
code = Word(alphas.upper(),exact=5)
end = "*"
COMMA = Suppress(',')
cksum = Word(hexnums,exact=2).setParseAction(lambda t:int(t[0],16))

# define basic data value forms, and attach conversion actions
word = Word(alphanums)
N,S,E,W = map(Keyword,"NSEW")
integer = Regex(r"-?d+").setParseAction(lambda t:int(t[0]))
real = Regex(r"-?d+.d*").setParseAction(lambda t:float(t[0]))
timestamp = Regex(r"d{2}d{2}d{2}.d+")
timestamp.setParseAction(lambda t: t[0][:2]+':'+t[0][2:4]+':'+t[0][4:])
def lonlatConversion(t):
    t["deg"] = int(t.deg)
    t["min"] = float(t.min)
    t["value"] = ((t.deg + t.min/60.0) 
                    * {'N':1,'S':-1,'':1}[t.ns] 
                    * {'E':1,'W':-1,'':1}[t.ew])
lat = Regex(r"(?P<deg>d{2})(?P<min>d{2}.d+),(?P<ns>[NS])").setParseAction(lonlatConversion)
lon = Regex(r"(?P<deg>d{3})(?P<min>d{2}.d+),(?P<ew>[EW])").setParseAction(lonlatConversion)

# define expression for a complete data record
value = timestamp | Group(lon) | Group(lat) | real | integer | N | S | E | W | word
item = lead + code("code") + COMMA + delimitedList(Optional(value,None))("datafields") + end + cksum("cksum")


def parseGGA(tokens):
    keys = "time lat lon qual numsats horiz_dilut alt _ geoid_ht _ last_update_secs stnid".split()
    for k,v in zip(keys, tokens.datafields):
        if k != '_':
            tokens[k] = v
    #~ print tokens.dump()

def parseGSA(tokens):
    keys = "auto_manual _3dfix prn prn prn prn prn prn prn prn prn prn prn prn pdop hdop vdop".split()
    tokens["prn"] = []
    for k,v in zip(keys, tokens.datafields):
        if k != 'prn':
            tokens[k] = v
        else:
            if v is not None:
                tokens[k].append(v)
    #~ print tokens.dump()

def parseRMC(tokens):
    keys = "time active_void lat lon speed track_angle date mag_var _ signal_integrity".split()
    for k,v in zip(keys, tokens.datafields):
        if k != '_':
            if k == 'date' and v is not None:
                v = "%06d" % v
                tokens[k] = '20%s/%s/%s' % (v[4:],v[2:4],v[:2])
            else:
                tokens[k] = v
    #~ print tokens.dump()


# process sample data
data = open("gpsstream.txt").read().expandtabs()

count = 0
for i,s,e in item.scanString(data):
    # use checksum to validate input 
    linebody = data[s+1:e-3]
    checksum = reduce(lambda a,b:a^b, map(ord, linebody))
    if i.cksum != checksum:
        continue
    count += 1

    # parse out specific data fields, depending on code field
    fn = {'GPGGA' : parseGGA, 
          'GPGSA' : parseGSA,
          'GPRMC' : parseRMC,}[i.code]
    fn(i)

    # print out time/position/speed values
    if i.code == 'GPRMC':
        print "%s %8.3f %8.3f %4d" % (i.time, i.lat.value, i.lon.value, i.speed or 0) 


print count

The $GPRMC records in your pastebin don’t seem to quite match with the ones you included in your post, but you should be able to adjust this example as necessary.

I suggest a small fix in your code because if used to parse data from the previous century the date looks like sometime in the future (for instance 2094 instead of 1994)

My fix is not fully accurate, but I take the stand that prior to the 70’s no GPS data existed.

In the def parse function for RMC sentences just replace the format line by:

p = int(v[4:])
print "p = ", p
if p > 70:
    tokens[k] = '19%s/%s/%s' % (v[4:],v[2:4],v[:2])
else:
    tokens[k] = '20%s/%s/%s' % (v[4:],v[2:4],v[:2])

This will look at the two yy digits of the year and assume that past year 70 we are dealing with sentences from the previous century.
It could be better done by comparing to today’s date and assuming that every time you deal with some data in the future, they are in fact from the past century

Thanks for all the pieces of code your provided above… I had some fun with this.

You could use a library like pynmea2 for parsing the NMEA log.

>>> import pynmea2
>>> msg = pynmea2.parse('$GPGGA,142927.829,2831.4705,N,08041.0067,W,1,07,1.0,7.9,M,-31.2,M,0.0,0000*4F')
>>> msg.timestamp, msg.latitude, msg.longitude, msg.altitude
(datetime.time(14, 29, 27), 28.524508333333333, -80.683445, 7.9)

Disclaimer: I am the author of pynmea2

This is a GPRMC string. After splitting the string, you need to parse latitude and longitude values.

line = "$GPRMC,199304.973,3248.7780,N,11355.7832,W,1,06,02.2,25722.5,M,,,*00"
line = line.split(",")

In latitude and longitude part ([..., '3248.7780', 'N', '11355.7832, 'W', ...]):

• The first number is not a pure number, it is a number which is concatenated like a string. I mean, 3248.7780 refers 32 degree, 48.7780 minutes (latitude)
• The second number (11355.7832) refers 113 degree, 55.7832 minutes (longitude)

They cannot be used in a formula as they are. They have to be converted to decimal degree.

def toDD(s):
    d = float(s[:-7])
    m = float(s[-7:]) / 60
    return d + m

lat_lon = (toDD(line[2]), line[3], toDD(line[4]), line[5])
print(lat_lon)

# (32.81296666666667, 'N', 113.92972, 'W')

This is an old question, and it was asking for a regular expression.
My contribution here is an expression validated on regex101.com:

RMC,[d.]*,A,([d.]+),([NS]),([d.]+),([EW]),[d.]*,([d.]*)

You can use something like this:

import re

p = re.compile(r'RMC,[d.]*,A,([d.]+),([NS]),([d.]+),([EW]),[d.]*,([d.]*)')

nmea = r'$GPRMC,150714.696,A,2829.6203,N,08039.0335,W,0.00,,211108,,*0A'

x = p.search(nmea)
if not x is None:
  print(f"lat: {x.group(1)} {x.group(2)}")
  print(f"lon: {x.group(3)} {x.group(4)}")
  print(f"heading: {x.group(5) if len(x.group(5)) > 0 else 'None'}")

It has the 5 capturing groups that you needed (latitude, N/S, longitude, E/W, heading) and it will filter only valid fixes (those with an ‘A’ as 3rd parameter).
Some observations:

• 14 years ago it was common to have only GPS receivers, so it was reasonable to use a regular expression that started with "$GPRMC", but nowadays you can have a multiconstellation GNSS receiver, that outputs messages that can be related to GPS, GLONASS, BEIDOU, GALILEO or something mixed, and the prefix can change accordingly to the constellation. For this reason, I’ve kept only the "RMC" identifier.
• I’ve let heading to be optional in case there’s no movement.
• Remember that you must convert the latitude and longitude to decimal values (degrees.fraction_of_degrees°) or to degrees°minutes’seconds” format to make some sense in regular maps. NMEA sentences use an strange format, with degrees concatenated to the minutes with fractional part of minutes, so you’ll have to separate the degrees and minutes and then mix them again with something like this:

  lat_nmea = float(x.group(1))
  lat_deg = lat_nmea//100
  lat_min = lat_nmea - 100*lat_deg
  lat_deg = lat_deg + lat_min/60
  print(f"conv lat {x.group(1)} -> {lat_deg}")

  lon_nmea = float(x.group(3))
  lon_deg = lon_nmea//100
  lon_min = lon_nmea - 100*lon_deg
  lon_deg = lon_deg + lon_min/60
  print(f"conv lon {x.group(3)} -> {lon_deg}")