Parse key value pairs in a text file
Question:
I am a newbie with Python and I search how to parse a .txt file.
My .txt file is a namelist with computation informations like :
myfile.txt
var0 = 16
var1 = 1.12434E10
var2 = -1.923E-3
var3 = 920
How to read the values and put them in myvar0, myvar1, myvar2, myvar3
in python?
Answers:
I suggest storing the values in a dictionary instead of in separate local variables:
myvars = {}
with open("namelist.txt") as myfile:
for line in myfile:
name, var = line.partition("=")[::2]
myvars[name.strip()] = float(var)
Now access them as myvars["var1"]
. If the names are all valid python variable names, you can put this below:
names = type("Names", [object], myvars)
and access the values as e.g. names.var1
.
As @kev suggests, the configparser module is the way to go.
However in some scenarios (a bit ugly, I admit) but very simple and effective way to do to this is to rename myfile.txt
to myfile.py
and do a from myfile import *
(after you fix the typo var 0
-> var0
)
However, this is very insecure, so if the file is from an external source or can be written by a malicious attacker, use something that validates the data instead of executing it blindly.
If there are multiple comma-separated values on a single line, here’s code to parse that out:
res = {}
pairs = args.split(", ")
for p in pairs:
var, val = p.split("=")
res[var] = val
I personally solved this by creating a .py file that just contains all the parameters as variables – then did:
include PARAMETERS.py
in the program modules that need the parameters.
It’s a bit ugly, but VERY simple and easy to work with.
Use pandas.read_csv when the file format becomes more fancy (like comments).
val = u'''var0 = 16
var1 = 1.12434E10
var2 = -1.923E-3
var3 = 920'''
print(pandas.read_csv(StringIO(val), # or read_csv('myfile.txt',
delimiter='s*=s*',
header=None,
names=['key','value'],
dtype=dict(key=numpy.object,value=numpy.object), # or numpy.float64
index_col=['key']).to_dict()['value'])
# prints {u'var1': u'1.12434E10', u'var0': u'16', u'var3': u'920', u'var2': u'-1.923E-3'}
Dict comprehensions (PEP 274) can be used for a shorter expression (60 characters):
d = {k:float(v) for k, v in (l.split('=') for l in open(f))}
EDIT: shortened from 72 to 60 characters thanks to @jmb suggestion (avoid .readlines()
).
Similar to @lauritz-v-thaulow but, just a line by line read into a variable.
Here is a simple Copy-Pasta so you can understand a bit more.
As the config file has to be a specific format.
import os
# Example creating an valid temp test file to get a better result.
MY_CONFIG = os.path.expanduser('~/.test_credentials')
with open(MY_CONFIG, "w") as f:
f.write("API_SECRET_KEY=123456789")
f.write(os.linesep)
f.write("API_SECRET_CONTENT=12345678")
myvars = {}
with open(MY_CONFIG, "r") as myfile:
for line in myfile:
line = line.strip()
name, var = line.partition("=")[::2]
myvars[name.strip()] = str(var)
# Iterate thru all the items created.
for k, v in myvars.items():
print("{} | {}".format(k, v))
# API_SECRET_KEY | 123456789
# API_SECRET_CONTENT | 12345678
# Access the API_SECRET_KEY item directly
print("{}".format(myvars['API_SECRET_KEY']))
# 123456789
# Access the API_SECRET_CONTENT item directly
print("{}".format(myvars['API_SECRET_CONTENT']))
# 12345678
I am a newbie with Python and I search how to parse a .txt file.
My .txt file is a namelist with computation informations like :
myfile.txt
var0 = 16
var1 = 1.12434E10
var2 = -1.923E-3
var3 = 920
How to read the values and put them in myvar0, myvar1, myvar2, myvar3
in python?
I suggest storing the values in a dictionary instead of in separate local variables:
myvars = {}
with open("namelist.txt") as myfile:
for line in myfile:
name, var = line.partition("=")[::2]
myvars[name.strip()] = float(var)
Now access them as myvars["var1"]
. If the names are all valid python variable names, you can put this below:
names = type("Names", [object], myvars)
and access the values as e.g. names.var1
.
As @kev suggests, the configparser module is the way to go.
However in some scenarios (a bit ugly, I admit) but very simple and effective way to do to this is to rename myfile.txt
to myfile.py
and do a from myfile import *
(after you fix the typo var 0
-> var0
)
However, this is very insecure, so if the file is from an external source or can be written by a malicious attacker, use something that validates the data instead of executing it blindly.
If there are multiple comma-separated values on a single line, here’s code to parse that out:
res = {}
pairs = args.split(", ")
for p in pairs:
var, val = p.split("=")
res[var] = val
I personally solved this by creating a .py file that just contains all the parameters as variables – then did:
include PARAMETERS.py
in the program modules that need the parameters.
It’s a bit ugly, but VERY simple and easy to work with.
Use pandas.read_csv when the file format becomes more fancy (like comments).
val = u'''var0 = 16
var1 = 1.12434E10
var2 = -1.923E-3
var3 = 920'''
print(pandas.read_csv(StringIO(val), # or read_csv('myfile.txt',
delimiter='s*=s*',
header=None,
names=['key','value'],
dtype=dict(key=numpy.object,value=numpy.object), # or numpy.float64
index_col=['key']).to_dict()['value'])
# prints {u'var1': u'1.12434E10', u'var0': u'16', u'var3': u'920', u'var2': u'-1.923E-3'}
Dict comprehensions (PEP 274) can be used for a shorter expression (60 characters):
d = {k:float(v) for k, v in (l.split('=') for l in open(f))}
EDIT: shortened from 72 to 60 characters thanks to @jmb suggestion (avoid .readlines()
).
Similar to @lauritz-v-thaulow but, just a line by line read into a variable.
Here is a simple Copy-Pasta so you can understand a bit more.
As the config file has to be a specific format.
import os
# Example creating an valid temp test file to get a better result.
MY_CONFIG = os.path.expanduser('~/.test_credentials')
with open(MY_CONFIG, "w") as f:
f.write("API_SECRET_KEY=123456789")
f.write(os.linesep)
f.write("API_SECRET_CONTENT=12345678")
myvars = {}
with open(MY_CONFIG, "r") as myfile:
for line in myfile:
line = line.strip()
name, var = line.partition("=")[::2]
myvars[name.strip()] = str(var)
# Iterate thru all the items created.
for k, v in myvars.items():
print("{} | {}".format(k, v))
# API_SECRET_KEY | 123456789
# API_SECRET_CONTENT | 12345678
# Access the API_SECRET_KEY item directly
print("{}".format(myvars['API_SECRET_KEY']))
# 123456789
# Access the API_SECRET_CONTENT item directly
print("{}".format(myvars['API_SECRET_CONTENT']))
# 12345678