Math in python – converting data files to matrices
Question:
Today, as I tried to put together a script in Octave, I thought, this may be easier in python. Indeed the math operators of lists are a breeze, but loading in the file in the format is not as easy. Then I thought, it probably is, I am just not familiar with the module to do it!
So, I have a typical data file with four columns of numbers. I would like to load each column into separate lists. Is there a module I should use to make this easier?
Answers:
For fast calculations with matrices you should try Numpy, it has some functions to load data from files.
I don’t know whether this is applicable to your problem, but you might try it with numpy, especially its loadtxt and savetxt functions. You should then use only numpy arrays and avoid Python lists as they are not apt for numerical computations.
If you’re dealing with 2-dimensional data or enormously long lists, Numpy is the way to go, but if you’re not looking to do terribly advanced math, you can get by with regular Python.
>>> table = []
>>> a = "32 42 63 1123"
>>> table.append(a.split(" ")) # this would be some loop where you file.readline()...
>>> table.append(a.split(" "))
>>> table.append(a.split(" "))
>>> table.append(a.split(" "))
>>> table
[['32', '42', '63', '1123'], ['32', '42', '63', '1123'],
['32', '42', '63', '1123'], ['32', '42', '63', '1123']]
>>> zip(*table) # this "transposes" the list of lists
[('32', '32', '32', '32'), ('42', '42', '42', '42'),
('63', '63', '63', '63'), ('1123', '1123', '1123', '1123')]
>>>
The easiest way to get Numpy working is to download Enthought Python Distribution. This is especially true for Mac since installing numpy, scipy, … from scratch will take you a lot of effort.
For loading and saving some files like:
# This is some comment
1 2 3
4 5 6
7 8 9
You do
import numpy as np
data = np.loadtxt(input_filename, comment='#')
Today, as I tried to put together a script in Octave, I thought, this may be easier in python. Indeed the math operators of lists are a breeze, but loading in the file in the format is not as easy. Then I thought, it probably is, I am just not familiar with the module to do it!
So, I have a typical data file with four columns of numbers. I would like to load each column into separate lists. Is there a module I should use to make this easier?
For fast calculations with matrices you should try Numpy, it has some functions to load data from files.
I don’t know whether this is applicable to your problem, but you might try it with numpy, especially its loadtxt and savetxt functions. You should then use only numpy arrays and avoid Python lists as they are not apt for numerical computations.
If you’re dealing with 2-dimensional data or enormously long lists, Numpy is the way to go, but if you’re not looking to do terribly advanced math, you can get by with regular Python.
>>> table = []
>>> a = "32 42 63 1123"
>>> table.append(a.split(" ")) # this would be some loop where you file.readline()...
>>> table.append(a.split(" "))
>>> table.append(a.split(" "))
>>> table.append(a.split(" "))
>>> table
[['32', '42', '63', '1123'], ['32', '42', '63', '1123'],
['32', '42', '63', '1123'], ['32', '42', '63', '1123']]
>>> zip(*table) # this "transposes" the list of lists
[('32', '32', '32', '32'), ('42', '42', '42', '42'),
('63', '63', '63', '63'), ('1123', '1123', '1123', '1123')]
>>>
The easiest way to get Numpy working is to download Enthought Python Distribution. This is especially true for Mac since installing numpy, scipy, … from scratch will take you a lot of effort.
For loading and saving some files like:
# This is some comment
1 2 3
4 5 6
7 8 9
You do
import numpy as np
data = np.loadtxt(input_filename, comment='#')