Linear regression of arrays containing NANs in Python/Numpy
Question:
I have two arrays, say varx and vary. Both contain NAN values at various positions. However, I would like to do a linear regression on both to show how much the two arrays correlate.
This was very helpful so far: http://glowingpython.blogspot.de/2012/03/linear-regression-with-numpy.html
However, using this:
slope, intercept, r_value, p_value, std_err = stats.linregress(varx, vary)
results in nans for every output variable. What is the most convenient way to take only valid values from both arrays as input to the linear regression? I heard about masking arrays, but am not sure how it works exactly.
Answers:
You can remove NaNs using a mask:
mask = ~np.isnan(varx) & ~np.isnan(vary)
slope, intercept, r_value, p_value, std_err = stats.linregress(varx[mask], vary[mask])
I have two arrays, say varx and vary. Both contain NAN values at various positions. However, I would like to do a linear regression on both to show how much the two arrays correlate.
This was very helpful so far: http://glowingpython.blogspot.de/2012/03/linear-regression-with-numpy.html
However, using this:
slope, intercept, r_value, p_value, std_err = stats.linregress(varx, vary)
results in nans for every output variable. What is the most convenient way to take only valid values from both arrays as input to the linear regression? I heard about masking arrays, but am not sure how it works exactly.
You can remove NaNs using a mask:
mask = ~np.isnan(varx) & ~np.isnan(vary)
slope, intercept, r_value, p_value, std_err = stats.linregress(varx[mask], vary[mask])