What is python's equivalent of R's NA?
Question:
What is python’s equivalent of R’s NA?
To be more specific:
R has NaN, NA, NULL, Inf and -Inf. NA is generally used when there is missing data. What is python’s equivalent?
How libraries such as numpy and pandas handle missing values?
How does scikit-learn handle missing values?
Is it different for python 2.7 and python 3?
Answers:
for pandas take a look at this.
http://pandas.pydata.org/pandas-docs/dev/missing_data.html
pandas uses NaN. You can test for null values using isnull() or not null(), drop them from a data frame using dropna() etc. The equivalent for datetime objects is NaT
nan in numpy is handled well with many functions:
>>> import numpy as np
>>> a = [1, np.nan, 2, 3]
>>> np.nanmean(a)
2.0
>>> np.nansum(a)
6.0
>>> np.isnan(a)
array([False, True, False, False], dtype=bool)
Scikit-learn doesn’t handle missing values currently.
For most machine learning algorithms, it is unclear how to handle missing values, and so we rely on the user of handling them prior to giving them to the algorithm.
Numpy doesn’t have a “missing” value. Pandas uses NaN, but inside numeric algorithms that might lead to confusion. It is possible to use masked arrays, but we don’t do that in scikit-learn (yet).
What is python’s equivalent of R’s NA?
To be more specific:
R has NaN, NA, NULL, Inf and -Inf. NA is generally used when there is missing data. What is python’s equivalent?
How libraries such as numpy and pandas handle missing values?
How does scikit-learn handle missing values?
Is it different for python 2.7 and python 3?
for pandas take a look at this.
http://pandas.pydata.org/pandas-docs/dev/missing_data.html
pandas uses NaN. You can test for null values using isnull() or not null(), drop them from a data frame using dropna() etc. The equivalent for datetime objects is NaT
nan in numpy is handled well with many functions:
>>> import numpy as np
>>> a = [1, np.nan, 2, 3]
>>> np.nanmean(a)
2.0
>>> np.nansum(a)
6.0
>>> np.isnan(a)
array([False, True, False, False], dtype=bool)
Scikit-learn doesn’t handle missing values currently.
For most machine learning algorithms, it is unclear how to handle missing values, and so we rely on the user of handling them prior to giving them to the algorithm.
Numpy doesn’t have a “missing” value. Pandas uses NaN, but inside numeric algorithms that might lead to confusion. It is possible to use masked arrays, but we don’t do that in scikit-learn (yet).