Pandas DataFrame select the specific columns with NaN values
Question:
I have a two-column DataFrame, I want to select the rows with NaN in either column.
I used this method df[ (df['a'] == np.NaN) | (df['b'] == np.NaN) ]
However it returns an empty answer. I don’t know what’s the problem
Answers:
You could apply isnull() to the whole dataframe then check if the rows have any nulls with any(1)
df[df.isnull().any(1)]
Timing
df = pd.DataFrame(np.random.choice((np.nan, 1), (1000000, 2), p=(.2, .8)), columns=['A', 'B'])
I did not expect these results…
You can use isna() as well:
df[df['merged_key_words'].isna() | df['key_words'].isna()]
I have a two-column DataFrame, I want to select the rows with NaN in either column.
I used this method df[ (df['a'] == np.NaN) | (df['b'] == np.NaN) ]
However it returns an empty answer. I don’t know what’s the problem
You could apply isnull() to the whole dataframe then check if the rows have any nulls with any(1)
df[df.isnull().any(1)]
Timing
df = pd.DataFrame(np.random.choice((np.nan, 1), (1000000, 2), p=(.2, .8)), columns=['A', 'B'])
I did not expect these results…
You can use isna() as well:
df[df['merged_key_words'].isna() | df['key_words'].isna()]