find string values in pandas
Question:
I have nearly 200+variables and ~2M records in Pandas dataframe. All variables should contain numeric data. But due to some error all the variables having some random text inside. I don’t know what are those text and all the variables not having same kind of text.
I want to extract those string in variable wise.
Ex:
Input
Output
Could you please help me on this.
Answers:
Use DataFrame.melt with filter by to_numeric with Series.isna:
out = df.melt().loc[lambda x: pd.to_numeric(x['value'], errors='coerce').isna()]
out = df.melt()
out = out[pd.to_numeric(x['value'], errors='coerce').isna()]
I have nearly 200+variables and ~2M records in Pandas dataframe. All variables should contain numeric data. But due to some error all the variables having some random text inside. I don’t know what are those text and all the variables not having same kind of text.
I want to extract those string in variable wise.
Ex:
Input
Output
Could you please help me on this.
Use DataFrame.melt with filter by to_numeric with Series.isna:
out = df.melt().loc[lambda x: pd.to_numeric(x['value'], errors='coerce').isna()]
out = df.melt()
out = out[pd.to_numeric(x['value'], errors='coerce').isna()]