Silencing warnings in Pandas
Question:
When running:
mydf = pd.read_csv(p_file, sep=',', error_bad_lines=False, index_col=False)
I get many lines like the following:
...
Skipping line 77432: expected 15 fields, saw 16
Skipping line 77497: expected 15 fields, saw 16
Skipping line 77528: expected 15 fields, saw 16
Skipping line 77560: expected 15 fields, saw 16
Skipping line 77625: expected 15 fields, saw 16
Skipping line 77656: expected 15 fields, saw 16
...
How can I silence these warnings? How can I find the list of warning classes in Pandas?
Answers:
The function read_csv has the kwarg on_bad_lines, so you should be able to do the following:
mydf = pd.read_csv(p_file, sep=',', index_col=False, on_bad_lines = 'skip')
From the documentation
on_bad_lines{‘error’, ‘warn’, ‘skip’} or callable, default ‘error’. Specifies what to do upon encountering a bad line (a line with too many fields).
When running:
mydf = pd.read_csv(p_file, sep=',', error_bad_lines=False, index_col=False)
I get many lines like the following:
... Skipping line 77432: expected 15 fields, saw 16 Skipping line 77497: expected 15 fields, saw 16 Skipping line 77528: expected 15 fields, saw 16 Skipping line 77560: expected 15 fields, saw 16 Skipping line 77625: expected 15 fields, saw 16 Skipping line 77656: expected 15 fields, saw 16 ...
How can I silence these warnings? How can I find the list of warning classes in Pandas?
The function read_csv has the kwarg on_bad_lines, so you should be able to do the following:
mydf = pd.read_csv(p_file, sep=',', index_col=False, on_bad_lines = 'skip')
From the documentation
on_bad_lines{‘error’, ‘warn’, ‘skip’} or callable, default ‘error’. Specifies what to do upon encountering a bad line (a line with too many fields).