Suppress error message during reading of file in Python
Question:
I am reading a text file using pandas.read_csv. A small example of the file can be found here. Below, is the way in which I am reading the file.
df = pd.read_csv('file.txt', error_bad_lines=False, sep=r't+', header=None, dtype='|U', engine='python')
I am able to read the file properly and do all work. But, I keep getting messages like the ones shown below, and I will like to ignore them.
Skipping line 689: Expected 81 fields in line 689, saw 265. Error could possibly be due to quotes being ignored when a multi-char delimiter is used.
There are hundreds of such lines (for different line numbers). I have already tried the answer for hiding of warnings shown in here (Hide all warnings). Unfortunately, it does not work. Also, I will like to suppress (not show) only the specific kind of error messages that I have mentioned in this post (Skipping line ... delimiter is used.).
How can I do this?
Answers:
Use argument warn_bad_lines=False:
pd.read_csv(error_bad_lines=True, warn_bad_lines=False)
I am reading a text file using pandas.read_csv. A small example of the file can be found here. Below, is the way in which I am reading the file.
df = pd.read_csv('file.txt', error_bad_lines=False, sep=r't+', header=None, dtype='|U', engine='python')
I am able to read the file properly and do all work. But, I keep getting messages like the ones shown below, and I will like to ignore them.
Skipping line 689: Expected 81 fields in line 689, saw 265. Error could possibly be due to quotes being ignored when a multi-char delimiter is used.
There are hundreds of such lines (for different line numbers). I have already tried the answer for hiding of warnings shown in here (Hide all warnings). Unfortunately, it does not work. Also, I will like to suppress (not show) only the specific kind of error messages that I have mentioned in this post (Skipping line ... delimiter is used.).
How can I do this?
Use argument warn_bad_lines=False:
pd.read_csv(error_bad_lines=True, warn_bad_lines=False)