Polars for Python: How to get rid of "Ensure you pass a path to the file instead of a python file object" warning when reading to a dataframe?
Question:
The statement
- I’m reading data sets using
Polars.read_csv()
method via a Python file handler:
with gzip.open(os.path.join(getParameters()['rdir'], dataset)) as compressed_file:
df = pl.read_csv(compressed_file, sep = 't', ignore_errors=True)
- A performance warning keeps popping up:
Polars found a filename. Ensure you pass a path to the file instead of a python file object when possible for best performance.
Possible solutions
- I already tried Python warning suppression, but it seems Polars literally just prints out this statement without any default warning associated.
- Another possibility would be to read using non-handler methods?
Any ideas on how to get rid of this annoying message will be highly appreciated.
Answers:
I had a similar issue with opening from a ZipFile
object. The solution was to add a .read()
method to the filename. Maybe the same would work in your case?
with gzip.open(os.path.join(getParameters()['rdir'], dataset)) as compressed_file:
df = pl.read_csv(compressed_file.read(), sep = 't', ignore_errors=True)
The statement
- I’m reading data sets using
Polars.read_csv()
method via a Python file handler:
with gzip.open(os.path.join(getParameters()['rdir'], dataset)) as compressed_file:
df = pl.read_csv(compressed_file, sep = 't', ignore_errors=True)
- A performance warning keeps popping up:
Polars found a filename. Ensure you pass a path to the file instead of a python file object when possible for best performance.
Possible solutions
- I already tried Python warning suppression, but it seems Polars literally just prints out this statement without any default warning associated.
- Another possibility would be to read using non-handler methods?
Any ideas on how to get rid of this annoying message will be highly appreciated.
I had a similar issue with opening from a ZipFile
object. The solution was to add a .read()
method to the filename. Maybe the same would work in your case?
with gzip.open(os.path.join(getParameters()['rdir'], dataset)) as compressed_file:
df = pl.read_csv(compressed_file.read(), sep = 't', ignore_errors=True)