Polars for Python: How to get rid of "Ensure you pass a path to the file instead of a python file object" warning when reading to a dataframe?

Question:

The statement

  • I’m reading data sets using Polars.read_csv() method via a Python file handler:
 with gzip.open(os.path.join(getParameters()['rdir'], dataset)) as compressed_file:
    df = pl.read_csv(compressed_file, sep = 't', ignore_errors=True)
  • A performance warning keeps popping up:
Polars found a filename. Ensure you pass a path to the file instead of a python file object when possible for best performance.

Possible solutions

  • I already tried Python warning suppression, but it seems Polars literally just prints out this statement without any default warning associated.
  • Another possibility would be to read using non-handler methods?

Any ideas on how to get rid of this annoying message will be highly appreciated.

Asked By: pabloagn

||

Answers:

I had a similar issue with opening from a ZipFile object. The solution was to add a .read() method to the filename. Maybe the same would work in your case?

 with gzip.open(os.path.join(getParameters()['rdir'], dataset)) as compressed_file:
    df = pl.read_csv(compressed_file.read(), sep = 't', ignore_errors=True)
Answered By: RustyPython
Categories: questions Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.