How to save panda data frame as CSV in MinIO.?

Question

I am new in MinIo and i am using minio python library and trying to save panda data frame as CSV.
as per there documentation, I am using put_object to insert the data in remote cloud location. below is my code.

from minio import Minio
from minio.error import ResponseError
from io import StringIO, BytesIO
import pandas as pd
import os
minioClient = Minio('mydomain.com',
              access_key='my_access_key',
              secret_key='scret_key',
              secure=False)
df = panda data frame
csv_buffer = StringIO()
df.to_csv(csv_buffer)
minioClient.put_object('mu_bucket',
                       'mypath/test.csv',
                        data = csv_buffer.getvalue(),
                        length = csv_buffer.tell(),
                        content_type='application/csv')

in there documention all example are saving physical file but i need to save from dataframe.
thats why i am using StringIO to create a string buffer. but getting the below error.

AttributeError: 'str' object has no attribute 'read

Any help is greatly appreciated Thanks.

Asked By: om tripathi

||

Source

Answer 1

You have to use BytesIO instead of StringIO. BytesIO allows you to wrap byte arrays up in a stream which you can give to minio.


from io import BytesIO

import pandas as pd
from minio import Minio

minioClient = Minio('mydomain.com',
              access_key='my_access_key',
              secret_key='secret_key',
              secure=False)

df = pd.DataFrame()
csv_bytes = df.to_csv().encode('utf-8')
csv_buffer = BytesIO(csv_bytes)

minioClient.put_object('mu_bucket',
                       'mypath/test.csv',
                        data=csv_buffer,
                        length=len(csv_bytes),
                        content_type='application/csv')

Answered By: Erik Moroz

How to save panda data frame as CSV in MinIO.?

Question:

Answers: