pysftp putfo creates an empty file on SFTP server but not streaming the content from StringIO

Question

My code first writes lines to a CSV in io.StringIO():

fileBuffer = io.StringIO()

# write header
header_writer = csv.DictWriter(fileBuffer, fieldnames=columnNames)
header_writer.writeheader()

# write lines
writer = csv.writer(fileBuffer, delimiter=',') 

for line in data:
            line_dec = line.decode('ISO-8859-1') 
            # print([line_dec])
            writer.writerow([line_dec])

The following code also prints all expected rows:

$print(fileBuffer.getvalue())    # -> prints all expected rows

I can also successfully connect to the SFTP Server using pysftp and even in the with pysftp the code successfully returns all expected rows:

with pysftp.Connection(host, username=user, password=pw, cnopts=cnopts) as sftp: 
            print('sucessfully connected to {} via Port 22'.format(host))
            print(fileBuffer.getvalue())    # -> prints all expected rows
            sftp.putfo(fileBuffer, file2BeSavedAs)   # -> no rows put on FTP Server

Here comes the actual problem:
Unfortunately, the code only creates the file without writing the data respectively the body into it. On the other hand, my code does not return any error message.

How can I put a CSV from StringIO to an SFTP server?

Asked By: C.Tomas

||

Source

Answer 1

You have to seek a read pointer of the buffer back to the beginning, before you try to upload the buffer:

fileBuffer.seek(0)
sftp.putfo(fileBuffer, file2BeSavedAs)

Though a better approach is to write the CSV directly to the server, without an intermediate buffer. Use Connection.open to obtain a file-like object representing a file on the SFTP server:

with sftp.open(file2BeSavedAs, mode='w', bufsize=32768) as f:
    writer = csv.writer(f, delimiter=',') 
    # ...

For the purpose of the bufsize argument, see:
Writing to a file on SFTP server opened using Paramiko/pysftp "open" method is slow

For a similar question, with progress display, see:
How to use Paramiko getfo to download file from SFTP server to memory to process it

Though pysftp is dead. You better use Paramiko. See pysftp vs. Paramiko. With Paramiko the code would be pretty much the same.

Answered By: Martin Prikryl

pysftp putfo creates an empty file on SFTP server but not streaming the content from StringIO

Question:

Answers: