How does the Python BaseRequestHandler rfile.read() method work?

Question

I’m sending a text file with a string in a Python script via POST to my server:

fo = open('data.txt','a')
fo.write("hi, this is my testing data")
fo.close()

with open('data.txt', 'rb') as f:
    r = requests.post("http://XXX.XX.X.X", data = {'data.txt':f})
    f.close()

And receiving and handling it here in my server handler script, built off an example found online:

def do_POST(self):
    data = self.rfile.read(int(self.headers.getheader('Content-Length')))
    empty = [data]
    with open('processing.txt', 'wb') as file:
        for item in empty:
            file.write("%sn" % item)
                
    file.close()
    self._set_headers()
    self.wfile.write("<html><body><h1>POST!</h1></body></html>")

My question is, how does:

self.rfile.read(int(self.headers.getheader('Content-Length')))

take the length of my data (an integer, # of bytes/characters) and read my file? I am confused how it knows what my data contains. What is going on behind the scenes with HTTP?

It outputs data.txt=hi%2C+this+is+my+testing+data

to my processing.txt, but I am expecting "hi this is my testing data"

I tried but failed to find documentation for what exactly rfile.read() does, and if simply finding that answers my question I’d appreciate it, and I could just delete this question.

Asked By: kasool

||

Source

Answer 1

Your client code snippet reads contents from the file data.txt and makes a POST request to your server with data structured as a key-value pair. The data sent to your server in this case is one key data.txt with the corresponding value being the contents of the file.

Your server code snippet reads the entire HTTP Request body and dumps it into a file. The key-value pair structured and sent from the client comes in a format that can be decoded by Python’s built in library urlparse.

Here is a solution that could work:

def do_POST(self):
    length = int(self.headers.getheader('content-length'))
    field_data = self.rfile.read(length)
    fields = urlparse.parse_qs(field_data)

This snippet of code was shamefully borrowed from: https://stackoverflow.com/a/31363982/705471

If you’d like to extract the contents of your text file back, adding the following line to the above snippet could help:

data_file = fields["data.txt"]

To learn more about how such information is encoded for the purposes of HTTP, read more at: https://en.wikipedia.org/wiki/Percent-encoding

Answered By: Axe

How does the Python BaseRequestHandler rfile.read() method work?

Question:

Answers: