Problem with Django StreamingHttpResponse

Question

I’m having a problem with Django response class StreamingHttpResponse. When I return a generator as response using StreamingHttpResponse and make a request I excepted to retrieve each data block one by one, instead of that i retrieve the full data at once when the generator loop has finished.

My Django View:

def gen_message(msg):
    return 'ndata: {}nn'.format(msg)


def iterator():
    for i in range(100):
        yield gen_message('iteration ' + str(i))
        print(i)
        time.sleep(0.1)

class test_stream(APIView):
    def post(self, request):
        stream = iterator()
        response = StreamingHttpResponse(stream, status=200, content_type='text/event-stream')
        response['Cache-Control'] = 'no-cache'
        return response

And I make the request like that:

r = requests.post('https://******/test_stream/', stream=True)


for line in r.iter_lines():

    if line:
        decoded_line = line.decode('utf-8')
        print(decoded_line)

When I see the output of the Django server I can see the print every 0.1 seconds. But the response in the second code only shows when the for loop is finished.
Am I misunderstanding the StreamingHttpResponse class or the request class or is there another problem?
Thanks:)

Asked By: javier diez

||

Source

Answer 1

The problem is the line

stream = iterator()

Here, you’re calling the function iterator, and the variable stream will contain the return value.

Instead, try:

stream = iterator

because you want to pass the iterator function itself as the first argument to StreamingHttpResponse.

(or just do response = StreamingHttpResponse(iterator, status=200, content_type='text/event-stream') )

Answered By: Rob Bricheno

Answer 2

You will need to ensure that there is no middleware or webserver in between that first "buffers" the response.

In Nginx that is possible if the proxy_buffering setting is turned on. You will thus need to disable this with:

proxy_buffering off;

Answered By: Willem Van Onsem

Problem with Django StreamingHttpResponse

Question:

Answers: