Python3.4 read content stream web request

Question

How, in Python3.4, can I read the data from a web request which has an event-stream as content type? The stream data is updated every 30 seconds and for each of them I would extract and process data.

Example:

I performed a request on http://example.com/value121
In the headers, I can see: content-type: text/event-stream

Each 30 seconds, I can see:

name: value121
data: {‘old_value’: xx, ‘new_value’: xx}

And I would extract new_value.

Asked By: Matt

||

Source

Answer 1

Give sseclient a try: it handles SSE messages using a simple iterator-style interface.

Answered By: nneonneo

Answer 2

It’s actually relatively easy to implement text/event-stream on both the server and client. Say the server at example.com is this simple server running on port 80 (the default port):

import http.server
import json
import socketserver
import time

class RequestHandler(http.server.SimpleHTTPRequestHandler):

    def do_GET(self):
        self.send_response(200)
        self.send_header('Connection', 'Keep-Alive')
        self.send_header('Content-Type', 'text/event-stream')
        self.end_headers()

        name = self.path.strip('/').encode('UTF-8')

        old_value = None
        new_value = None
        while True:
            old_value, new_value = new_value, time.time()
            data = json.dumps({
                'old_value': old_value,
                'new_value': new_value,
            }).encode('UTF-8')
            self.wfile.write(b'name: ')
            self.wfile.write(name)
            self.wfile.write(b'rn')
            self.wfile.write(b'data: ')
            self.wfile.write(data)
            self.wfile.write(b'rn')
            self.wfile.write(b'rn')
            self.wfile.flush()
            time.sleep(30)

server = socketserver.TCPServer(('', 80), RequestHandler)
server.serve_forever()

Every 30 seconds, it outputs an event:

name: value121<CR><LF>
data: {"old_value": <old-timestamp>, "new_value": <new-timestamp>}<CR><LF>
<CR><LF>

Then, a client can connect to http://example.com/value121 on port 80 and parse each event:

import http.client
import json

connection = http.client.HTTPConnection('http://example.com', 80)
connection.request('GET', '/value121', headers={
    'Accept': 'text/event-source',
    'Connection': 'Keep-Alive',
})
with connection.getresponse() as response:
    while not response.closed:
        event = {}
        for line in response:
            line = line.decode('UTF-8')
            if line == 'rn':
                # End of event.
                break
            elif line.startswith(':'):
                # Comment, ignore.
                pass
            else:
                # Data line.
                key, value = line.split(':', 1)
                value = value.strip()
                if key == 'data':
                  value = json.loads(value)
                event[key] = value

        # Handle event, extract values, etc.
        print(event)

And every 30 seconds the client prints out:

'data': {'old_value': <old-timestamp>, 'new_value': <new-timestamp>}, 'name': 'value121'}

NOTE: This is a very basic implementation. See Using server-sent events § Event stream format for more information.

Answered By: Uyghur Lives Matter

Python3.4 read content stream web request

Question:

Answers: