Writing a Python Music Streamer
Question:
I would like to implement a server in Python that streams music in MP3 format over HTTP. I would like it to broadcast the music such that a client can connect to the stream and start listening to whatever is currently playing, much like a radio station.
Previously, I’ve implemented my own HTTP server in Python using SocketServer.TCPServer (yes I know BaseHTTPServer exists, just wanted to write a mini HTTP stack myself), so how would a music streamer be different architecturally? What libraries would I need to look at on the network side and on the MP3 side?
Answers:
You’ll want to look into serving m3u or pls files. That should give you a file format that players understand well enough to hit your http server looking for mp3 files.
A minimal m3u file would just be a simple text file with one song url per line. Assuming you’ve got the following URLs available on your server:
/playlists/<playlist_name/playlist_id>
/songs/<song_name/song_id>
You’d serve a playlist from the url:
/playlists/myfirstplaylist
And the contents of the resource would be just:
/songs/1
/songs/mysong.mp3
A player (like Winamp) will be able to open the URL to the m3u file on your HTTP server and will then start streaming the first song on the playlist. All you’ll have to do to support this is serve the mp3 file just like you’d serve any other static content.
Depending on how many clients you want to support you may want to look into asynchronous IO using a library like Twisted to support tons of simultaneous streams.
You’ll want to have a .m3u or .pls file that points at a static URI (e.g. http://example.com/now_playing.mp3) then give them mp3 data starting wherever you are in the song when they ask for that file. Probably there are a bunch of minor issues I’m glossing over here…However, at least as forest points out, you can just start streaming the mp3 data from any byte.
The mp3 format was designed for streaming, which makes some things simpler than you might have expected. The data is essentially a stream of audio frames with built-in boundary markers, rather than a file header followed by raw data. This means that once a client is expecting to receive audio data, you can just start sending it bytes from any point in an existing mp3 source, whether it be live or a file, and the client will sync up to the next frame it finds and start playing audio. Yay!
Of course, you’ll have to give clients a way to set up the connection. The de-facto standard is the SHOUTcast (ICY) protocol. This is very much like HTTP, but with status and header fields just different enough that it isn’t directly compatible with Python’s built-in http server libraries. You might be able to get those libraries to do some of the work for you, but their documented interfaces won’t be enough to get it done; you’ll have to read their code to understand how to make them speak SHOUTcast.
Here are a few links to get you started:
https://web.archive.org/web/20220912105447/http://forums.winamp.com/showthread.php?threadid=70403
https://web.archive.org/web/20190214132820/http://www.smackfu.com/stuff/programming/shoutcast.html
http://en.wikipedia.org/wiki/Shoutcast
I suggest starting with a single mp3 file as your data source, getting the client-server connection setup and playback working, and then moving on to issues like live sources, multiple encoding bit rates, inband meta-data, and playlists.
Playlists are generally either .pls or .m3u files, and essentially just static text files pointing at the URL for your live stream. They’re not difficult and not even strictly necessary, since many (most?) mp3 streaming clients will accept a live stream URL with no playlist at all.
As for architecture, the field is pretty much wide open. You have as many options as there are for HTTP servers. Threaded? Worker processes? Event driven? It’s up to you. To me, the more interesting question is how to share the data from a single input stream (the broadcaster) with the network handlers serving multiple output streams (the players). In order to avoid IPC and synchronization complications, I would probably start with a single-threaded event-driven design. In python 2, a library like gevent will give you very good I/O performance while allowing you to structure your code in a very understandable way. In python 3, I would prefer asyncio coroutines.
Since you already have good python experience (given you’ve already written an HTTP server) I can only provide a few pointers on how to extend the ground-work you’ve already done:
-
Prepare your server for dealing with Request Headers like: Accept-Encoding, Range, TE (Transfer Encoding), etc. An MP3-over-HTTP player (i.e. VLC) is nothing but an mp3 player that knows how to “speak” HTTP and “seek” to different positions in the file.
-
Use wireshark or tcpdump to sniff actual HTTP requests done by VLC when playing an mp3 over HTTP, so you know how what request headers you’ll be receiving and implement them.
Good luck with your project!
I would like to implement a server in Python that streams music in MP3 format over HTTP. I would like it to broadcast the music such that a client can connect to the stream and start listening to whatever is currently playing, much like a radio station.
Previously, I’ve implemented my own HTTP server in Python using SocketServer.TCPServer (yes I know BaseHTTPServer exists, just wanted to write a mini HTTP stack myself), so how would a music streamer be different architecturally? What libraries would I need to look at on the network side and on the MP3 side?
You’ll want to look into serving m3u or pls files. That should give you a file format that players understand well enough to hit your http server looking for mp3 files.
A minimal m3u file would just be a simple text file with one song url per line. Assuming you’ve got the following URLs available on your server:
/playlists/<playlist_name/playlist_id>
/songs/<song_name/song_id>
You’d serve a playlist from the url:
/playlists/myfirstplaylist
And the contents of the resource would be just:
/songs/1
/songs/mysong.mp3
A player (like Winamp) will be able to open the URL to the m3u file on your HTTP server and will then start streaming the first song on the playlist. All you’ll have to do to support this is serve the mp3 file just like you’d serve any other static content.
Depending on how many clients you want to support you may want to look into asynchronous IO using a library like Twisted to support tons of simultaneous streams.
You’ll want to have a .m3u or .pls file that points at a static URI (e.g. http://example.com/now_playing.mp3) then give them mp3 data starting wherever you are in the song when they ask for that file. Probably there are a bunch of minor issues I’m glossing over here…However, at least as forest points out, you can just start streaming the mp3 data from any byte.
The mp3 format was designed for streaming, which makes some things simpler than you might have expected. The data is essentially a stream of audio frames with built-in boundary markers, rather than a file header followed by raw data. This means that once a client is expecting to receive audio data, you can just start sending it bytes from any point in an existing mp3 source, whether it be live or a file, and the client will sync up to the next frame it finds and start playing audio. Yay!
Of course, you’ll have to give clients a way to set up the connection. The de-facto standard is the SHOUTcast (ICY) protocol. This is very much like HTTP, but with status and header fields just different enough that it isn’t directly compatible with Python’s built-in http server libraries. You might be able to get those libraries to do some of the work for you, but their documented interfaces won’t be enough to get it done; you’ll have to read their code to understand how to make them speak SHOUTcast.
Here are a few links to get you started:
https://web.archive.org/web/20220912105447/http://forums.winamp.com/showthread.php?threadid=70403
https://web.archive.org/web/20190214132820/http://www.smackfu.com/stuff/programming/shoutcast.html
http://en.wikipedia.org/wiki/Shoutcast
I suggest starting with a single mp3 file as your data source, getting the client-server connection setup and playback working, and then moving on to issues like live sources, multiple encoding bit rates, inband meta-data, and playlists.
Playlists are generally either .pls or .m3u files, and essentially just static text files pointing at the URL for your live stream. They’re not difficult and not even strictly necessary, since many (most?) mp3 streaming clients will accept a live stream URL with no playlist at all.
As for architecture, the field is pretty much wide open. You have as many options as there are for HTTP servers. Threaded? Worker processes? Event driven? It’s up to you. To me, the more interesting question is how to share the data from a single input stream (the broadcaster) with the network handlers serving multiple output streams (the players). In order to avoid IPC and synchronization complications, I would probably start with a single-threaded event-driven design. In python 2, a library like gevent will give you very good I/O performance while allowing you to structure your code in a very understandable way. In python 3, I would prefer asyncio coroutines.
Since you already have good python experience (given you’ve already written an HTTP server) I can only provide a few pointers on how to extend the ground-work you’ve already done:
-
Prepare your server for dealing with Request Headers like:
Accept-Encoding,Range,TE (Transfer Encoding), etc. An MP3-over-HTTP player (i.e. VLC) is nothing but an mp3 player that knows how to “speak” HTTP and “seek” to different positions in the file. -
Use wireshark or tcpdump to sniff actual HTTP requests done by VLC when playing an mp3 over HTTP, so you know how what request headers you’ll be receiving and implement them.
Good luck with your project!