RSS feed parser library in Python
Question:
I am looking for a good library in python that will help me parse RSS feeds. Has anyone used feedparser? Any feedback?
Answers:
First hit on G.
In answer to your followup. You could use BeautifulSoup – but feedparser is much better geared towards RSS handing.
Not to snark – but have you read feedparsers documentation? I don’t know how it could be simpler to use.
If you want an alternative, try xml.dom.minidom.
Like “Django is Python”, “RSS is XML”.
Feedparser is very powerful, configurable and sooo easy to use. A very friendly learning curve, if at all.
Example
Programatically determine how many answers your question has:
easy_install feedparser
python -c 'import feedparser; print len(feedparser.parse("http://bit.ly/c785aj")["entries"])'
Using feedparser is a much better option than rolling your own with minidom or BeautifulSoup.
- It normalizes the differences between all versions of RSS and Atom so you don’t have to have different code for each type.
- It’s good about detecting different date formats and other variations in feeds.
- It automatically follows HTTP redirects.
- It sanitizes HTML content.
- It has support for ETag and Last-Modified headers so you can see if the feed has changed just by downloading the HTTP header and not the whole feed.
- It has support for authenticated feeds.
- It has support for HTTP proxies.
Like others have mentioned, just try it. It’s like 2 lines of code to parse a feed. My only complaint is that it just uses dictionaries as its data model and some attributes can be missing from the dictionary if they weren’t in the feed, so you have to check for that in your code. But the documentation is very clear on which attributes will always be in the dictionary and which might be missing.
Finally, I can vouch for it, as I’ve written an application that uses it. See here: http://www.feednotifier.com/
I Strongly recommend feedparser.
I know this is a very old topic, but for what it worth, I was using feedparser (Universal feed parser) version 5.1.3 and I recently swiched to speedparser (0.1.8) for performance reasons. It has pretty much the same interfaces, but run faster.
I’m using it for an amateur Python-for-Android application and speedparser runs about 5 times faster on my feeds.
As of 2019, atoma is a possible alternative to feedparser, although I have not used it.
I am looking for a good library in python that will help me parse RSS feeds. Has anyone used feedparser? Any feedback?
First hit on G.
In answer to your followup. You could use BeautifulSoup – but feedparser is much better geared towards RSS handing.
Not to snark – but have you read feedparsers documentation? I don’t know how it could be simpler to use.
If you want an alternative, try xml.dom.minidom.
Like “Django is Python”, “RSS is XML”.
Feedparser is very powerful, configurable and sooo easy to use. A very friendly learning curve, if at all.
Example
Programatically determine how many answers your question has:
easy_install feedparser
python -c 'import feedparser; print len(feedparser.parse("http://bit.ly/c785aj")["entries"])'
Using feedparser is a much better option than rolling your own with minidom or BeautifulSoup.
- It normalizes the differences between all versions of RSS and Atom so you don’t have to have different code for each type.
- It’s good about detecting different date formats and other variations in feeds.
- It automatically follows HTTP redirects.
- It sanitizes HTML content.
- It has support for ETag and Last-Modified headers so you can see if the feed has changed just by downloading the HTTP header and not the whole feed.
- It has support for authenticated feeds.
- It has support for HTTP proxies.
Like others have mentioned, just try it. It’s like 2 lines of code to parse a feed. My only complaint is that it just uses dictionaries as its data model and some attributes can be missing from the dictionary if they weren’t in the feed, so you have to check for that in your code. But the documentation is very clear on which attributes will always be in the dictionary and which might be missing.
Finally, I can vouch for it, as I’ve written an application that uses it. See here: http://www.feednotifier.com/
I Strongly recommend feedparser.
I know this is a very old topic, but for what it worth, I was using feedparser (Universal feed parser) version 5.1.3 and I recently swiched to speedparser (0.1.8) for performance reasons. It has pretty much the same interfaces, but run faster.
I’m using it for an amateur Python-for-Android application and speedparser runs about 5 times faster on my feeds.
As of 2019, atoma is a possible alternative to feedparser, although I have not used it.