Private RSS Feeds: Support for security in aggregators
We’ve been experimenting with security options for RSS feeds for our intranet product. However, we found that there weren’t many resources or guidelines for how encryption or authentification should be handled (either in feeds or in readers/aggregators).
I did some testing and came up with the following results for HTTPS/SSL encryption and HTTP Authentification in various RSS readers:
| HTTPS/SSL | HTTP Authentification | |
| NetNewsWire | Yes | Yes (through URL) |
| NewzCrawler | No | Yes |
| NewsGator | Yes | Yes |
| AmphetaDesk | No | No |
| FeedDemon | Yes | Yes (through URL) |
| Radio Userland | Yes | Yes (through URL) |
| SharpReader | Yes | No |
| Synderalla | No | No |
HTTP authetification alone isn’t sufficient to create true private feeds, as the password is sent as clear text. However, when HTTP authentification is combined with SSL, it becomes a usable security mechanism.
After posting my results in Yahoo Groups: Aggregators, I got several helpful responses from the developers of RSS readers. The consensus seemed to be that there were two problems. First, that not enough people use private RSS feeds to make it worth worrying about. And second, that there are no good resources for testing.
In response to the first problem, that not many people use private RSS feeds, I have two counter-points. By the very nature of private RSS feeds, it is difficult to know how many there are - people don’t advertise them. More importantly, there is a catch-22 here. People won’t produce private RSS feeds until they know the RSS readers support the security features.
As for the second problem, that there are not good testing resources, following my post in the Aggregators group, Danny Ayers has taken to expanding the testing results and created a permanent page dedicated to reporting Aggregator features like these.
For the sake of the developers, we’ve created four RSS feeds for testing purposes:
| HTTPS/SSL | HTTP Auth | |
| Plain old RSS (no authentification or SSL) | no | no |
| RSS with SSL, but not HTTP Auth |  |
no |
| RSS with HTTP Auth, but no SSL | no | |
| RSS with both SSL and HTTP Auth | |
|
| username/password for the HTTP auth-protected feeds is: testuser/testpass | ||
In the end, we found that there was enough support to proceed with feeds protected with both SSL and HTTP authentification. That said, many of the RSS readers only supported HTTP authentification by putting the username/password in the URL. Ideally there would be a more secure method for storing the password. Note to the makers of NewzCrawler: please support SSL, or I’ll have to use another reader, and I like yours.
I just tried the https url in SharpReader, and it loaded fine for me. If you got an error-message or stacktrace when you tried it, please email that to me so I can try and figure out why it did not work for you. Thanks!
(I helped develop MySmartChannels, so I’m quite biased of course, but it's still worth a look. ;-)
This is true of HTTP Basic Authentication. HTTP Digest Authentication does not send the password as clear text.
BTW, I just installed Newzcrawler and password protected a RSS feed with our product AuthentiX (http://www.flicks.com/prod.htm#authnx) . Newzcrawler works like a dream. I tried BottomFeeder but it choked on the authentication.
GET /local/solabs/rsstest/httpauth/rss_with_auth.xml HTTP/1.1
Host: labs.silverorange.com
User-agent: BottomFeeder/Development
Accept: */*
Authorization: Www-authenticate: Basic realm=rsstest
dGVzdHVzZXI6dGVzdHBhc3M=
Connection: Keep-Alive
and getting an auth failure. I'm unclear on what's wrong.
The basic idea is that you can schedule a job to retrieve the private RSS file regularly to your local hard drive and then point your favorite rs reader to that file.
If there is enough interest I can post instructions on my personal blog.
Recently I get it work with RSS Bandit also (http://www.rssbandit.org)
Awasu supports https and authenticated feeds - all four of your test feeds work fine with us.
Just tried it for the first time. It's the only aggregator that I can get to work with my corporate syndication feeds that are protected by Windows authentication.
Tried NewsGator and NewzCrawler on the same feeds and got "401 Unauthorized" however I set up the authentication.
This page is a wonderful resource, but I'd like to make a suggestion. At least one aggregator, FeedReader, appears to ignore the https:// specification and instead tries to fetch the given resource using http://. Because your HTTPS links also serve identical content when fetched with HTTP (such as http://secure3.silverorange.com/rsstest/httpauth/rss_with_ssl_and_auth.xml -- note that I changed the https to http), one will erroneously conclude that FeedReader supports SSL, which as far as I can tell it doesn't.
It may take a few server setting twiddlings to adjust this, but I recommend that you change your https links to *require* SSL, and serve 404s or obviously erroneous content if requests are received for them over HTTP.
Thanks again!
I am new to the Rss feed technology. This page is a wonderful resource for all of us. As you provide the link for "RSS with SSL, but not HTTP Auth". I also want to do same in my feed. Whats the step you follow to work the link rss feed to work properly. As when i try to add https url in the feeddemon i got the error message - "The newfeed url could not be auto-discovered. This site may not have a newsfeed, or it may not support auto-discovery of its newfeed."
Your tool is of interest as a screening device to limit spam. Your security system could be applied to comment-feeds: A blog-feed might be public, but the comment-feed could be by subscription-selection.
There's an anti-spam conference that you may be interested in: spamsummit. More discussion. Good luck with your projects.
I get these errors:
1. No SSL, but HTTP auth: invalid credentials, auth. required
2. SSL, but no HTTP Auth: HTTP 404: not found
3. SSL and HTTP Auth: HTTP 404: not found
Can you fix it, please?
Great article: A Primer for Publishers & Content Providers
http://www.eevl.ac.uk/rss_primer/
And yes, our RSS feeds are password protected. The information isn't super sensitive, but when you know exactly who the user is, you can do some interesting things. For example, you have a news feed and the customer is in your "international region". You can tailor the RSS contents just for that user.
Our feeds are all XML-based views from Lotus Domino. User authentication, XML views, "reader" security... it's all baked-in... and this made it incredibly simple to make user specific content for RSS.
I tried all the given RSS links with various security configurations within Acrobat 7 professional. And they all work like a charm!
A brandable "secure" rss aggregator with enclosure support with full online statistical control.
I am testing the 4 test links with Readers installed in my pc. They work.
But when i try to test http authentication test links with online readers like myyahoo, bloglines, newsgator online, they dont work. Some show the feeds but dont ask for username and pwd. Others throw, "No feed found". Can any body tell me the reason for this.
Thanks
Official website: http://www.rssxpress.net
And thank you Steven for this very interesting web page!
Worked fine for both :
Plain RSS
RSS with HTTP Auth, but no SSL
But with either one with the SSL - it give an error : Wrong Feed URL
any ideas?
See, when a client is a BROWSER, it can send BASIC/DIGEST authentication...
But i am wondering how does a RSS READER, is gets authenticated ? coz ITS NOT A BROWSER..right ?
so when we click from any RSS cleint, to a protected RSS page,it asks for a uname/pssword. and the RSS server keeps this info as a CACHE...So next time the RSS client(NOTE:NOT A BROWSER) it doesn't ask him for username/password.
HOW THIS IS HAPPENING..I AM REALLY CAN UNDERSTAND....
COZ,every time the RSS client has to pass some token or any god damn thing(cookie,authentication header,session id) to the RSS server..But it cant do so,coz IT IS NOT A BROWSER...
Then how it works ?
It's calling scraping and is theft!
I am trying to use the test RSS with HTTP auth/no SSL and the username/password do not seem to work. I have even tried testing the URL at http://feedvalidator.org/ and it still fails. I enter the URL in the form HTTP://username:password@domain.name/rest/of/path.xml
What am I missing?
Thanks for keeping this valuable resource online!
base64_encode(user_id . sha1(password . 'randomstring');
add this to the end of the .rss?auth=...
I tend to treat news like email. This comes from subscribing to list serves. So, I prefer to get my RSS and Atom in Thunderbird. I already have one feed that I would like to see secured, and this seems to be an area where some developement would be nice.
Thanks muchly
Great article though, thanks.
/B.
thanks in advance. please reply at the earliest....
Oil and retaining its nutrient content
Where I can find the new range of accessories with variety of designs
www.porntubebestmovies3.tk
