Download Feeds avoiding Feedburner redirects

When trying to access the original feed of a blog (or website) which makes use of Feedburner, your crawler ( or browser or script ) usually receives a 301 header, redirecting it to the Google version of the feed. This means that if you need to access the original source ( for faster aggregation, to deal with a known rss syntax or whatever) you simply can not!

You can avoid the redirect and have complete access to the source using a simple UserAgent hack which can be achieved both via your scripts and browser. Below are some examples for common programming languages and media. The following hack has been tested with WordPress Feedburner plugin and Blogger integration, but should work even with home-made redirects if they user the user-agent as discriminant function.

Php using Curl

<?php
	/* Initializing curl object */
	$ch = curl_init(); 
	curl_setopt($ch, CURLOPT_URL, 'http://sourceblog.com/feed/'); 
	curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); 
	curl_setopt($ch, CURLOPT_TIMEOUT, 5); 
	/* Setting custom useragent */
	curl_setopt($ch, CURLOPT_USERAGENT, 'FeedBurner/1.0 (http://www.FeedBurner.com)');
		
	$res = curl_exec($ch); 
?>

Perl

#! /usr/bin/perl 

use strict;
use LWP::UserAgent;

my $ua = new LWP::UserAgent;
$ua->agent('FeedBurner/1.0 (http://www.FeedBurner.com)');
my $req = new HTTP::Request GET => "http://sourceblog.com/feed/";
my $content = $ua->request($req)->content();

Bash using Wget

wget --header="User-Agent: FeedBurner/1.0 (http://www.FeedBurner.com) " http://sourceblog.com/feed/ -O feed.xml -o /dev/null

Firefox

To set a full custom useragent on Firefox I suggest you to use Modify Headers extension. Please refer to their documentation section in the help tab located inside the extension window and remember to set the useragent to

FeedBurner/1.0 (http://www.FeedBurner.com)

Thanks @chrisvoo for suggesting the extension.

Chrome / Chromium

To change the full useragent on Google Chrome / Chromium, I followed this full guide , obviously changing the Iphone useragent with feedburner one (see above section).

Side notes

Please note that if someone creates a redirect to feedburner this usually means he doesn’t want you to connect directly to his webserver. This decision might have many valid reasons: too much load on the server, the needing of precise stats and so on.

Having said that, always make sure you have the rights to download an original source and ask the webmaster for permission if you need to put this hack on a cronjob running several times!