Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

Modify RSS feeds on the fly via a Perl script Internet
I use Safari to subscribe to a bunch of RSS feeds, among them VersionTracker and MacUpdate. The VersionTracker feed was often very slow to load, and I realised it was because every item in the RSS includes two images to be downloaded (even though the images were 1x1 pixels, the extra requests slowed down the entire process a lot). I therefore created a perl script that acts like a webserver on localhost (127.0.0.1) port 8080. Upon a request, it fetches the RSS from VersionTracker and strips the images from it before sending the resulting RSS to Safari (or whatever program that requested it in the first place).

I set up launchd to keep the script running at all times (using Lingon). I also made the perl script change the RSS from MacUpdate. The original feed only contains a rather long title for the items, and no descriptions. I changed this so that only the name and version number is the title, and the rest of the title (that actually is a description of the program) is the description. This made it a lot easier to read in my opinion.

The script is made to work, and I suppose it can be optimised and improved in many ways, for example to allow multiple connections at once. I'm also quite sure that the extra controls to check that the calling IP is 127.0.0.1 are redundant, since I beleive only 127.0.0.1 can connect to 127.0.0.1. The script uses the modules Sys::Syslog, Socket, and LWP::Simple, and these may need to be installed on your system before it will work.

Unmodified, the script listens to the URLs http://127.0.0.1/versiontracker and http://127.0.0.1/macupdate, but I would suggest that you look through the script and add or remove URLs to your liking.

[robg adds: I haven't tested this one.]
    •    
  • Currently 2.29 / 5
  • 1
  • 2
  • 3
  • 4
  • 5
  (7 votes cast)
 
[8,059 views]  

Modify RSS feeds on the fly via a Perl script | 5 comments | Create New Account
Click here to return to the 'Modify RSS feeds on the fly via a Perl script' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
much of this should be possible with Privoxy
Authored by: hayne on Dec 04, '06 10:46:47AM
Much of this RSS-feed manipulation should be possible to do via Privoxy (http://www.privoxy.org/) which is a web proxy that runs on your Mac in the same way as the Perl script described in this article.

[ Reply to This | # ]
Mac Aggregate Tracker
Authored by: ttt on Dec 04, '06 09:32:57PM
While this is generally a good hint for playing with RSS feeds, for the specific task of Mac software I'm surprised more people don't know about MAT. Particularly useful is the de-duping of software that is listed on multiple sites.

[ Reply to This | # ]
Modify RSS feeds on the fly via a Perl script
Authored by: dustpuppy0001 on Dec 05, '06 07:22:15AM
I really like this Idea to modify feeds on the fly. I have rewritten your script for my needs, perhaps you find it useful. This version uses IO::Socket and IO::Select for the networking functions and should in theory allow concurrent access (not tested).

rss_proxy.pl - The actual proxy
RssProxyConfig.pm - Configuration module (a central place to configure your feeds and modifications).

cheers

[ Reply to This | # ]
Modify RSS feeds on the fly via a Perl script
Authored by: fredrikw on Dec 11, '06 02:00:42PM

Thanks a lot for that much-needed cleanup of my code... It's really interesting to see one's code reworked!

Regards,
Fredrik



[ Reply to This | # ]
Modify RSS feeds on the fly via a Perl script
Authored by: bex on Jan 13, '07 05:43:02AM

I have a similar program I use for ical feeds. This way you can strip events that are not of value to you (like Away games from a sports schedule). If anyone needs it, drop me a line as it isn't posted online yet (I love endless site redesign).



[ Reply to This | # ]