Neil Turner's Blog

Blogging about technology and randomness since 2002

Decentralised Blogrolling – a thought barf

As you know, BlogRolling got hacked, and several bloggers (in particular Dave, Ken and Les) have suggested that they would want to use something similar, but perhaps on their own site – basically a decentralised system that doesn’t rely on one site. Therefore, if that sites messes up, you’re okay. So, here’s a thought barf on the subject – basically transferring my thoughts to the computer in a not-necessarily logical order. It’s also known as brainstorming or the more politically correct ‘mind-mapping’, since the former may be offensive to those who suffer from migraines, apparently. Anyway…

The main reason why more savvy bloggers use BlogRolling is because it can indicate which blogs have updated recently – otherwise, the only advantage it offers is that it’s a little easier to organise links, and anyone who is recently savvy with HTML will have no problems in that arena. So basically, we need a way of emulating BlogRolling’s most recently updated functionality.

There are two main ways I can think of doing this. The first is much like BlogRolling currently works – retrieving the changes.xml file from weblogs.com, or, perhaps hypocritically, the feed from fresh.blogrolling.com which includes sites which ping weblogs.com, blogrolling.com directly (through the Pinger) and recently updated Blogger blogs. The second method involves querying each blog individually to see if it has updated.

The advantage of the first method is that you don’t hit any of the individual blogs, so you don’t use up any of their bandwidth, but since not all blogs ping weblogs.com you’ll find some blogs never show as updated. This is particularly true for TypePad-powered blogs, I’ve found, and for Blogger users who don’t use a tool like w.bloggar which support pinging.

The second method does mean you are eating into a site’s bandwidth, but as long as you don’t poll every 5 minutes you probably won’t make too many enemies. 3 hours is more acceptable and what most aggregators now default to (don’t poll more often than 1 hour), but you could check for a <ttl> element and choose an appropriate polling time (see my ttl article). The issue is how to tell if the site has updated, and again there are two ways:

  1. Polling the home page to see when it was last updated. Simple, but, let’s say you’re polling a Movable Type blog – every time a comment is posted, or a trackback ping is received, that page is regenerated. Some people even have scripts linked into cron-jobs which refresh the home page every couple of hours, for various reasons (see mt-rebuild if that thing interests you). So, some sites will constantly show as updated, or show as updated even when they’re not.
  2. So we have the other option – using an RSS feed. This is an additional complexity because you have to provide two URLs when adding sites (and some sites aren’t too upfront about the location of their RSS feeds), but will at least allow you to track updates better. In effect, you’re adding an RSS aggregator to the site. Of course, this all goes arse-over-tit when you get a blog that doesn’t have an RSS feed (like most Blogger blogs at the moment).
  3. So there’s option 3, a hybrid solution. Using RSS feeds is probably best, but the first method could be employed as a fallback.

Either way, it’s quite easy to see from this why BlogRolling is popular – it’s able to do something that is very difficult to do yourself. Of course, if you don’t care about the ‘updated’ functionality then it’s possible that someone could put together a relatively simple method of creating link lists, and I believe that Six Apart put something like that in TypePad (so it may make it to Movable Type Pro if and when that ever comes out).

I hope this has provided food for thought for people who are looking into this. Of course, I’d love to have a solution at the end of this but I don’t, although I keep meaning to learn PHP so you never know.

3 Comments

  1. I disagree. I personally use Blogrolling.com to easily work with the hundreds of links I keep on my site. Do I need hundreds of links? Probably not, but I like to point to site that I find interesting or useful.
    Editing the links by hand is a royal pain, and I’m not always at the computer that I use to edit my website anyway.
    Using Blogrolling allowed me to maintain my blogrolls without having to remember to add the links into the HTML later.
    Les’s site has a comment about a script called MySQLinks. This sounds like the exact tool I am looking for. I’m going to be installing it later this week when I get some time and I’ll post about what I think about it after that. From what I can see, I’ll probably be switching over to it after I get it installed. It looks that good. 🙂

  2. I will have to check out MySQLinks. It sounds like the least amount of work, which is why BlogRolling is so nice, as Neil pointed out. For the time being I am just going to use MT, as I do not have to install anything else, and learn how that app works 😛