Blog of Jonathan Jaikoo Conway, web technologist, entrepreneur and daddy.

This entry is a bit late as I actually got back from RailsConf last week, but I’ve been trying to shake off a rather nasty illness and trying to do some work at the same time, not with much success.

I posted some images on the New Bamboo Flickr account. Pablo took a lot more and better quality images so I’ll try and borrow those when I have time and upload those as well.

It was great to meet so many new faces and old friends while also experiencing some excellent german beer. The only downer I had about the whole event was the fact that I was foolish to fly from Luton rather than Stanstead. Luton airport really is the worst airport I’ve had the misfortune of traveling from in a while. Next time I’ll be using Stanstead.

If you’ve emailed me in the past two weeks I’m slowly getting through the back log now so don’t worry if you haven’t had a reply yet, it’s coming!

Well, I’m just getting ready to leave for RailsConf in Berlin. I’m actually a bit ill at the minute so hopefully I’ll have perked up by the time the conference starts. Packing lots of parecetomol just in case. See you all there!

With Web 2.0 applications facing growing audiences with data hungry appetites MOM has suddenly had a resurgence.

Messaging systems are a great way of scaling your site by placing operations/messages on a queue and processing them asynchronously. This allows you to get quite imaginative with your architecture and distribute worker processes in geographically disparate locations if you so wished and also distribute/delay the actual processing needs as and when you can.

This can be used with great effect with services such as Amazons own simple messaging offering known as SQS and EC2. As soon as load starts peaking on one EC2 node you can have a script that calls out to a mother-ship to instantiate another EC2 instance and as soon as it’s ‘born’ starts processing messages off the SQS queue. If you wanted you could even have the original load stricken node contact the mother-ship via SQS (probably a more reliable solution). This is one of the architectures I’m currently advocating as EC2 is stateless and rather than worry about database replication across transient nodes, I try and share as much information as I can using messaging and web-services. I’m currently backing off exposing REST style interfaces for non public facing services purely for performance and scaling reasons. Don’t get me wrong, I love REST and for public a public facing API it makes life a lot easier. But as Zed Shaw says, HTTP isn’t exactly the most performant protocol.

I’ve seen people try and achieve similar architectures using a database and their own cobbled together mesh of scripts and while this may work great on small sites, you’re just pushing your scaling issues and problems onto your poor DB. And as anyone who’s had the misfortune to be stuck in a room with me would tell you, I hate relational databases on multiple fronts, they’re so 1980’s and I’ll tell you why in another blog post.

It’s seems these days that we’re currently quite spoiled with the cheap/open messaging solutions that are available. A few years ago most of what I had to play with were proprietary and expensive offerings from various large vendors. A few that I’ve played around with lately have been:

• ActiveMQ – An open source solution utilising STOMP. Check out ActiveMessaging for Ruby/Rails.
• Jabber – While it’s not really intended for reliable delivery of messages, it does have a lot of support and with quite a few Ruby clients available.
• SQS – Amazons offering that is easy to plugin solution if you’re using EC2 as you don’t need to worrying about setting up the messaging infrastructure between distributed nodes that aren’t necessarily sharing a VPN. ActiveMessaging has support for SQS though I kinda feel it’s bit of a chain saw to a bunny rabbit as SQS really is a very simple service as it’s name suggests. I wrote my own lightweight SQS Rails plugin a while ago and will probably dust it off in the next week or so.
• RabbitMQ – Now this is the matrons knockers when it comes to kick ass, ultra fast and scalable messaging. It simply rocks, with performance off the hook. It’s written in Erlang and supports the AMPQ protocol. Unfortunately I haven’t seen much in the way of Ruby support. I’ve played with QPid’s (Apaches own open source AMPQ offering) Ruby client library but haven’t actually tested this against RabbitMQ, so you’re mileage may vary.

So there you have it, messaging. It rocks, it’s sexy and what’s more it’ll make you sexy… Honest guv’nor.

It looks like Transmit just got even better. I’d previously been using S3Fox to manage my S3 buckets but now my favourite SFTP/FTP app now has built-in support.

Like all the Panic products, they’ve not just bolted this feature on without any thought and have added another polished feature that so far hasn’t fallen down on me yet. If there was a reason to upgrade to version 3 from 2 of Transmit it looks like this could be it.