Loosely coupled, distributed, even driven, asynchronous - all these are terms that you will hear when you want to build an application that is scalable and handles large amounts of work load as if nothing happened, no sweat. There are many ways of building these, but when you talk about an application distributed across servers and talking to each other, the obvious mechanism is messaging - whether you send your own message or even using simple sockets like with Twisted or if you use a full blown broker like RabbitMQ. Even if you use something like Celery, you will have to choose a messaging backend which you want to use.

And this choice is the problem - how to evaluate the various options?

The choice is not governed by external factors, although things like licence, cost, support etc make an impact on your final decision. The choice is primarily driven by what your application is built to do, whether it wants to handle only the functionality part and wants the underlying messaging layer to provide guarantees about message delivery and caching, querying etc.

This talk will distill my experience with seeing, building and designing such messaging applications and provide a methodology that you can use.