[rabbitmq-discuss] RabbitMQ 2.6.0 is released

[Matthew Sackman]

Hi Jason,

Thanks for your input.

On Wed, Aug 31, 2011 at 03:34:16PM -0600, Jason J. W. Williams wrote:
> Personally what I'd like to see is the ability to create "cluster
> groups" to which nodes could be assigned when they are joined to a
> cluster. These groups would then be the basis of "rack-aware" or
> "datacenter-aware" decisions. It would make a lot more sense in our
> environment to tell queue.declare, set the replication-factor to 4
> (example) and spread the copies across the nodes in rack1_group and
> rack2_group (letting RabbitMQ figure out the best way to do that given
> current loading and node-presence).

The problem there is where do you stop? You're programming into your
group definition the fact that you know that these nodes are in
different racks, behind different switches, on different machines,
possibly on different UPSs. One problem is that Rabbit knows none of
this. If you were doing this in a cloud, or on EC2, you have no such
knowledged. The last thing you want to end with is all your queue
replicated between nodes that turn out to be different guests on the
same physical machine! Where do you think Rabbit should draw the line in
terms of load balancing - what should it try to monitor and what is
considered unreasonable? Even monitoring CPU usage is rabbit looking
around at its host, but should it go further and try to detect
hypervisors? From TTLs on network packets try to detect whether other
nodes are under the same switch etc?

It all gets very very messy rather quickly. One of the things we've not
really looked into is what do other bits of software that can cluster
like Rabbit do in these scenarios? What's actually useful? What do
admins expect to have to control themselves, and what do they trust the
software to be able to figure out?

Matthew