[rabbitmq-discuss] Possible clustering layout

[Jerry Kuch]

> From: "Adam Rabung" <adamrabung at gmail.com>
> Cc: rabbitmq-discuss at lists.rabbitmq.com
> Sent: Wednesday, October 19, 2011 11:16:24 AM
> Subject: Re: [rabbitmq-discuss] Possible clustering layout
>
> Don't worry, I'm definitely the confused one :)

Hi, Adam... no worries.  The language around the words "clustering"
and "HA" in the Rabbit world is tricky and has a complex history. :-)

> I'm having a hard time finding info on the Active/Active HA in
> 2.6. The FAQ and Clustering Guide led me to believe this feature
> didn't exist yet. Do you know where I can find good info on this new
> feature?

It's pretty much brand new, having appeared in the most recent
releases.  You can read details here:

http://www.rabbitmq.com/ha.html

This describes the mirrored queue functionality and how to request and
specify mirroring for your queues.

Release notes for RabbitMQ 2.6.1 are here:

http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20110912/35da167c/attachment.txt

The active/active, mirrored queues flavor of HA actually first
appeared immediately previous in 2.6.0, with release notes here:

http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20110831/36015373/attachment.txt

Look for the word "mirrored" in that write up.

> I'd say our main goal is horizontal scalability for performance
> sake. We've come to the conclusion that we can RAID disks, and if a
> queue node goes down, we can bring up a new node with the old disk in
> human-scale times.
>
> What do you think? Thank you for any feedback.

The new active/active, mirrored queue HA is something you likely want
to investigate.  The older RabbitMQ clustering for capacity involved
building a cluster of RabbitMQ nodes, although for implementation
reasons each queue really only existed on one of the nodes, and would
become unavailable with that node's failure.  Thus, you could get greater
throughput overall, but not greater throughput through any particular
queue, which is probably roughly the type of horizontal scalability
you're after.  On the downside, with clustering alone you'd be
vulnerable to that queue's host node failing.  The remedy in those
times was to back queues by shared storage (preferably a decent SAN
solution or the like) and run a hot standby of each node, using
something like Pacemaker to detect failure and do the switch over from
passive to active.  Should you prefer that approach to the new one,
you can still do it, but the new model has fewer moving parts to set
up.

Also, be mindful that with the new active/active HA being so recently
released, you'll have to apply a careful eye to any documentation you
read, since most of it will still refer to the old model.

Best regards,
Jerry