[rabbitmq-discuss] Node/Broker failover and recreating queues on another node

[Matthew Sackman]

Hi Matthew,

On Tue, Mar 02, 2010 at 08:57:45AM +1030, Matthew Dunn wrote:
> *It's OK for a few messages to be dropped when a broker goes down
> *I need to load balance these messages
> *Availability is more important than dropping the ocassional message
> *I would prefer messages only be proccessed once.

Is this in fact rather similar to what the standard MySQL master/slave
HA setup is - i.e. async from the master to the slave, but instant
availability of the slave when the master fails, and the slave is
(pretty much) unusable up until the master fails?

> If the broker hosting the public queue crashes or is unavailable. Each
> consumer will connect to a new available broker and recreate the queue there
> with the same name.

Your entire plan can be implemented using the new rabbitmq-shovel
plugin. I would recommend you try this out as it was written for exactly
these situations.

> The question I have is, if I have created a public queue on a different
> broker, when the crashed broker comes back online will there be a problem
> with conflicting queues?

No, provided you don't use clustering.

> *Broker A with Queue A Crashes
> *Consumers failover to Broker B
> *A consumer recreates Queue A on Broker B
> *Broker A is restored -- Will Broker A have problems starting because Queue
> A has been recreated elsewhere?

I would suggest a slight variation. Brokers A and B are both up. The
shovel connects from your local "leaf" broker to both brokers, creating
queues and consuming from both queues. The publishers can then publish
to either broker and the messages will get through to the consumers, who
are consuming via their local leaf brokers.

In event of failure, hopefully the queues on A and B will be pretty
short, if not empty, so you shouldn't lose much. The shovels will
continue, and as soon as the failed node comes back up, will reconnect
to the node, recreating the queues and consuming as necessary. The
publishers simply need to have some logic to be able to switch between
either broker if they find a node has failed.

Is that sufficient for your needs?

Matthew