[rabbitmq-discuss] Node/Broker failover and recreating queues on another node

[Matthew Dunn]

Hi Matthew,

Thank you very much. That was very informative and has given me plenty to
think about for now.

I will be engaging in some prototyping of this over the next few weeks.

Thanks again,

Matt.

On Tue, Mar 2, 2010 at 11:13 PM, Matthew Sackman <matthew at lshift.net> wrote:

> Hi Matthew,
>
> On Tue, Mar 02, 2010 at 08:57:45AM +1030, Matthew Dunn wrote:
> > *It's OK for a few messages to be dropped when a broker goes down
> > *I need to load balance these messages
> > *Availability is more important than dropping the ocassional message
> > *I would prefer messages only be proccessed once.
>
> Is this in fact rather similar to what the standard MySQL master/slave
> HA setup is - i.e. async from the master to the slave, but instant
> availability of the slave when the master fails, and the slave is
> (pretty much) unusable up until the master fails?
>
> > If the broker hosting the public queue crashes or is unavailable. Each
> > consumer will connect to a new available broker and recreate the queue
> there
> > with the same name.
>
> Your entire plan can be implemented using the new rabbitmq-shovel
> plugin. I would recommend you try this out as it was written for exactly
> these situations.
>
> > The question I have is, if I have created a public queue on a different
> > broker, when the crashed broker comes back online will there be a problem
> > with conflicting queues?
>
> No, provided you don't use clustering.
>
> > *Broker A with Queue A Crashes
> > *Consumers failover to Broker B
> > *A consumer recreates Queue A on Broker B
> > *Broker A is restored -- Will Broker A have problems starting because
> Queue
> > A has been recreated elsewhere?
>
> I would suggest a slight variation. Brokers A and B are both up. The
> shovel connects from your local "leaf" broker to both brokers, creating
> queues and consuming from both queues. The publishers can then publish
> to either broker and the messages will get through to the consumers, who
> are consuming via their local leaf brokers.
>
> In event of failure, hopefully the queues on A and B will be pretty
> short, if not empty, so you shouldn't lose much. The shovels will
> continue, and as soon as the failed node comes back up, will reconnect
> to the node, recreating the queues and consuming as necessary. The
> publishers simply need to have some logic to be able to switch between
> either broker if they find a node has failed.
>
> Is that sufficient for your needs?
>
> Matthew
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20100303/e9c8b6f8/attachment.htm