[rabbitmq-discuss] rabbitmq cluster failover for persistent queues

Mon Jul 27 01:00:56 BST 2009

Alexis,

Can you clarify what you mean by "failover of session state" please?
> I am assuming you mean that when a node dies, then another node can
> 'take over' from it, without loss of any state that the first node has
> committed (to some other node and/or persistent store), and that the
> second node can 'take over' in some sense 'faster' than recovery by
> restarting the first node from disk would.

More or less.  I'm personally not that concerned with the timeliness of
recovery for committed messages.  I'm mostly concerned with any messages
being dropped by the system because no bindings matching a particular
message's routing key existed in the exchange at the time of the enqueue
command (because the one node which contained the queue which matched that
routing key has suffered a disaster).  To go further I really want the
ability to define something like "failover bindings" (e.g. create queue Q on
node A as master and node B for failover)  in the exchange such that if
delivery to to a queue "Q" on node "A" fails then the exchange will declare
that node "B" is the new master for queue "Q" and all subsequent routing
will be adjusted.  It seems that if the persister were using mnesia to
record session state (queue data) and one could define "failover bindings"
like I mentioned above then the exchange could failover a queue to a
secondary node transparently to all producers and consumers in the system,
save for the ones that were directly connected to the node which suffered
the disaster.

>  I ask because this case is
> not quite the same as 'exactly once' delivery.

Yes, I was just referring to my belief that if I want to eliminate this
single point of failure I would need to create redundant queues on separate
nodes and have some kind of out-of-band mechanism to guard against duplicate
processing (or that I would need to make sure that all downstream operations
were idempotent).

>
>
> Cheers,
>
> alexis
>
>
>
>
>
> On Sun, Jul 26, 2009 at 6:45 PM, charles
> woerner<charleswoerner.lists at gmail.com> wrote:
> > Thanks Arvind.  That seems to confirm our belief that rabbitmq doesn't
> > handle failover of session state at the moment.  So in the case of
> competing
> > consumers taking messages from a rabbitmq broker cluster it sounds like
> > once-and-only-once delivery is somewhat up to the application to
> implement
> > by arranging for each message to be delivered to redundant queues (ie. 2
> > separate queues with similar bindings residing on different hosts), then
> > coordinate among your consumers to ensure once-and-only-once delivery
> using
> > a database or simply to make your workflow idempotent with respect to the
> > duplicate messages.
> >
> > Ideally, I'd like my broker cluster to handle this.  I'm looking into
> apache
> > qpid as it seems to be more mature with respect to availability.  The
> main
> > downside with rabbitmq for me right now is the lack of at least an option
> > for session state replication.  It sounds like a future release of
> rabbitmq
> > with pluggable persisters will make distributed session state possible
> via
> > mnesia.
> >
> >
> > _______________________________________________
> > rabbitmq-discuss mailing list
> > rabbitmq-discuss at lists.rabbitmq.com
> > http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20090726/c09eb7be/attachment.htm 