[rabbitmq-discuss] RabbitMQ mirrored queues and node failure

Wed Nov 16 21:44:45 GMT 2011

Hey there,

I'm looking at using the mirrored queue functionality in RabbitMQ cluster and I had a question about node failure and what happens to the messages in the queues.

Here's an example: consider you have node A, B, and C (all disk nodes) and a single durable mirrored queue.  They start like this:

node A: 10 messages in the 'foo' queue
node B: 10 messages in the 'foo' queue
node C: 10 messages in the 'foo' queue

Let's say node C is in a rack that loses power and it drops offline.  We end up with this:

node A: 10 messages in the 'foo' queue
node B: 10 messages in the 'foo' queue
node C: offline

While repairs are being made to C's rack, consumers still talk to nodes A and B to pick up and drop off messages.  Let's say the queues on A and B are reduced to zero several times while C is offline and then C comes back online:

node A: has a bunch of messages node C has never seen
node B: has a bunch of messages node C has never seen
node C: back online with the 10 messages it had before it failed?

What I'm interested to know is:

1) Does C come back up with the 10 messages that it had on it before it failed?
2) What happens to those 10 stale, old messages that node C had on it before it failed (that were already picked up from nodes A and B by consumers)?

Thanks in advance for the the help and I apologize if I misunderstood the documentation[1]!

[1] http://www.rabbitmq.com/ha.html#behaviour

--
Major Hayden
major at mhtx.net