[rabbitmq-discuss] Cluster failover behaviour

Mon Dec 14 22:13:02 GMT 2009

Hi,

Can you please help me understand what is going on when I lose a member of a
cluster?
I'm seeing an odd behaviour that makes me worry. This queue is supposed to
hold
credit card transactions, so we _really_ want to avoid double-processing
messages.

Cluster consists of 2 members: (rabbit at mq1, rabbit at mq2).

1) client connects to mq1, creates durable queue, pushes 100 persistent
messages in
    root at mq1:~# rabbitmqctl list_queues
    Listing queues ...
    testqueue       100
    ...done.

2) mq1 rabbitmq is stopped

3) client connects to mq2, creates durable queue, pushes 100 persistent
messages in
    root at mq2:~# rabbitmqctl list_queues
    Listing queues ...
    testqueue       100
    ...done.

4) mq1 rabbitmq is started
    root at mq2:~# rabbitmqctl list_queues
    Listing queues ...
    testqueue       200
    ...done.

5) client drains the queue
    rborkows at mq1:~/client/bunny$ ./bunny-subscriber
    Drained 200 messages in 0.475371 seconds

6) queue is now empty
    root at mq2:~# rabbitmqctl list_queues
    Listing queues ...
    testqueue       0
    ...done.

7) mq1 rabbitmq is restarted, 100 credit card transactions pop back into
existence, unhappy customers, etc.
    root at mq1:~# /etc/init.d/rabbitmq-server restart
    Restarting rabbitmq-server: SUCCESS
    rabbitmq-server.
    root at mq1:~# rabbitmqctl list_queues
    Listing queues ...
    testqueue       100
    ...done.

The expected behaviour is that after step 5 the messages are _gone_ from the
queue, never to be seen again.
What's going on here?

-- 
Robert Borkowski
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20091214/8159bc19/attachment.htm 