[rabbitmq-discuss] Mirrored queue failover

[Katsushi Fukui]

Hi,

I'm a RabbitMQ newbie and am trying to use mirrored queue in the cluster (3 disc nodes on the 3 servers). I use RabbitMQ 2.8.1, Erlang R15B, CentOS 6.2. I wanted to see the behavior of the failover in the case of any issue, so I tested some scenarios and I have a little confusion with the case below:

a) 3 nodes (rabbit1, rabbit2, rabbit3) are running, and I created que1 on the rabbit1 with an argument "x-ha-policy":"all". So now que1 have a master node "rabbit1 and slave nodes "rabbit2" and "rabbit3", I confirmed it with the command "rabbitctl list_queues name durable pid slave_pids synchronised_slave_pids".

b) I stopped one of the slave node "rabbit2" with "rabbitctl stop". Now que1 has only one slave node "rabbit3". 

c) Next, I started the node "rabbit2" using "rabbit-server -detached" and checked the cluster status with "rabbitctl cluster_status". It seems good 3 nodes has 3 disc nodes and running. But when I checked the queue status, que1 has just one slave "rabbit3". I restarted rabbit2 again, but rabbit2 doesn't become a slave of que1, also I can find a log like this:
=ERROR REPORT==== 29-Mar-2012::15:12:40 ===
Discarding message {'$gen_call',{<0.1876.0>,#Ref<0.0.0.8584>},info} from <0.1876.0> to <0.426.0> in an old incarnation (2) of this node (3)

In my understanding, when a new node is added in the cluster, that node will become a slave of the mirrored queues in that cluster. Is this correct?

Thanks in advance,
Kats