[rabbitmq-discuss] Cluster Pathology

Mon Feb 9 20:45:42 GMT 2009

To follow-up. In playing around with Setup B some more. After Producer
1 recreates the queue on node B, node B reports 0 consumers on the
queue. So it appears that when the queue died on node A, consumer 2
was not notified that the queue died, and also node B did not migrate
his subscription to the recreated queue of the same name on node B. As
a result, consumer 2 is hung in limbo listening to a non-existent
queue.

-J

On Mon, Feb 9, 2009 at 1:29 PM, Jason J. W. Williams
<jasonjwwilliams at gmail.com> wrote:
> Hi Y'all,
>
> I think I've stumbled on a rather pathological cluster behavior that
> may be related to the fact that queues aren't recreated (w/o messages)
> on other cluster nodes post failover.
>
> Setup A:
>
> * Consumer 1 attached to MQ node A and creates queue and binding.
> * Consumer 2 attaches to MQ node B and creates queue and binding (same
> as Consumer 1 and therefore no-op'd).
> * Producer 1 attaches to MQ node B (it also creates queue and binding
> same as Consumer 1...no-op'd) and publishes messages. Connection to MQ
> node B is persistent.
> * Consumer 1 dies.
> * MQ node A dies. Queues are not recreated on node B, and produced
> messages are black holed. Queues are not re-created because Consumer 2
> and Producer 1 are not notified by node B to reconnect or any other
> error (therefore their reconnect/recreate queue code never gets
> triggered).
>
> Setup B:
>
> * Same as Setup A except:
> * Producer 1 attaches to MQ node A.
> * MQ node A and Consumer 1 fail. Producer 1 reconnects to node B and
> recreates queues and bindings. Messages Producer 1 publishes are
> placed in the recreated queue. However, Consumer 2 never is handed
> messages by node B (which it has been persistently attached to) after
> the recreation of the queue.
>
> Any insight is greatly appreciated.
>
> Best Regards,
> J
>