[rabbitmq-discuss] queue gets removed from a node

[Matthew Sackman]

Hi Cezary,

On Tue, Aug 30, 2011 at 04:06:03PM +0100, Cezary Siwek wrote:
> >>The consumer script on Node1 creates a durable queue and waits for
> >>messages.

> I have reproduced the issue by breaking up the link between two
> nodes. The queue on node2 gets deleted when the node1 becomes
> unavailable.

No, I think you're confused. A queue only exists on one node, and that
node will be the node to which the client was connected. So if your
consumer created the queue on Node1 then Node2 is forwarding msgs
destined for that queue to Node1. Thus it's not a question of the queue
being deleted, more a question of the queue becoming un-findable once
Node2 loses contact with Node1.

> I know that rabbitmq cluster is not designed to work on unreliable
> links but is it a correct behavior when a queue gets deleted from
> the node? At least the node should allow producer to re-create the
> queue and keep all the messages until connection between nodes is
> back up.

We used to allow this, but it's wrong: your queue still exists on Node1,
it's just that Node2 can't see Node1 any more. If the producer was
allowed to recreate the queue on Node2 then when the cluster reforms,
you'll then have two queues with the same name, which then leads to all
sorts of crazy problems. Thus we forbid it.

> Is there any way to re-establish connection between two nodes when
> the link goes up?

If it is just a network issue and both Node1 and Node2 are really still
up, then I'm not sure if Erlang itself attempts reconnections, and this
may well be the problem (and hence why the stop_app/start_app helps). We
might need to add something to help them re-see each other. Essentially
I think we want to try running a "net_kernel:connect_node(Node)"
somewhere, but quite when, or where, or even whether it's really the
solution, isn't yet clear.

Matthew