[rabbitmq-discuss] Weird Issue when cluster member restarts

[Arun Suresh]

Hello Mathew..

Thanks for ur reply.. was out of town so couldn attend to mail earlier.. my
comments inline...

On Fri, Nov 6, 2009 at 4:56 PM, Matthew Sackman <matthew at lshift.net> wrote:

> Hi Arun,
>
> On Fri, Nov 06, 2009 at 07:58:09AM +0530, Arun Suresh wrote:
> > My setup is as follows :
> > * 2 RabbitMQ (1.7.0 running on R13B02.1) servers in a cluster (say
> rabbit at m1,
> > rabbit at m2).. both disk nodes.
> > * 2 Clients (client at m1 connected to rabbit at m1, client at m2 connected to
> > rabbit at m2). I am using the Erlang client (which currently has an issue
> when
> > connecting to a server which is part of a cluster... this I fixed by
> setting
> > "insist = true" in the #'connection.open' record)
>
> Can you elaborate on this issue a bit? Is this redirection coming back
> from the server?
>
Yes.. I guess its a redirection request coming from the server..
basicaly.. the response to my 'connection.open' request is a
'conncetion.redirect' which on recipt, the client throws an exception and
exits..
ive gotten around this issue by setting 'insist = true' in the request
record...
Maybe ill look at the Java client and see whats done there ?

>
> > * both clients have processes that have subscribed to the same durable
> Queue
> > (Q) using the same key (K). Q is bound to a durable topic exchange (X). X
> > and Q are created on rabbit at m1.
>
> I don't understand what you mean by a queue with a key. I take it you
> have one topic exchange, one queue bound to the exchange with one
> binding key (is this the K you refer to above?), and two consumers on
> the queue. (From reading the rest of your email, yes this is clearly
> what you mean).
>
> > Now this is what i do..
> > * I bring down rabbit at m1, and client at m1 (They are co hosted... so both
> are
> > brought down together). I do a "rabbitmqctl list_queues" on rabbit at m2...
> I
> > do not see Q there..
>
> Ok, so the queue has been created on m1, not m2.
>
> > * I bring rabbit at m1 up again... After which I see Q when i do a
> "rabbitmqctl
> > list_queues".
> > * Before I bring up client at m2, I publish a messge to X with a routing
> key
> > K...
> > Issue  : Since client at m2 is up and has a process subscribed to Q, I
> expect
> > that process to receive the message... It doesnt..
>
> Indeed it won't.
>
> Clustering makes all exchanges and queues available on all nodes of the
> cluster. *However*, a queue is still located on a single node, even
> though it is accessible via all nodes. The queue will be created on the
> node the client is connected to when it creates the queue. Thus your
> queue has been created on m1, thus it's accessible from both m1 and m2,
> but when m1 goes down, the queue will go down with it.
>
> From that point on, m2 has no record of the queue, so msgs sent to X
> with K to go to Q will not ever reach Q - as far as m2 is concerned, Q
> just does not exist.
>
> When m1 comes back up, Q will be recovered.
>
> > Further more... I notice that when i bring client at m2 back up, It
> restarts
> > its subscribers... now all messages published to X with K are sent ONLY
> to
> > client at m2.
>
> Well client at m1 died. This means its subscriptions to Q were lost - there
> is no concept of persisting subscriptions, so when rabbit at m1 comes back
> up, the Q will have no consumers at all. If you restart client at m2 then
> yes, it'll recreate its subscription to Q, and thus receive all the msgs
> sent to that Q. If you also restart client at m1 then that too will
> recreate its subscription to Q and share the msgs with client at m2 as
> before.
>

Confirms what ive been thinking... But Is there any way for clients to be
notified of dead subscriptions ? That would be helpful, else i would have to
keep polling something to get that information..

>
> Hope this is of help.
>

Definitely.. thanx

>
> Matthew
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20091109/137fd0ef/attachment.htm