[rabbitmq-discuss] Shutdown Listener on Java Client API

[Dave Greggory]

We had something similar happen again last night. For some reason, one of our 
producers lost the connection to RabbitMQ at 7:47 PM but the shutDownCompleted 
event did not fire on that producer till 8:58 PM. We have a separate thread to 
send out messages from our app so that the app is unaffected should it lose 
messaging functionality. Starting at 7:47 PM, this separate thread started 
blocking and we lost all messages sent beyond its internal buffer's capacity. At 
about 8:58 the shutdown event fired, we automatically reconnected and everything 
returned to normal.

Both of our RabbitMQ nodes stayed up the whole time. No errors or VM HIGH 
WATERMARK events in the logs. This is on RabbitMQ 1.8.1 in our production 
environment. This is a very low volume environment, RabbitMQ nodes were only 
using about 30 MB of the 993 MB it allocated itself. No CPU / Memory / Disk 
spikes.

Any thoughts? Is there something in the java client api that could prevent the 
shutdownEvent from firing quickly? AMQP Heartbeats were mentioned in another 
thread, would that help and how do we enable these?



----- Original Message ----
From: Vlad Alexandru Ionescu <vlad at rabbitmq.com>
To: Dave Greggory <davegreggory at yahoo.com>
Cc: rabbitmq-discuss at lists.rabbitmq.com
Sent: Wed, September 15, 2010 7:51:27 AM
Subject: Re: [rabbitmq-discuss] Shutdown Listener on Java Client API


Hi Dave,

AFAIK, the event should fire regardless of the shutdown reason (be it 
connection loss, server initiated connection close, sudden socket 
closed, heartbeat timeout etc.). It appears to me that you are using it 
correctly.

Depending on the type of load balancer you are using, perhaps the 
clients never got disconnected? It is also possible that there is a bug 
in the client code.

Rgds,
Vlad.


On 09/13/2010 04:12 PM, Dave Greggory wrote:
> We register a shutdown listener to listen for connection-loss events and
> automatically re-connect clients to other RabbitMQ nodes in the cluster. This
> weekend we had to restart the load balancer that sits between rabbitmq and 
it's
> clients for some maintenance, but the 
ShutdownCompleted/ShutdownSignalException
> event never fired in the clients.
>
>
> I'm trying to understand how the shutdown signal works. Does it only fire the
> ShutDownCompleted event if RabbitMQ node shuts down and sends the signal back 
>to
> clients? If that's the case, I'm probably misusing it to detect 
>connection-loss.
> Is there a different way of detecting connection loss?
>
> Thanks,
> Dave
>
>
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>