[rabbitmq-discuss] Java client: channel shutdown notification while closing connection

[Jiri Krutil]

Matthias,

> The shutdown listeners get called in the main connection thread. That 
> means they should not perform any possibly blocking AMQP operations. 
> If they do then a deadlock can ensue.
...
> Why are you calling connection.close() at all? The connection will 
> already have been closed. After all, you are in the connection 
> shutdown handler code.

To achieve some kind of resiliency to short network and broker downtimes 
in our application, we have a synchronized disconnect-and-start-over 
method that is used as both connection and channel shutdown listener. If 
this fails we retry in regular periods. The goal is to make sure that 
the application will reconnect and re-register its consumers as soon as 
possible.

This is used in part due to the (in my opinion unfortunate) feature of 
AMQP that an error in an operation on a channel closes that channel.

To keep things simple, we close the connection in this method, in case 
it would be invoked by a channel shutdown. And that's where our problems 
seem to appear: when closing the connection, channels are closed and 
their shutdown listeners invoked, which in turn attempt to acquire the 
same lock, resulting in a deadlock.

We seem to have solved the problem by explicitly unregistering the 
channel shutdown listeners before closing the connection.

Cheers
Jiri