[rabbitmq-discuss] Memory limit cleanup, hanging connections.

[Simon MacMullen]

On 15/10/12 10:49, Irmo Manie wrote:
> Hi,
>
> We ran into a situation where we hit a memory limit by queues being
> filled upt, causing all connections to be in a blocked or blocking state.
>
> We're killed all connections from the client side and deleted the filled
> up queues.
> While the broker is still cleaning up /msg_store_transient (which is
> horribly slow), it also still lists all the connections as alive, being
> still blocked.
>
> Is it possible that when the broker is still in memory alarm mode, the
> connections (blocked) are not checked for their heartbeat and therefore
> are kept alive?

Yes, that's absolutely correct. The blocked connections don't perform 
any network operations at all - which means that not only don't they 
check for heartbeats, they don't even detect the TCP connection being 
torn down. As soon as the memory alarm clears, the connections will 
unblock, notice the TCP connection having gone away, and vanish.

> And what is the cause of the slowness of the cleanup when deleting
> queues with only transient messages?

If you've "filled up" all your queues then they will have been paging to 
disc for some time - cleaning all that up can be quite slow.

Cheers, Simon

-- 
Simon MacMullen
RabbitMQ, VMware