[rabbitmq-discuss] of queues, memory and heartbeats...
anthony-rabbitmq at hogan.id.au
Tue Sep 1 08:36:02 BST 2009
We're in a particular situation where occasionally a client will be
disconnected from the server (often through some unclean network break, such
as the random stuff that can happen with 3G data links and virtual
machines), but the queues associated with that client don't die and continue
Eventually some of the queues grow so large, that we start seeing txamqp
(the client side library we use) start whining that it doesn't know what to
do with "channel_flow" - presumably this is the memory based throttling
kicking in (though I don't recall explicitly enabling it - unless it's
something auto-enabled by default in the .deb packages?)...
Questions would be:
1. Even though we haven't explicitly enabled it, if we do start to see
channel_flow messages, this would be the memory based throttling, correct?
2. Besides rabbit noticing when a TCP connection dies - is there any
"heartbeat" functionality built into rabbit and/or the AMQP spec that would
actually test whether or not a connection is active, and if it isn't, kill
the relevant queues? Do I as the person who monitors the server configure
this, or do the programmers who write our clients need to incorporate
something into their code?
3. In calculating the "5%" of memory available threshold, what would
rabbit be considering memory.. Real? Swap? Virtual? All of the above?
# cat /proc/meminfo | grep -i "total\|free"
MemTotal: 514056 kB
MemFree: 144860 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 514056 kB
LowFree: 144860 kB
SwapTotal: 1044184 kB
SwapFree: 838244 kB
VmallocTotal: 442360 kB
As a failsafe against runaway queues, I'm thinking of implementing something
that culls queues over a given order of magnitude on a timed basis, or
triggered by a lower memory threshold than rabbit's..
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the rabbitmq-discuss