[rabbitmq-discuss] RabbitMQ 1.6.0 Memory Usage

[Matthias Radestock]

Gavin,

Gavin M. Roy wrote:
> I wanted to get some feedback on what I am noticing with Rabbit and
> memory use.  In this test case, I loaded up 4 million messages that were
> roughly 12 bytes or so of data, not counting protocol overhead.  At
> various stages of processing I had everything from 0 consumers to 80
> consumers running against RabbitMQ using basic_consume.  The machine
> that Rabbit is running on is dedicated to Rabbit and Alice
> (http://github.com/auser/alice)
> 
> Some odd things:
> 
>     * In the 6am hour where you see swap spike (red in graph), consumers
>       stopped consuming.  I could not find out at the time if it was
>       client or Rabbit that stopped dispatching messages.  Restarting
>       consumers resumed the previous rates of message consumption.

Next time this happens I suggest you gather some diagnostics with
  rabbitmqctl list_queues name messages_ready messages_unacknowledged
consumers

If the consumer count is 0 then there are no consumers. If you have a
consumer count above 0 and a messages_ready count above 0, and the
broker appears to be idle, then that is a good indication that the
consumers are there but stuck and the tcp connection to them is backlogged.

>     * At the time where you see the last major purple graph spike to 10G
>       free, we had consumed all of the messages in the broker.  The
>       broker is currently sitting idle with no messages, but still has
>       2+GB of ram allocated.
> 
> I did notice there seems to be some sort of cleanup worker that runs
> roughly every 30 minutes, but it's not reclaimed any of the 2+GB of
> allocation since the broker has gone idle. Is this expected behavior?

It is quite common for a gc'ed VM to hold on to large chunks of memory
instead of handing them back to the OS.

Are you monitoring queues with rabbitmqctl or Alice? If that happens
more frequently than once per second then the queue processes are kept
active, which delays the freeing up of memory.

To get some diagnostics on where the memory has gone, shell into rabbit
(erl -sname shell -remsh rabbit@<hostname>) and run
  memory().


Regards,

Matthias.