[rabbitmq-discuss] Possible memory leak?

[Marek Majkowski]

David,

On Thu, Jul 1, 2010 at 06:31, David King <dking at ketralnis.com> wrote:
> I have an EC2 node running rabbitmq 1.7.2. Take a look at its memory over the last month <http://i.imgur.com/LMf02.png>. Looks like a possible memory leak?

In this graph, have you restarted RabbitMQ around Saturday? Or had
this sharp edge happened without your intervention?

> It only has 13 queues which generally sit at 0 (this is a change from the last time we restarted it, obvious from that graph, where we had one queue that would sit at ~70k because it was largely duplicates: we no longer do that), but are medium-traffic (maybe a few tens of new items per second, tops, its CPU over the same period: <http://i.imgur.com/BFhMK.png>), so it's not particularly loaded, and it's not clear where all of that memory is going (although it does belong to beam.smp, so it's not some other process). rabbitmq is the only thing this node does.
>
> We can restart once a month, but this might be a bigger deal for higher-traffic users than us

It won't hurt if you upgraded to the newest RabbitMQ release - 1.8.0.

There might be several causes of the behaviour you described. First,
your specific usage patterns might not play well with garbage
collection. For example, we force full garbage collection run when the
queue is unused for more than 10 seconds. If your queue is constantly
pinged - you're left to Erlang default garbage collection.  That works
well most of the time, but sometimes it's not optimal.

Second thing, currently the broker keeps all currently handled
messages in memory. If your RabbitMQ instance handle 70K messages,
they are all stored in RAM. If that's the case, you could consider
running our experimental code branch, which tries to page out messages
to disk and save RAM.

Cheers,
  Marek Majkowski