[rabbitmq-discuss] Possible memory leak?

Thu Jul 1 21:23:07 BST 2010

David,

David King wrote:
>> this sharp edge happened without your intervention?
> 
> Yes, that's when I last restarted rabbit. We have to take our site
> down to do so, and that's not desirable.

If you just let it run, what happens?

> According to that graph I'm going to have to restart the node soon
> anyway, so I'll try that, but if it doesn't fix a known leak that's
> unlikely to help, right? As for Erlang, we're running R13B-4.

We have seen a few instances of peculiar interaction between the rabbit 
code and the Erlang VM when it comes to memory usage. Even minor and 
unrelated changes can alter the memory consumption pattern 
significantly. It is for that reason, and in order to make it easier for 
us to investigate the problem further, that we recommend running the 
latest release of Rabbit. As for Erlang/OTP, R13B-4 is recent enough.

>> your specific usage patterns might not play well with
>> garbage collection. For example, we force full garbage collection
>> run when the queue is unused for more than 10 seconds. If your
>> queue is constantly pinged - you're left to Erlang default garbage
>> collection.  That works well most of the time, but sometimes it's
>> not optimal.
> 
> That's possible, but Erlang's default garbage collector is a
> stop-the-world (per-process) collector, so it's shouldn't be *that*
> dependent on load, it just just introduce pauses per-queue (which
> would actually be fine for our use-case)

The default Erlang gc performs garbage collections after a certain 
number of reductions, and that number is so high that a queue process 
which does work infrequently may not perform a gc for a very long time 
indeed. That's particularly noticeable when the message sizes are large. 
What's the average and max message size in your set up?

> So to be clear, we have 13
> queues that all sit at 0 backlog basically all of the time. Certainly
> no queues that grow at the same rate our memory does. 

How are you measuring the queue length? With 'rabbitmqctl list_queues'?

Are any of the other items which can be listed with rabbitmqctl - 
exchanges, bindings, connections, channels, consumers - growing?

Also, have you installed any plug-ins?

Regards,

Matthias.