[rabbitmq-discuss] Possible memory leak?

Sat Jul 3 21:52:28 BST 2010

David,

David King wrote:
>> If you just let it run, what happens?
> 
> I don't know. I guess we can see, but my guess is that it will wake
> me up at two in the morning to fix the site. That's what usually
> happens when rabbit crashes, anyway.

As long as you are running rabbit 1.8.0, and given the usage pattern you 
describe, it is unlikely that rabbit would actually crash, though it may 
refuse to accept messages from producers (which, I guess, would still be 
a problem). It would be informative to find out whether it actually 
reaches that point.

What memory limit does rabbit think it has? Check the rabbit log for an 
info report along the lines of

=INFO REPORT==== 29-Oct-2009::15:43:27 ===
Memory limit set to 2048MB.

Also, are there any memory alarms in the logs? Look for entries like

=INFO REPORT==== 3-Jul-2010::21:28:31 ===
     alarm_handler: {set,{vm_memory_high_watermark,[]}}

As Marek mentioned your graphs would seem to indicate that rabbit is 
consuming a few gigs of memory right after startup. That doesn't look 
right. A healthy rabbit should eat much less memory than that. For 
example, the rabbit on my development machine starts with a 80MB virtual 
memory footprint, 25 of which is resident.

I am assuming the graph shows the memory usage of the entire machine. 
You said that the memory "does belong to beam.smp", but do you have a 
graph, or even just some data points for only that process?

If the above doesn't yield any clues, perhaps you could grant one of the 
rabbit engineers temporary access to the machine so they can delve into 
the innards of rabbit to figure out where the memory is going? This 
shouldn't affect the operation of your service.

>> What's the average and max message size in your set up?
> 
> They're just IDs, so they'll be between 5 and 10 bytes each.

Are any of the messages published as persistent? Another reason to 
upgrade to 1.8.0 is that there only messages published as persistent 
that go to durable queues actually hit the persister - a change we made 
partially as a result of the investigation into your previous problems.

>> How are you measuring the queue length? With 'rabbitmqctl
>> list_queues'?

How frequently do you run rabbitmqctl?

Regards,

Matthias.