[rabbitmq-discuss] RabbitMQ crashes hard when it runs out of memory

[Matthew Sackman]

On Fri, Nov 06, 2009 at 10:06:20AM +0000, Matthew Sackman wrote:
> That's fine. I have to say that it's unlikely this patch will make it
> through - the memory management code has gone through a lot of change
> recently as we're getting a much better handle on resource management.
> Whilst you've obviously been working from the head of our default branch
> (many thanks!), there are a couple of issues with garbage collecting
> every process like that, for example, it's possible that garbage
> collecting vast numbers of processes will take longer than the
> memory_check_interval, making messages queue up for the memory manager
> process. This would become a problem if the garbage collection is unable
> to reclaim any memory at all - eg millions of queues, all of which are
> empty.

Some immediate ideas to improve this a little.
1) Only do the GC when you initially hit the memory alarm. I.e. in the
first case when going from non-alarmed to alarmed, put the gc in there,
then maybe recurse again (though you'll likely want another param on the
function to stop infinite recursion).

2) Only put GC in processes that are known to eat lots of RAM. Eg if
it's the persister, then putting in a manual GC right after it does a
snapshot is probably a good idea.

Matthew