[rabbitmq-discuss] rabbit disk_mode branch eating up all RAM, including swap, dying

[Matthew Sackman]

On Mon, Oct 05, 2009 at 02:23:14PM -0400, Brian Whitman wrote:
> > How many messages did you have in there, and do you know the average
> > size?
> >
> 
> Can't know for sure but my guess is about 20 queues, about 500K messages in
> each queue, message sizes are about 1KB each.

Hmm, 10million rows in mnesia should be fine, but I can definitely
believe that mnesia would take more than 30 seconds to start up, esp if
this is on EC2 which is known to have disk bandwidth issues (correct me
if I'm wrong there). It's very possible that nothing was wrong at all -
it just timed out waiting for mnesia to start up and load in all the
tables, which would in turn stop Rabbit from starting up.

> Does bug21444 have the disk mode auto-pinning stuff?
> 
> I understand about the warnings, the stable branch though would crash almost
> immediately with our message load.

Yes. Basically, a lot of the various features of 20980 got split out
into different branches. The main bulk of the work, in terms of the new
persister was in 21368, but that lacked the manual controls for pinning
queues to disk - everything it did was automatic only. 21368 then went
through a lot of QA - the code is in good shape (although flawed due to
issues with mnesia), and 21444 adds to 21368 the manual controls.

Matthew