[rabbitmq-discuss] rabbit disk_mode branch eating up all RAM, including swap, dying

[Brian Whitman]

Hi Matthew, all, I had a couple people here test out this new branch vs. our
old branch on a local VM.  One thing we didn't know is that our messages
were relatively big -- 10KB each (I thought they were always much smaller.)
They report:
On a virtual machine with 1.5GB ram, we saw the following behaviour:

latest stable erlang + rabbitmq (20980) died once, and another time became
unresponsive, both times just before it hit ~2M messages (9KB each). CPU was
pegged around 90 %, and swap grew to almost 5GB.

latest stable erlang + rabbitmq (21444) never died, even with 3.5M+
messages. CPU was 90 %, and swap never grew past ~1.7GB.


We're going to upgrade to 21444 on our host and see how it goes.

Thanks
Brian




On Thu, Oct 8, 2009 at 7:20 AM, Matthew Sackman <matthew at lshift.net> wrote:

> Hi Brian,
>
> On Mon, Oct 05, 2009 at 11:04:21AM +0100, Matthew Sackman wrote:
> > However, if you just had billions of
> > messages in there, then even in a non clustered setup, I could believe
> > mnesia would be taking a long time to start up and that might cause the
> > above error.
>
> I've just come across this thread:
> http://www.trapexit.org/forum/viewtopic.php?p=44433
> which does indeed seem to support the idea that even modest mnesia
> databases can take rather more than the 30 seconds we give them to
> start up. Given that I would imagine this could happen even in v1.7
> with, say, a couple million durable queues, exchanges and bindings,
> this is certainly something we will look at as the 30 second timeout
> may well be grossly too short.
>
> Matthew
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20091009/f26ecbd3/attachment.htm