[rabbitmq-discuss] Help with my rabbitmq crashing

[Emile Joubert]

Hi,

On 09/10/12 08:10, Geocast wrote:
> This rabbitmq has been running for long days without problems.
> Today I restarted it but failed. The info:
> 
> root at rabbitmq2:/var/log/rabbitmq# cat startup_err

The reason for the broker crashing in the first place is more likely to
be in the main broker logfile or the broker sasl file. You should look
at that also.

> Crash dump was written to: erl_crash.dump
> eheap_alloc: Cannot allocate 2850821240 bytes of memory (of type "heap").
> Aborted

I assume the quoted error was generated upon attempted startup after the
crash. The crash would have caused the broker to stop in such a way that
an expensive and lengthy recovery process is required at startup, in
order to recover messages.

> root at rabbitmq2:/var/log/rabbitmq# free -m
>              total       used       free     shared    buffers     cached
> Mem:          8004       1242       6761          0         13        870

> As you see, my system has enough free memory, but rabbitmq aborted in
> the process of starting.

It does looks like there is enough free memory and that allocation
should succeed, but this depends on when the "free" command was run.
There may have been less free memory while the broker was trying to
start up. Is there any other reason why the OS might refuse? Are there
any ulimits in effect that might prevent it? Does the OS syslog say
anything? The root user might not be subject to the same limit, so it
might be worth running the broker as root, just to start up.

Can you get Erlang to allocate that much memory independently from the
running the broker? Try this test - start an Erlang VM

 erl

and enter this:

 size(<<0:2850821240/unit:8>>).

The result should be 2850821240. If you receive an error instead then
the broker is unlikely to have enough RAM to recover messages at startup.


-Emile