[rabbitmq-discuss] Rabbit won't restart - no next heap size found

[Mike Khristo]

Got it going but it was just sheer luck... the 25th attempt to start it got
it working, somehow. Even with 200G swap, it didn't help.


On Tue, Oct 9, 2012 at 11:30 AM, Matthias Radestock
<matthias at rabbitmq.com>wrote:

> Mike,
>
> did you make any progress on this?
>
> We have since discovered that rabbit can use lot of memory on recovery
> from an unclean shutdown, so this may well be the problem you are running
> into. We'll fix that in an upcoming release, but meanwhile the suggestions
> below should get you going.
>
> Matthias.
>
> On 29/09/12 08:41, Matthias Radestock wrote:
>
>> Mike,
>>
>> On 28/09/12 22:06, Mike Khristo wrote:
>>
>>> Added a 200G swap... it used some of it, but still seg faults at
>>> startup.
>>>
>>
>> Did it fail with the same error, i.e. "eheap_alloc: Cannot allocate ....
>> bytes of memory"? That would be kinda surprising.
>>
>> Is there anything in the log files?
>>
>> There are scenarios in which an error can trigger the writing of an
>> enormous log message, and it's the latter that is exhausting memory. If
>> so, the log should contain the error.
>>
>>  Is there any way to split up the data on disk so I can get 2 rabbits
>>> up with half the data each?
>>>
>>
>> Hmm. The following should work:
>>
>> 1) make a backup of the rabbit database dir
>>
>> 2) cd into the database dir and remove the "clean shutdown" markers with
>>     rm -f `find . -name "clean.dot"`
>> If rabbit was shut down cleanly there should be one of these per queue
>> plus one each in the msg_store_persistent and msg_store_transient dirs.
>> If rabbit wasn't shut down cleanly and those files aren't present then
>> not to worry.
>>
>> 3) in the msg_store_persistent dir remove half the *.rdq files
>>
>> 4) start rabbit
>>
>> Then process all the messages in that rabbit, stop it, restore the
>> backup and repeat the above but this time removing the other half of the
>> *.rdq files.
>>
>>
>> Regards,
>>
>> Matthias.
>>
>
>


-- 
-mk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20121009/9fd937e5/attachment.htm>