[rabbitmq-discuss] Rabbit MQ Crash

Tue Apr 8 02:50:54 BST 2014

Hi Matthias,

Full logs are attached.  You'll notice that it crashes pretty often now.

Here are the output of the commands

$ sudo rabbitmqctl eval 'rabbit_misc:os_cmd("/bin/df -kP
/var/lib/rabbitmq/mnesia/")'
Error: syntax error before:
$ /bin/df -kP /var/lib/rabbitmq/mnesia/
Filesystem                                             1024-blocks    Used
Available Capacity Mounted on
/dev/disk/by-uuid/36fd30d4-ea87-419f-a6a4-a1a3cf290ff1    10320184 6358284
  3437664      65% /

I just noticed the following in the non sasl log, I should have included it
in the original:

=ERROR REPORT==== 7-Apr-2014::10:08:34 ===
** Generic server rabbit_disk_monitor terminating
** Last message in was update
** When Server state == {state,"/var/lib/rabbitmq/mnesia/rabbit at ocr-proc-2",
                               50000000,3702104064,100,10000,
                               #Ref<0.0.1.41329>,false}
** Reason for termination ==
** {{badmatch,[]},
    [{rabbit_disk_monitor,parse_free_unix,1,[]},
     {rabbit_disk_monitor,internal_update,1,[]},
     {rabbit_disk_monitor,handle_info,2,[]},
     {gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,607}]},
     {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]}

=ERROR REPORT==== 7-Apr-2014::10:08:35 ===
** Generic server rabbit_mgmt_external_stats terminating
** Last message in was emit_update
** When Server state == {state,32768}
** Reason for termination ==
** {{{badmatch,[]},
     [{rabbit_disk_monitor,parse_free_unix,1,[]},
      {rabbit_disk_monitor,internal_update,1,[]},
      {rabbit_disk_monitor,handle_info,2,[]},
      {gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,607}]},
      {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]},
    {gen_server,call,[rabbit_disk_monitor,get_disk_free_limit,infinity]}}

Michael Sander
mes65 at cornell.edu
607-227-9859

On Mon, Apr 7, 2014 at 5:29 PM, Michael Sander <mes65 at cornell.edu> wrote:

> Wow, I had completely forgotten about that, I apologies for the duplicate.
>  It also looks like I dropped the ball on answering your questions.  Again
> my apologies, I was very busy at that time.
>
> In a few hours I will send you:
> - The complete sasl and normal logs
> - The output of the command rabbitmqctl eval 'rabbit_misc:os_cmd("/bin/df
> -kP /var/lib/rabbitmq/mnesia/")
>
> Best,
>
> Michael Sander
>
>
> On Mon, Apr 7, 2014 at 5:23 PM, Matthias Radestock <matthias at rabbitmq.com>wrote:
>
>> Michael,
>>
>>
>> On 07/04/14 22:06, Michael Sander wrote:
>>
>>> I am using a relatively memory-limited machine (n1-highcpu-2 on google
>>> compute engine, details here
>>> <https://developers.google.com/compute/docs/machine-types#highcpu>), and
>>>
>>> there's a possibility that I ran out of memory at some point, but I did
>>> not see any direct evidence of that.
>>>
>>
>> Ah! Your original report did not mention that this is on GCE. Looking at
>> our bug tracker, I see now that you reported the same issue previously,
>> back in October: http://rabbitmq.1065348.n5.nabble.com/rabbitmq-nodedown-
>> nodedown-Generic-server-rabbit-disk-monitor-terminating-td30939.html.
>>
>>
>>  The logs I sent were just snippets.
>>>
>>
>> The complete logs should hopefully shed some further light on the root
>> cause.
>>
>> Matthias.
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20140407/3b0ed15d/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: rabbit-sasl.log
Type: application/octet-stream
Size: 37076 bytes
Desc: not available
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20140407/3b0ed15d/attachment-0002.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: rabbit.log
Type: application/octet-stream
Size: 1872137 bytes
Desc: not available
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20140407/3b0ed15d/attachment-0003.obj>