<div dir="ltr">I found my rabbitmq node is down in the cluster. <div><br></div><div><div>Here is the log</div><div><br></div><div><div>=ERROR REPORT==== 9-Jan-2014::03:17:27 ===</div><div>** Generic server rabbit_disk_monitor terminating</div>
<div>** Last message in was update</div><div>** When Server state == {state,"/var/lib/rabbitmq/mnesia/rabbitmq@localhost",</div><div>                               1000000000,6201094144,10000,</div><div>                               {interval,#Ref<0.0.0.977>},</div>
<div>                               false}</div><div>** Reason for termination ==</div><div>** {badarg,[{erlang,list_to_integer,["systems"],[]},</div><div>            {rabbit_disk_monitor,parse_free_unix,1,[]},</div>
<div>            {rabbit_disk_monitor,internal_update,1,[]},</div><div>            {rabbit_disk_monitor,handle_info,2,[]},</div><div>            {gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,607}]},</div>
<div>            {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]}</div><div><br></div><div>=INFO REPORT==== 9-Jan-2014::03:17:27 ===</div><div>Disabling disk free space monitoring on unsupported platform: {{'EXIT',</div>
<div>                                                                {badarg,</div><div>                                                                 [{erlang,</div><div>                                                                   list_to_integer,</div>
<div>                                                                   ["systems"],</div><div>                                                                   []},</div><div>                                                                  {rabbit_disk_monitor,</div>
<div>                                                                   parse_free_unix,</div><div>                                                                   1,[]},</div><div>                                                                  {rabbit_disk_monitor,</div>
<div>                                                                   init,1,[]},</div><div>                                                                  {gen_server,</div><div>                                                                   init_it,6,</div>
<div>                                                                   [{file,</div><div>                                                                     "gen_server.erl"},</div><div>                                                                    {line,</div>
<div>                                                                     304}]},</div><div>                                                                  {proc_lib,</div><div>                                                                   init_p_do_apply,</div>
<div>                                                                   3,</div><div>                                                                   [{file,</div><div>                                                                     "proc_lib.erl"},</div>
<div>                                                                    {line,</div><div>                                                                     239}]}]}},</div><div>                                                               16723951616}</div>
</div><div><br></div><div>After a while, the log reports</div><div><br></div><div><div>=ERROR REPORT==== 9-Jan-2014::03:20:15 ===</div><div>Mnesia(rabbitmq@localhost): ** ERROR ** (core dumped to file: "/var/lib/rabbitmq/MnesiaCore.rabbitmq@localhost_1389_237615_166554")</div>
<div> ** FATAL ** {error,{"Cannot rename disk_log file",latest_log,</div><div>                     "/var/lib/rabbitmq/mnesia/rabbitmq@localhost/PREVIOUS.LOG",</div><div>                     {log_header,trans_log,"4.3","4.8",rabbitmq@localhost,</div>
<div>                                 {1389,237615,128040}},</div><div>                     {file_error,"/var/lib/rabbitmq/mnesia/rabbitmq@localhost/PREVIOUS.LOG",</div><div>                                 enoent}}}</div>
<div><br></div><div>=ERROR REPORT==== 9-Jan-2014::03:20:25 ===</div><div>Mnesia(rabbitmq@localhost): ** ERROR ** mnesia_event got unexpected event: {'EXIT',</div><div>                                                                            <0.111.0>,</div>
<div>                                                                            killed}</div></div><div><br></div><div>I found the dump file is actually in my disk, but how to open this dump file? /var/lib/rabbitmq/MnesiaCore.rabbitmq@localhost_1389_237615_166554</div>
<div><br></div><div>Also rabbitmqctl status command says its status is correct, but rabbitmqctl list_users, list_exchanges or other commands just print an error message.</div><div><br></div><div>Anyone can help with this issue? It can not be reproduced, but happens sometimes.<br>
-- </div>Best regards<br>
<br>
Sili Liu
</div></div>