<div dir="ltr"><pre style="color:rgb(72,72,72);font-size:12px">Got some really funky errors and a sudden crash of my entire cluster.  �Off hand, I&#39;m GUESSING it&#39;s a disk error, but I&#39;m not totally sure - thought I&#39;d see if anyone had any ideas?
Thanks!
Jason</pre><pre style="color:rgb(72,72,72);font-size:12px"><br></pre><pre style="color:rgb(72,72,72);font-size:12px">The web ui returns the following:</pre><pre style="color:rgb(72,72,72);font-size:12px">{error,{error,badarg,
              [{ets,match,[rabbit_registry,{{exchange,&#39;$1&#39;},&#39;$2&#39;}],[]},
               {rabbit_registry,lookup_all,1,[]},
               {rabbit_mgmt_external_stats,list_registry_plugins,2,[]},
               {rabbit_mgmt_wm_overview,to_json,2,[]},
               {webmachine_resource,resource_call,3,[]},
               {webmachine_resource,do,3,[]},
               {webmachine_decision_core,resource_call,1,[]},
               {webmachine_decision_core,decision,1,[]}]}}</pre><pre style="color:rgb(72,72,72);font-size:12px"><br></pre><pre style="color:rgb(72,72,72);font-size:12px">In the cluster.log file I see a TON of data and some really odd things like:</pre>
<pre><font color="#484848">  {msg_status,198128,
                               &lt;&lt;8,75,158,13,139,133,182,107,227,74,148,135,78,
                                 181,221,89&gt;&gt;,
                               undefined,true,false,true,true,
                               {message_properties,undefined,true,false}},
                              {msg_status,198127,&lt;&lt;&quot;¯?VõFþCT�
0k2­ÿI�&quot;&gt;&gt;,                  undefined,true,false,true,true,
                               {message_properties,undefined,true,false}},
                              {msg_status,198126,
                               &lt;&lt;207,120,238,6,2,192,31,43,235,1,10,138,104,
                                 207,34,236&gt;&gt;,</font><span style="color:rgb(72,72,72);font-size:12px">
</span></pre><div><br></div><div><div>=ERROR REPORT==== 14-Aug-2013::09:53:50 ===</div><div>** Generic server rabbit_mgmt_external_stats terminating</div><div>** Last message in was emit_update</div><div>** When Server state == {state,1024}</div>
<div>** Reason for termination ==</div><div>** {noproc,{gen_server,call,[rabbit_node_monitor,partitions,infinity]}}</div></div><div><br></div><div><div>** Reason for termination ==</div><div>** {{badmatch,{error,eio}},</div>
<div>� � [{file_handle_cache,soft_close,1,[]},</div><div>� � �{file_handle_cache,hard_close,1,[]},</div><div>� � �{file_handle_cache,close,1,[]},</div><div>� � �{rabbit_msg_store,terminate,2,[]},</div><div>� � �{gen_server2,terminate,3,[]},</div>
<div>� � �{proc_lib,wake_up,3,[{file,&quot;proc_lib.erl&quot;},{line,237}]}]}</div><div>** In &#39;terminate&#39; callback with reason ==</div><div>** shutdown</div></div><div><br></div><div><br></div><div><br></div><div>
<br></div><pre><font color="#484848">=ERROR REPORT==== 14-Aug-2013::09:53:50 ===
** Generic server msg_store_transient terminating
** Last message in was {&#39;EXIT&#39;,&lt;0.238.0&gt;,shutdown}
** When Server state == {msstate,
                            &quot;/data/rabbitmq/rabbitmq/mnesia/cluster/msg_store_transient&quot;,
                            rabbit_msg_store_ets_index,
                            {state,397381,
                                &quot;/data/rabbitmq/rabbitmq/mnesia/cluster/msg_store_transient&quot;},
                            0,#Ref&lt;0.0.0.1839&gt;,
                            {dict,0,16,16,8,80,48,
                                {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],
                                 []},
                                {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],
                                  [],[]}}},
                            undefined,0,1271,[],&lt;0.291.0&gt;,401478,393277,
                            405575,409672,
                            {set,0,16,16,8,80,48,
                                {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],
                                 []},
                                {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],
                                  [],[]}}},
                            {dict,48,16,16,8,80,48,
                                {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],
                                 []},</font><span style="color:rgb(72,72,72);font-size:12px">
</span></pre><div><br></div><pre style="color:rgb(72,72,72);font-size:12px"><br></pre><div><br></div>-- <br>Jason McIntosh<br><a href="http://mcintosh.poetshome.com/blog/">http://mcintosh.poetshome.com/blog/</a><br>573-424-7612
</div>