Hi Simon,<div><br></div><div>I checked the log and noticed every time when start this particular RabbitMQ, the log showed some error messages as follow:</div><div><br></div><div><div><span style="background-color:rgb(255,255,102)">=ERROR REPORT==== 4-Oct-2012::05:46:57 ===</span></div>
<div><span style="background-color:rgb(255,255,102)">Mnesia(nplay@app2): ** ERROR ** mnesia_event got {inconsistent_database, starting_partitioned_network, nplay@web1}</span></div><div><span style="background-color:rgb(255,255,102)"><br>
</span></div><div><span style="background-color:rgb(255,255,102)">=ERROR REPORT==== 4-Oct-2012::05:46:57 ===</span></div><div><span style="background-color:rgb(255,255,102)">Mnesia(nplay@app2): ** ERROR ** mnesia_event got {inconsistent_database, starting_partitioned_network, nplay@web2}</span></div>
</div><div><br></div><div>Why the errors logged? Is it related to my cluster setting?</div><div><br></div><div>This is production environment, I can guaranteed no one dare to execute the command "rabbitmqctl stop".�</div>
<div><br></div><div>Regards,</div><div>Wong<br><br><div class="gmail_quote">On Tue, Oct 16, 2012 at 6:33 PM, Simon MacMullen <span dir="ltr"><<a href="mailto:simon@rabbitmq.com" target="_blank">simon@rabbitmq.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hmm. The SASL log will not necessarily have to contain anything, it is really more of an error log. So I guess there was no error.<br>
<br>
Aha! The log contains:<br>
<br>
� =INFO REPORT==== 16-Oct-2012::00:48:36 ===<br>
� Halting Erlang VM<br>
<br>
We only log that after invocation of "rabbitmqctl stop". So the reason that node shut down was, umm, someone told it to.<br>
<br>
And regarding network partitions, we get information about that from Mnesia. Mnesia will log something like:<br>
<br>
� =ERROR REPORT==== 16-Oct-2012::00:04:19 ===<br>
� Mnesia(nplay@app2): ** ERROR ** mnesia_event got<br>
� � {inconsistent_database, running_partitioned_network, nplay@web2}<br>
<br>
when it has detected a network partition. Note the "running_partitioned_network" - it will also log a very similar message with "starting_partitioned_network" the first time it starts *after* a partition.<br>
<br>
Future versions of RabbitMQ will make this information more accessible.<br>
<br>
Cheers, Simon<div class="im"><br>
<br>
On 16/10/12 10:32, Wong Kam Hoong wrote:<br>
</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">
Hi Simon,<br>
<br>
I check in the sasl log, but it not being update for long time, the<br>
issue happened at 16-oct-2012 but the latest info showed in the log is<br>
only up to 2-oct-2012.<br>
<br>
Attached is the requested sasl log for the server.<br>
<br>
Yea, I remember you mentioned before RabbitMQ not recommend to run in<br>
partitioned network, we still waiting network team to tell us whether<br>
those RabbitMQs is it really deployed in a partitioned network..<br>
<br>
Just curios, how RabbitMQ identify whether the nodes deployed<br>
in partitioned network? I asked this question so that I can discuss<br>
better with network team.<br>
<br>
Regards,<br>
Wong<br>
<br>
<br>
On Tue, Oct 16, 2012 at 5:10 PM, Simon MacMullen <<a href="mailto:simon@rabbitmq.com" target="_blank">simon@rabbitmq.com</a><br></div><div class="im">
<mailto:<a href="mailto:simon@rabbitmq.com" target="_blank">simon@rabbitmq.com</a>>> wrote:<br>
<br>
� � Hi. There's nothing in that log to indicate why the node shut down -<br>
� � can you post the sasl log somewhere?<br>
<br>
� � I don't know if it's related to the network partition. But please<br>
� � bear in mind that network partitions are really bad for RabbitMQ<br>
� � clusters.<br>
<br>
� � Cheers, Simon<br>
<br>
<br>
� � On 16/10/12 03:20, Wong Kam Hoong wrote:<br>
<br>
� � � � Hi RabbitMQ Team,<br>
<br>
� � � � This morning while I checked the RabbitMQs status through web<br>
� � � � admin, I<br>
� � � � found that one of the RabbitMQ node stopped.<br>
<br>
� � � � RabbitMQ v2.8.7<br>
� � � � Erlang v*R14B04*<br>
� � � � *Cluster: Yes, 3 RabbitMQs*<br>
<br>
<br>
� � � � Attached is the log for your reference.<br>
<br>
� � � � After I restarted the service, then everything back to normal.<br>
<br>
� � � � I wonder is the problem related to partitioned network:<br>
<br></div>
� � � � <a href="http://rabbitmq.1065348.n5." target="_blank">http://rabbitmq.1065348.n5.</a>__<a href="http://nabble.com/Statistics-__database-could-not-be-__contacted-Message-rates-and-__queue-lengths-will-not-be-__shown-td22331.html" target="_blank">n<u></u>abble.com/Statistics-__<u></u>database-could-not-be-__<u></u>contacted-Message-rates-and-__<u></u>queue-lengths-will-not-be-__<u></u>shown-td22331.html</a><div class="im">
<br>
� � � � <<a href="http://rabbitmq.1065348.n5.nabble.com/Statistics-database-could-not-be-contacted-Message-rates-and-queue-lengths-will-not-be-shown-td22331.html" target="_blank">http://rabbitmq.1065348.n5.<u></u>nabble.com/Statistics-<u></u>database-could-not-be-<u></u>contacted-Message-rates-and-<u></u>queue-lengths-will-not-be-<u></u>shown-td22331.html</a>><br>
<br>
� � � � Thanks & Regards,<br>
� � � � Wong<br>
<br>
<br></div>
� � � � ______________________________<u></u>___________________<br>
� � � � rabbitmq-discuss mailing list<br>
� � � � rabbitmq-discuss@lists.__<a href="http://rabbitmq.com" target="_blank">rabbi<u></u>tmq.com</a><br>
� � � � <mailto:<a href="mailto:rabbitmq-discuss@lists.rabbitmq.com" target="_blank">rabbitmq-discuss@<u></u>lists.rabbitmq.com</a>><br>
� � � � <a href="https://lists.rabbitmq.com/__cgi-bin/mailman/listinfo/__rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/__<u></u>cgi-bin/mailman/listinfo/__<u></u>rabbitmq-discuss</a><div class="im"><br>
� � � � <<a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/<u></u>cgi-bin/mailman/listinfo/<u></u>rabbitmq-discuss</a>><br>
<br>
<br>
<br>
� � --<br>
� � Simon MacMullen<br>
� � RabbitMQ, VMware<br>
<br>
<br>
</div></blockquote><div class="HOEnZb"><div class="h5">
<br>
<br>
-- <br>
Simon MacMullen<br>
RabbitMQ, VMware<br>
</div></div></blockquote></div><br></div>