<div dir="ltr">Upgraded the other node (there are two) and it fails to start.<div>Here is the startup log:</div><div><br></div><div><br></div><div><div>+---+ � +---+</div><div>| � | � | � |</div><div>| � | � | � |</div><div>
| � | � | � |</div><div>| � +---+ � +-------+</div><div>| � � � � � � � � � |</div><div>| RabbitMQ �+---+ � |</div><div>| � � � � � | � | � |</div><div>| � v2.8.7 �+---+ � |</div><div>| � � � � � � � � � |</div><div>+-------------------+</div>
<div>AMQP 0-9-1 / 0-9 / 0-8</div><div>Copyright (C) 2007-2012 VMware, Inc.</div><div>Licensed under the MPL. �See <a href="http://www.rabbitmq.com/">http://www.rabbitmq.com/</a></div><div><br></div><div>node � � � � � : rabbit@mongo-qa2</div>
<div>app descriptor : /usr/lib/rabbitmq/lib/rabbitmq_server-2.8.7/sbin/../ebin/rabbit.app</div><div>home dir � � � : /var/lib/rabbitmq</div><div>config file(s) : (none)</div><div>cookie hash � �: pu0BlS9+G2N9yLfd51TkmA==</div>
<div>log � � � � � �: /var/log/rabbitmq/rabbit@mongo-qa2.log</div><div>sasl log � � � : /var/log/rabbitmq/rabbit@mongo-qa2-sasl.log</div><div>database dir � : /var/lib/rabbitmq/mnesia/rabbit@mongo-qa2</div><div>erlang version : 5.8.4</div>
<div><br></div><div>-- rabbit boot start</div><div>starting file handle cache server � � � � � � � � � � � � � � � � � � ...done</div><div>starting worker pool � � � � � � � � � � � � � � � � � � � � � � � � �...done</div>
<div>starting database � � � � � � � � � � � � � � � � � � � � � � � � � � ...</div><div><br></div><div>BOOT FAILED</div><div>===========</div><div><br></div><div>Error description:</div><div>� �{error,{failed_to_cluster_with,['rabbit@mongo-qa1'],</div>
<div>� � � � � � � � � � � � � � � � � "Mnesia could not connect to any disc nodes."}}</div><div><br></div><div>Log files (may contain more information):</div><div>� �/var/log/rabbitmq/rabbit@mongo-qa2.log</div>
<div>� �/var/log/rabbitmq/rabbit@mongo-qa2-sasl.log</div><div><br></div><div>Stack trace:</div><div>� �[{rabbit_mnesia,init_db,3},</div><div>� � {rabbit_mnesia,init,0},</div><div>� � {rabbit,'-run_boot_step/1-lc$^1/1-1-',1},</div>
<div>� � {rabbit,run_boot_step,1},</div><div>� � {rabbit,'-start/2-lc$^0/1-0-',1},</div><div>� � {rabbit,start,2},</div><div>� � {application_master,start_it_old,4}]</div><div><br></div><div>{"Kernel pid terminated",application_controller,"{application_start_failure,rabbit,{bad_return,{{rabbit,start,[normal,[]]},{'EXIT',{rabbit,failure_during_boot}}}}}"}</div>
<div><br></div><div dir="ltr"><div><br></div><div>Thanks<br clear="all"><div dir="ltr"><i style="color:rgb(102,102,102)">--</i><span style="color:rgb(102,102,102)">Raviv</span></div></div><div><font color="#666666"><i><br>
</i></font></div></div><br>
<br><br><div class="gmail_quote">On Thu, Oct 4, 2012 at 5:34 PM, Simon MacMullen <span dir="ltr"><<a href="mailto:simon@rabbitmq.com" target="_blank">simon@rabbitmq.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hard to believe that's to do with the server not starting - can you post a more complete log?<div class="im"><br>
<br>
On 04/10/12 16:32, Raviv Pavel wrote:<br>
</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">
After upgrading the server won't start. from the logs:<br>
<br>
=ERROR REPORT==== 4-Oct-<a href="tel:2012" value="+9722012" target="_blank">2012</a>::17:11:59 ===<br>
** Generic server <<a href="tel:0.341.0" value="+9723410" target="_blank">0.341.0</a>> terminating<br>
** Last message in was {'$gen_cast',<br>
� � � � � � � � � � � � � � {method,<br>
<br>
� {'queue.declare',0,<<"es1">>,<u></u>false,true,false,<br>
� � � � � � � � � � � � � � � � � � false,false,<br>
<br>
� [{<<"x-ha-policy">>,longstr,<<<u></u>"all">>}]},<br>
� � � � � � � � � � � � � � � � none,noflow}}<br>
<br>
<br>
Thanks<br></div>
/--/Raviv<br>
/<div class="im"><br>
/<br>
<br>
<br>
<br>
On Thu, Oct 4, <a href="tel:2012" value="+9722012" target="_blank">2012</a> at 5:05 PM, Raviv Pavel <<a href="mailto:raviv@gigya-inc.com" target="_blank">raviv@gigya-inc.com</a><br></div><div class="im">
<mailto:<a href="mailto:raviv@gigya-inc.com" target="_blank">raviv@gigya-inc.com</a>>> wrote:<br>
<br>
� � All queues are empty and "rabbitmqctl list_queues name memory" shows<br>
� � they use about 3K each.<br>
� � VM memory stats are much lower then the one shown in the overview.<br>
� � We're using 2.8.1 - I'll try upgrading to 2.8.7<br>
<br>
� � Thanks<br></div>
� � /--/Raviv<br>
� � /<br>
� � /<br>
<br>
<br>
<br>
� � On Thu, Oct 4, <a href="tel:2012" value="+9722012" target="_blank">2012</a> <tel:<a href="tel:2012" value="+9722012" target="_blank">2012</a>> at 4:57 PM, Matthias Radestock<div class="im"><br>
� � <<a href="mailto:matthias@rabbitmq.com" target="_blank">matthias@rabbitmq.com</a> <mailto:<a href="mailto:matthias@rabbitmq.com" target="_blank">matthias@rabbitmq.com</a>><u></u>> wrote:<br>
<br>
� � � � On 04/10/12 15:53, Simon MacMullen wrote:<br>
<br>
� � � � � � On 04/10/12 13:01, Raviv Pavel wrote:<br>
<br>
� � � � � � � � Based on the management UI, memory usage doesn't drop<br>
� � � � � � � � and publishers are<br>
� � � � � � � � still blocked.<br>
<br>
<br>
� � � � � � Hmm. In that case my first guess would be that the queue you<br>
� � � � � � deleted /<br>
� � � � � � purged wasn't the queue which was using all the memory.<br>
� � � � � � Check the memory<br>
� � � � � � use of other queues - this can be found on the queue details<br>
� � � � � � page, or<br>
� � � � � � with "rabbitmqctl list_queues name memory". Also, check the<br>
� � � � � � VM-wide<br>
� � � � � � memory statistics (on the node details page, under<br>
� � � � � � "advanced". This<br>
� � � � � � might give a clue as to where the memory is going.<br>
<br>
<br>
� � � � ...and please make sure you are running the latest version of<br>
� � � � RabbitMQ (2.8.7, atm), since we have fixed a few memory leaks in<br>
� � � � the in recent versions.<br>
<br>
� � � � Matthias.<br>
<br>
<br>
<br>
</div></blockquote>
<br><div class="HOEnZb"><div class="h5">
<br>
-- <br>
Simon MacMullen<br>
RabbitMQ, VMware<br>
</div></div></blockquote></div><br></div></div>