<div dir="ltr">Upgraded the other node (there are two) and it fails to start.<div>Here is the startup log:</div><div><br></div><div><br></div><div><div>+---+ +---+</div><div>| | | |</div><div>| | | |</div><div>
| | | |</div><div>| +---+ +-------+</div><div>| |</div><div>| RabbitMQ +---+ |</div><div>| | | |</div><div>| v2.8.7 +---+ |</div><div>| |</div><div>+-------------------+</div>
<div>AMQP 0-9-1 / 0-9 / 0-8</div><div>Copyright (C) 2007-2012 VMware, Inc.</div><div>Licensed under the MPL. See <a href="http://www.rabbitmq.com/">http://www.rabbitmq.com/</a></div><div><br></div><div>node : rabbit@mongo-qa2</div>
<div>app descriptor : /usr/lib/rabbitmq/lib/rabbitmq_server-2.8.7/sbin/../ebin/rabbit.app</div><div>home dir : /var/lib/rabbitmq</div><div>config file(s) : (none)</div><div>cookie hash : pu0BlS9+G2N9yLfd51TkmA==</div>
<div>log : /var/log/rabbitmq/rabbit@mongo-qa2.log</div><div>sasl log : /var/log/rabbitmq/rabbit@mongo-qa2-sasl.log</div><div>database dir : /var/lib/rabbitmq/mnesia/rabbit@mongo-qa2</div><div>erlang version : 5.8.4</div>
<div><br></div><div>-- rabbit boot start</div><div>starting file handle cache server ...done</div><div>starting worker pool ...done</div>
<div>starting database ...</div><div><br></div><div>BOOT FAILED</div><div>===========</div><div><br></div><div>Error description:</div><div> {error,{failed_to_cluster_with,['rabbit@mongo-qa1'],</div>
<div> "Mnesia could not connect to any disc nodes."}}</div><div><br></div><div>Log files (may contain more information):</div><div> /var/log/rabbitmq/rabbit@mongo-qa2.log</div>
<div> /var/log/rabbitmq/rabbit@mongo-qa2-sasl.log</div><div><br></div><div>Stack trace:</div><div> [{rabbit_mnesia,init_db,3},</div><div> {rabbit_mnesia,init,0},</div><div> {rabbit,'-run_boot_step/1-lc$^1/1-1-',1},</div>
<div> {rabbit,run_boot_step,1},</div><div> {rabbit,'-start/2-lc$^0/1-0-',1},</div><div> {rabbit,start,2},</div><div> {application_master,start_it_old,4}]</div><div><br></div><div>{"Kernel pid terminated",application_controller,"{application_start_failure,rabbit,{bad_return,{{rabbit,start,[normal,[]]},{'EXIT',{rabbit,failure_during_boot}}}}}"}</div>
<div><br></div><div dir="ltr"><div><br></div><div>Thanks<br clear="all"><div dir="ltr"><i style="color:rgb(102,102,102)">--</i><span style="color:rgb(102,102,102)">Raviv</span></div></div><div><font color="#666666"><i><br>
</i></font></div></div><br>
<br><br><div class="gmail_quote">On Thu, Oct 4, 2012 at 5:34 PM, Simon MacMullen <span dir="ltr"><<a href="mailto:simon@rabbitmq.com" target="_blank">simon@rabbitmq.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hard to believe that's to do with the server not starting - can you post a more complete log?<div class="im"><br>
<br>
On 04/10/12 16:32, Raviv Pavel wrote:<br>
</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">
After upgrading the server won't start. from the logs:<br>
<br>
=ERROR REPORT==== 4-Oct-<a href="tel:2012" value="+9722012" target="_blank">2012</a>::17:11:59 ===<br>
** Generic server <<a href="tel:0.341.0" value="+9723410" target="_blank">0.341.0</a>> terminating<br>
** Last message in was {'$gen_cast',<br>
{method,<br>
<br>
{'queue.declare',0,<<"es1">>,<u></u>false,true,false,<br>
false,false,<br>
<br>
[{<<"x-ha-policy">>,longstr,<<<u></u>"all">>}]},<br>
none,noflow}}<br>
<br>
<br>
Thanks<br></div>
/--/Raviv<br>
/<div class="im"><br>
/<br>
<br>
<br>
<br>
On Thu, Oct 4, <a href="tel:2012" value="+9722012" target="_blank">2012</a> at 5:05 PM, Raviv Pavel <<a href="mailto:raviv@gigya-inc.com" target="_blank">raviv@gigya-inc.com</a><br></div><div class="im">
<mailto:<a href="mailto:raviv@gigya-inc.com" target="_blank">raviv@gigya-inc.com</a>>> wrote:<br>
<br>
All queues are empty and "rabbitmqctl list_queues name memory" shows<br>
they use about 3K each.<br>
VM memory stats are much lower then the one shown in the overview.<br>
We're using 2.8.1 - I'll try upgrading to 2.8.7<br>
<br>
Thanks<br></div>
/--/Raviv<br>
/<br>
/<br>
<br>
<br>
<br>
On Thu, Oct 4, <a href="tel:2012" value="+9722012" target="_blank">2012</a> <tel:<a href="tel:2012" value="+9722012" target="_blank">2012</a>> at 4:57 PM, Matthias Radestock<div class="im"><br>
<<a href="mailto:matthias@rabbitmq.com" target="_blank">matthias@rabbitmq.com</a> <mailto:<a href="mailto:matthias@rabbitmq.com" target="_blank">matthias@rabbitmq.com</a>><u></u>> wrote:<br>
<br>
On 04/10/12 15:53, Simon MacMullen wrote:<br>
<br>
On 04/10/12 13:01, Raviv Pavel wrote:<br>
<br>
Based on the management UI, memory usage doesn't drop<br>
and publishers are<br>
still blocked.<br>
<br>
<br>
Hmm. In that case my first guess would be that the queue you<br>
deleted /<br>
purged wasn't the queue which was using all the memory.<br>
Check the memory<br>
use of other queues - this can be found on the queue details<br>
page, or<br>
with "rabbitmqctl list_queues name memory". Also, check the<br>
VM-wide<br>
memory statistics (on the node details page, under<br>
"advanced". This<br>
might give a clue as to where the memory is going.<br>
<br>
<br>
...and please make sure you are running the latest version of<br>
RabbitMQ (2.8.7, atm), since we have fixed a few memory leaks in<br>
the in recent versions.<br>
<br>
Matthias.<br>
<br>
<br>
<br>
</div></blockquote>
<br><div class="HOEnZb"><div class="h5">
<br>
-- <br>
Simon MacMullen<br>
RabbitMQ, VMware<br>
</div></div></blockquote></div><br></div></div>