<div dir="ltr"><div style>I have a RabbitMQ 3.0.1 cluster which I upgraded from 2.7.x</div><div style><br></div><div style>For the first couple of days it was quite stable.</div><div style><br></div><div style>This morning however I started noticing some odd behavior. Notably, I would issue BasicPublish commands and they would run forever before finally timing out. I attempted to restart the cluster and this ended with bad results.</div>
<div style><br></div><div style>I finally got the cluster back online, but now its not behaving properly.</div><div style><br></div><div style>Node2 Starts up like this:</div><div style><br></div><div><div>starting memory monitor � � � � � � � � � � � � � � � � � � � � � � � ...done</div>
<div>-- core initialized</div><div>starting federation � � � � � � � � � � � � � � � � � � � � � � � � � ...done</div><div>starting federation exchange decorator � � � � � � � � � � � � � � � �...done</div><div>starting federation parameters � � � � � � � � � � � � � � � � � � � �...done</div>
<div>starting federation upstream exchange type � � � � � � � � � � � � � �...done</div><div>starting management agent � � � � � � � � � � � � � � � � � � � � � � ...done</div><div>starting HA policy validation � � � � � � � � � � � � � � � � � � � � ...done</div>
<div>starting policy parameters � � � � � � � � � � � � � � � � � � � � � �...done</div><div>starting exchange, queue and binding recovery � � � � � � � � � � � � ...done</div><div>starting configured definitions � � � � � � � � � � � � � � � � � � � ...done</div>
<div>starting empty DB check � � � � � � � � � � � � � � � � � � � � � � � ...done</div><div>starting mirror queue slave sup � � � � � � � � � � � � � � � � � � � ...</div><div><br></div><div>BOOT FAILED</div><div>===========</div>
<div><br></div><div>Error description:</div><div>� �{shutdown,</div><div>� � � �{gen_server,call,</div><div>� � � � � �[rabbit_sup,</div><div>� � � � � � {start_child,</div><div>� � � � � � � � {rabbit_mirror_queue_slave_sup,</div>
<div>� � � � � � � � � � {rabbit_mirror_queue_slave_sup,start_link,[]},</div><div>� � � � � � � � � � transient,infinity,supervisor,</div><div>� � � � � � � � � � [rabbit_mirror_queue_slave_sup]}},</div><div>� � � � � � infinity]}}</div>
<div><br></div><div>Log files (may contain more information):</div><div>� �/var/log/rabbitmq/rabbit@egressqueue02.log</div><div>� �/var/log/rabbitmq/rabbit@egressqueue02-sasl.log</div><div><br></div><div>Stack trace:</div>
<div>� �[{gen_server,call,3,[{file,"gen_server.erl"},{line,188}]},</div><div>� � {rabbit_sup,start_supervisor_child,3,[]},</div><div>� � {rabbit,'-run_boot_step/1-lc$^1/1-1-',1,[]},</div><div>� � {rabbit,run_boot_step,1,[]},</div>
<div>� � {rabbit,'-start/2-lc$^0/1-0-',1,[]},</div><div>� � {rabbit,start,2,[]},</div><div>� � {application_master,start_it_old,4,</div><div>� � � � � � � � � � � � [{file,"application_master.erl"},{line,274}]}]</div>
<div><br></div><div><br></div><div><br></div><div>BOOT FAILED</div><div>===========</div><div><br></div><div>Error description:</div><div>� �{could_not_start,rabbit,</div><div>� � {bad_return,</div><div>� � �{{rabbit,start,[normal,[]]},</div>
<div>� � � {'EXIT',</div><div>� � � �{rabbit,failure_during_boot,</div><div>� � � � {shutdown,</div><div>� � � � �{gen_server,call,</div><div>� � � � � [rabbit_sup,</div><div>� � � � � �{start_child,</div><div>� � � � � � {rabbit_mirror_queue_slave_sup,</div>
<div>� � � � � � �{rabbit_mirror_queue_slave_sup,start_link,[]},</div><div>� � � � � � �transient,infinity,supervisor,</div><div>� � � � � � �[rabbit_mirror_queue_slave_sup]}},</div><div>� � � � � �infinity]}}}}}}}</div><div>
<br></div><div>Log files (may contain more information):</div><div>� �/var/log/rabbitmq/rabbit@egressqueue02.log</div><div>� �/var/log/rabbitmq/rabbit@egressqueue02-sasl.log</div><div><br></div><div>{"init terminating in do_boot",{rabbit,failure_during_boot,{could_not_start,rabbit,{bad_return,{{rabbit,start,[normal,[]]},{'EXIT',{rabbit,failure_during_boot,{shutdown,{gen_server,call,[rabbit_sup,{start_child,{rabbit_mirror_queue_slave_sup,{rabbit_mirror_queue_slave_sup,start_link,[]},transient,infinity,supervisor,[rabbit_mirror_queue_slave_sup]}},infinity]}}}}}}}}}</div>
</div><div><br></div>I'm not sure how to proceed. Any advice would be hugely helpful.<div><br></div><div>Thanks!<br clear="all"><div><br></div>-- <br><div style><font size="1">Dave</font></div>
</div></div>