Some other work came up so I needed to drop this thread for a few weeks. However, coming back to it, I can easily reproduce this issue within one or two tries.<br><br>In a nutshell, in a clustered environment, simply stop one node, wait a few seconds, then restart it. The last output seen is:<br>
<br>starting database����������������������<div id=":7q">������������������������������ ...<br><br></div>I've let it wait for much longer than 30 seconds and it has never come back.<br><br>Any chance this may have been stamped out in RabbitMQ 2.8? <br>
<br><br><br><div class="gmail_quote">On Fri, Feb 24, 2012 at 1:43 PM, Matt Pietrek <span dir="ltr"><<a href="mailto:mpietrek@skytap.com">mpietrek@skytap.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im">| So how long are you waiting when determining it's hanging? Less than 30 seconds?<br><br></div>Just to be double sure, I let it sit for an hour yesterday. I would have expected a timeout, but it never came.<br>
<br>It's a pretty easy scenario to script and try out. I'd send you my code, but it relies on other internal commands.<br>
<br>There may also be a timing issue. If I put a 10 second delay after restarting one broker, and before stopping the next, it seems to help.<br><br>That is:<br><br>for x in broker_list:<br>��� stop x<br>��� start x<br>��� sleep(10)<span class="HOEnZb"><font color="#888888"><br>
<br>Matt</font></span><div class="HOEnZb"><div class="h5"><br><br><div class="gmail_quote">On Fri, Feb 24, 2012 at 4:22 AM, Simon MacMullen <span dir="ltr"><<a href="mailto:simon@rabbitmq.com" target="_blank">simon@rabbitmq.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>On 23/02/12 21:00, Matt Pietrek wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
The nohup.out on the failing node ends with:<br>
</blockquote>
<br></div>
<snip><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
starting database � � � � � � � � � � � � � � � � � � � � � � � � � � ...<br>
</blockquote>
<br>
So how long are you waiting when determining it's hanging? Less than 30 seconds?<br>
<br>
Because that looks like Rabbit is waiting for another cluster node (if it was not the last to shut down, but is the first to start up, it will wait for the one that was the last to shut down. But it will only wait for 30 seconds before spitting out an error. I'm not sure how else you could get it to stop there *without* any further output though.<div>
<div><br>
<br>
Cheers, Simon<br>
<br>
-- <br>
Simon MacMullen<br>
RabbitMQ, VMware<br>
</div></div></blockquote></div><br>
</div></div></blockquote></div><br>