[rabbitmq-discuss] The rabbitmq-server stop command hangs

[Tim Watson]

Liz,

Ok - what does `rabbitmqctl eval 'application:which_applications().' look like?

Also, can you grab https://github.com/downloads/hyperthunk/suptree_inspect/suptree_inspect.beam and run it like so:

  $ escript suptree_inspect.beam rabbit at host rabbit_sup | tee output.log

Obviously replacing rabbit at host with the correct node name. This should put a bunch of noise into output.log which I'd like to see! Please feel free to scrub any sensitive data before posting, though I very much doubt there will be any.

That tool btw was hacked together very quickly and compiled against R15B01, so it *may* bomb out on you, in which case we'll fall back to doing some more 'eval' until we can figure out why your rabbit is stuck.

Cheers,
Tim

On 8 Nov 2012, at 18:42, Elizabeth Liao wrote:

> Hi Tim,
> 
> I finally reproduced an instance where rabbitmq-server appears to hang.  Here's the the output of the command:
> #  rabbitmqctl eval 'process_info(whereis(rabbit_channel_sup_sup)).'
> Error: {badarg,[{erlang,process_info,[undefined],[]},
>                {erl_eval,do_apply,6,[{file,"erl_eval.erl"},{line,572}]},
>                {rpc,'-handle_call_call/6-fun-0-',5,
>                     [{file,"rpc.erl"},{line,203}]}]}
> 
> Liz
> 
> ________________________________________
> From: rabbitmq-discuss-bounces at lists.rabbitmq.com [rabbitmq-discuss-bounces at lists.rabbitmq.com] on behalf of Tim Watson [tim at rabbitmq.com]
> Sent: Thursday, November 08, 2012 3:34 AM
> To: Discussions about RabbitMQ
> Subject: Re: [rabbitmq-discuss] The rabbitmq-server stop command hangs
> 
> Liz,
> 
> On 7 Nov 2012, at 18:55, Tim Watson wrote:
> 
>> Hi Liz,
>> 
>> On 11/07/2012 02:49 PM, Elizabeth Liao wrote:
>>> Hi,
>>> 
>>> I still haven't been able to get this working.  Anyone have any suggests to diagnose the problem?
>> 
>> Sorry we haven't got back to you sooner. I think some investigation on a stuck node will be necessary. If you can give me ssh access to the machine where the shutdown is hanging then I can do this for you, otherwise I'll write up a module you can compile and execute using escript and send it to you. I have actually started on that latter idea anyway, but it's getting a bit late now so I'll probably finish it off in the morning! :)
>> 
> 
> Actually come to think of it, we could check a couple of things briefly using rabbitmqctl first. Can you send the output of this please: rabbitmqctl eval 'process_info(whereis(rabbit_channel_sup_sup)).'
> 
>> Cheers,
>> Tim
>> 
>>> Thanks a lot!
>>> 
>>> Liz
>>> ________________________________________
>>> From: rabbitmq-discuss-bounces at lists.rabbitmq.com [rabbitmq-discuss-bounces at lists.rabbitmq.com] on behalf of Elizabeth Liao [eliao at seegrid.com]
>>> Sent: Monday, November 05, 2012 3:48 PM
>>> To: rabbitmq-discuss at lists.rabbitmq.com
>>> Subject: [rabbitmq-discuss] rabbitmq-server stop hangs
>>> 
>>> Hi,
>>> 
>>> I am currently running rabbitmq-server-2.8.7 and erlang-R15B on FC11.  I'm having a problem where rabbit sometimes hangs on this command:
>>> /etc/init.d/rabbitmq-server stop such that rabbitmqctl status hangs as well with this output:
>>> Status of node rabbit at test1 ...
>>> 
>>> The only error report that I see in in the rabbit at test1.log file is:
>>> =ERROR REPORT==== 5-Nov-2012::16:23:51 ===
>>> connection<0.526.0>, channel 5 - error:
>>> {amqp_error,channel_error,"expected 'channel.open'",'basic.ack'}
>>> 
>>> I don't see anything logged in the rabbit at test1-sasl.log file.
>>> 
>>> I realize that this isn't very much information to go on so if anyone could offer more insight into this problem, it would be greatly appreciated!
>>> 
>>> Thanks.
>>> 
>>> Liz
>>> 
>>> 
>>> Email Confidentiality Notice
>>> 
>>> The information contained in this transmission is confidential, proprietary or privileged and may be subject to protection under the law. This message is intended for the sole use of the individual or entity to whom it's addressed. If you are not the intended recipient, you are notified that any use, distribution or copying of the message is strictly prohibited and may subject you to criminal or civil penalties. If you received this transmission in error, please contact the sender immediately by replying to this email and delete the material from any computer.
>>> _______________________________________________
>>> rabbitmq-discuss mailing list
>>> rabbitmq-discuss at lists.rabbitmq.com
>>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>> Email Confidentiality Notice
>>> 
>>> The information contained in this transmission is confidential, proprietary or privileged and may be subject to protection under the law. This message is intended for the sole use of the individual or entity to whom it's addressed. If you are not the intended recipient, you are notified that any use, distribution or copying of the message is strictly prohibited and may subject you to criminal or civil penalties. If you received this transmission in error, please contact the sender immediately by replying to this email and delete the material from any computer.
>>> _______________________________________________
>>> rabbitmq-discuss mailing list
>>> rabbitmq-discuss at lists.rabbitmq.com
>>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>> 
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.rabbitmq.com
>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> 
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> Email Confidentiality Notice
> 
> The information contained in this transmission is confidential, proprietary or privileged and may be subject to protection under the law. This message is intended for the sole use of the individual or entity to whom it's addressed. If you are not the intended recipient, you are notified that any use, distribution or copying of the message is strictly prohibited and may subject you to criminal or civil penalties. If you received this transmission in error, please contact the sender immediately by replying to this email and delete the material from any computer.
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss