[rabbitmq-discuss] RabbitMQ 'hang' when trying to access HA queues

[Andrew Rendell]

Hi Matthias,

Suchak and I got together and came up with a set of questions regarding our issue which if you were able to give feedback on, would be most useful.

We cannot be sure that during our 'hang' that publishers were blocked but from application behaviour, it seems likely, or at least they did not receive confirms. We do know that calls to queue declare (via the Java client) and rabbitmqctl list_queues both failed to return.

Our questions are:

- Are there any known conditions that you are aware of that would cause list_queues or declare queue to hang? If you are only able to indicate possible areas of the system that cause issue that would be helpful, e.g. if the MNESIA tables holding the queue metadata failed in some way (due to an infrastructure issue at the networking or disk level), could that explain one queue 'hanging'?
- The current documentation for RabbitMQ 3.x describes many situations where flow control kicks and publishers are blocked. We understand that flow control at 2.7.1 is rudimentary but it would simply involve publishers blocking (on basic publish) but could it in any way explain what we saw? (We are guessing not but wanted somebody who knows to confirm!).
- Some of our RabbitMQ clients are written in erlang and are embedded (with a significant amount of other application code) within the same cluster as RabbitMQ. Are there issues with this approach at 2.7.1?
- We are running the broker (and collocated application code) on Erlang R15B (erts-5.9) on Solaris SPARC. Again, are there any known issues with this configuration?

Thanks again for any support you can give us on this issue, I am aware you are going above and beyond the usual level of support especially given the version of RabbitMQ in use.

Regards, Andrew.

-----Original Message-----
From: Matthias Radestock [mailto:matthias at rabbitmq.com] 
Sent: 12 September 2013 11:55
To: Andrew Rendell
Cc: Discussions about RabbitMQ; Nick Gill (ngill at gopivotal.com); Suchak Jani
Subject: Re: [rabbitmq-discuss] RabbitMQ 'hang' when trying to access HA queues

Andrew,

On 12/09/13 11:41, Andrew Rendell wrote:
>> Was there anything unusual in the logs?
>
> No. This was what was most disconcerting about this issue.

Hmm. Quite.

> Would it be correct to say that since 2.7.1 there has been major 
> rewrites of the HA queue implementation

It's not been rewritten as such, but has undergone substantial change and improvement. There have been nearly 40 bug fixes to mirrored queues since 2.7.1.

> it is probable there were very few people using 2.7.1 in HA mode

...for a considerable length of time and/or with extensive testing...

> so a defect might have remain undiscovered?

Correct.

Matthias.



*****************************************************
This email is issued by a VocaLink group company. It is confidential and intended for the exclusive use of the addressee only. You should not disclose its contents to any other person. If you are not the addressee (or responsible for delivery of the message to the addressee), please notify the originator immediately by return message and destroy the original message. The contents of this email will have no contractual effect unless it is otherwise agreed between a specific VocaLink group company and the recipient.

The VocaLink group companies include, among others: VocaLink Limited (Company No 06119048, VAT No. 907 9619 87) which is registered in England and Wales at registered office Drake House, Homestead Road, Rickmansworth, WD3 1FX. United Kingdom, Voca Limited (Company no 1023742, VAT No. 907 9619 87) which is registered in England and Wales at registered office Drake House, Three Rivers Court, Homestead Road, Rickmansworth, Hertfordshire. WD3 1FX. United Kingdom, LINK Interchange Network Limited (Company No 3565766, VAT No. 907 9619 87) which is registered in England and Wales at registered office Drake House, Homestead Road, Rickmansworth, WD3 1FX and VocaLink Holdings Limited (Company No 06119036, VAT No. 907 9619 87) which is registered in England and Wales at registered office Drake House, Homestead Road, Rickmansworth, WD3 1FX. United Kingdom.

The views and opinions expressed in this email may not reflect those of any member of the VocaLink group. This message and any attachments have been scanned for viruses prior to leaving the VocaLink group network; however, VocaLink does not guarantee the security of this message and will not be responsible for any damages arising as a result of any virus being passed on or arising from any alteration of this message by a third party. The VocaLink group may monitor emails sent to and from the VocaLink group network.

*************************************************************