[rabbitmq-discuss] blocked producers

[Kobi Biton]

Folks,

 We are running a high scale setup with rabbitmq 3.0.1 , flow:

logstash 1.1.12 producer --> exchange -->   fanout queue 
                                                              |         |
                                                              |         |
                                                         consumer Consumer
                                                              A             
B

Consumers as logstash , consuming to graphite as output , our message rate 
us ~ 12000 messages/s , FanOut Exchange , Durable: False , Persistent: 
False , No Memory problems

We are facing a problem where at random times rabbitmq starts to block 
producers due to flow control (rabbitmqctl list_connections) , this is to 
our understanding 
in order to protect the broker and the consumer as the producers are faster 
then the consumer ?

The problem is that at some point (when the READY messages count is high ~ 
1M)  rabbitmq blocks most of our producers and does not seem to release 
them , until we restart them (logstash daemon) on every one of them , we 
tried purging the queue / restarting rabbitmq , only restarting the 
producers seems to bring things to normal state.

I guess my questions are:

 - Is my problem on the consumer side ? I am unable to debug the consumer 
speed or state 
 - Can I force flow control to release blocked producers?
 - Can I tune rabbitmq for lots of connections and high message rate?
 - We use fanout exchange , when a consumer creates a new queue under this 
exchange and does not consume fast enough can he effect the producers from 
the other queue (i.e cause them to be blocked?)

Thanks
Kobi 

 

-- 
The above terms reflect a potential business arrangement, are provided solely 
as a basis for further discussion, and are not intended to be and do not 
constitute a legally binding obligation. No legally binding obligations will 
be created, implied, or inferred until an agreement in final form is executed 
in writing by all parties involved.

This email and any attachments hereto may be confidential or privileged. 
 If you received this communication by mistake, please don't forward it to 
anyone else, please erase all copies and attachments, and please let me 
know that it has gone to the wrong person. Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130718/09e46f90/attachment.htm>