Folks,<div><br></div><div> We are running a high scale setup with rabbitmq 3.0.1 , flow:</div><div><br></div><div>logstash 1.1.12 producer --> exchange --> fanout queue </div><div> | |</div><div> | |</div><div> consumer Consumer</div><div> A B</div><div><br></div><div>Consumers as logstash , consuming to graphite as output , our message rate us ~ 12000 messages/s , FanOut Exchange , Durable: False , Persistent: False , No Memory problems</div><div><br></div><div>We are facing a problem where at random times rabbitmq starts to block producers due to flow control (rabbitmqctl list_connections) , this is to our understanding </div><div>in order to protect the broker and the consumer as the producers are faster then the consumer ?</div><div><br></div><div>The problem is that at some point (when the READY messages count is high ~ 1M) rabbitmq blocks most of our producers and does not seem to release them , until we restart them (logstash daemon) on every one of them , we tried purging the queue / restarting rabbitmq , only restarting the producers seems to bring things to normal state.</div><div><br></div><div>I guess my questions are:</div><div><br></div><div> - Is my problem on the consumer side ? I am unable to debug the consumer speed or state </div><div> - Can I force flow control to release blocked producers?</div><div> - Can I tune rabbitmq for lots of connections and high message rate?</div><div> - We use fanout exchange , when a consumer creates a new queue under this exchange and does not consume fast enough can he effect the producers from the other queue (i.e cause them to be blocked?)</div><div><br></div><div>Thanks</div><div>Kobi </div><div><br></div><div> </div>
<br>
<div><font size="1"><font face="Arial, Helvetica, sans-serif">The above terms reflect a potential business arrangement, are provided </font><font face="Arial, Helvetica, sans-serif">solely as a basis for further discussion, and are not intended to be and do </font><span style="font-family:Arial,Helvetica,sans-serif">not constitute a legally binding obligation. No legally binding obligations </span><span style="font-family:Arial,Helvetica,sans-serif">will be created, implied, or inferred until an agreement in final form is </span><span style="font-family:Arial,Helvetica,sans-serif">executed in writing by all parties involved.</span></font></div><div><font face="Arial, Helvetica, sans-serif" size="1"><br></font></div><div><font size="1"><font face="Arial, Helvetica, sans-serif">This email and any attachments hereto may be confidential or privileged. If you received this </font><span style="font-family:Arial,Helvetica,sans-serif">communication by mistake, please don't forward it to anyone else, please </span><span style="font-family:Arial,Helvetica,sans-serif">erase all copies and attachments, and please let me know that it has gone </span><span style="font-family:Arial,Helvetica,sans-serif">to the wrong person. Thanks.</span></font></div>