[rabbitmq-discuss] blocked producers
kbiton at outbrain.com
Thu Jul 18 22:11:35 BST 2013
We are running a high scale setup with rabbitmq 3.0.1 , flow:
logstash 1.1.12 producer --> exchange --> fanout queue
Consumers as logstash , consuming to graphite as output , our message rate
us ~ 12000 messages/s , FanOut Exchange , Durable: False , Persistent:
False , No Memory problems
We are facing a problem where at random times rabbitmq starts to block
producers due to flow control (rabbitmqctl list_connections) , this is to
in order to protect the broker and the consumer as the producers are faster
then the consumer ?
The problem is that at some point (when the READY messages count is high ~
1M) rabbitmq blocks most of our producers and does not seem to release
them , until we restart them (logstash daemon) on every one of them , we
tried purging the queue / restarting rabbitmq , only restarting the
producers seems to bring things to normal state.
I guess my questions are:
- Is my problem on the consumer side ? I am unable to debug the consumer
speed or state
- Can I force flow control to release blocked producers?
- Can I tune rabbitmq for lots of connections and high message rate?
- We use fanout exchange , when a consumer creates a new queue under this
exchange and does not consume fast enough can he effect the producers from
the other queue (i.e cause them to be blocked?)
The above terms reflect a potential business arrangement, are provided solely
as a basis for further discussion, and are not intended to be and do not
constitute a legally binding obligation. No legally binding obligations will
be created, implied, or inferred until an agreement in final form is executed
in writing by all parties involved.
This email and any attachments hereto may be confidential or privileged.
If you received this communication by mistake, please don't forward it to
anyone else, please erase all copies and attachments, and please let me
know that it has gone to the wrong person. Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the rabbitmq-discuss