<div dir="ltr">We currently use RabbitMQ as a broker for a subset of our overall logging platform. Recently, we attempted to add an additional category of logging that would result in additional load of approximately 2,000 msg/sec with an average message size of approximately 2KB.<br>
<br><div>Upon adding this additional category of logging, we noticed a couple things:<br><br>(1) Our consumers were not able to keep up with the flow of messages, so they began to build up in the queue (our developers are working on fixing this).<br>
<br>(2) We began to see RabbitMQ invoke flow-control in the form of TCP back-pressure on our producers.</div><div><br></div><div>My initial thought was that since we were using durable queues with persistent messages that when the consumers weren't immediately picking up the messages, RabbitMQ was attempting to persist them to disk in order to satisfy the persistence & durability requirements and it couldn't keep up.<br>
<br>But in following up with my developers this morning it seems that they are in fact no publishing the messages as persistent and being that (based on my understanding) a non-persistent message published to a durable queue is not written to disk, this threw my initial theory out the windows (I still have to verify their claim of non-persistence).<br>
<br>So, my question is - what other things can I look at to determine the cause of the invocation of flow control. Additional relevant information is as follows:<br><br>(*) 2x RabbitMQ Nodes Running 3.1.3 in a clustered configuration.<br>
(*) Each node has 8x 2GHzCPU cores and 4GB of RAM.<br>(*) At no time during the window in question did CPU utilization reach >15%<br>(*) At no time during the window in question did memory utilization reach >50%<br>
(*) In looking at the logs during the window in question I can see events which I think are associated with the various publishers disconnecting (I am not sure if our producer code is handling flow-control gracefully) but nothing that seems to indicate why this happened.<br>
<br>Any help would be greatly appreciated.</div><div><br></div><div>Regards,</div><div><br></div><div>Richard</div></div>