[rabbitmq-discuss] RabbitMQ queue out-rate settles to in-rate when many messages in queue

[Jelle Smet]

Hi all,

RabbitMQ: 3.3.1
Erlang: 17

We have following situation:


   - A queue contains a large amount of messages (5 million because the 
   consumers are shutdown)
   - There's a constant  incoming message flow at a rate of ~2000 msg/s.
   - Messages are persistent though diskIO is far from being saturated.
   - Consumers have a prefetch value of 1000
   - We presume backpressure is applied to the producer in this situation 
   (difficult to observe).


What we observe is that out-rate settles to the in-rate, which I don't 
understand because:

   - There a large amount of messages in the queue so plenty to consume.
   - We KNOW consumers are able to write at a much higher throughput even 
   after bumping prefetch value.
   - When the queue is empty, in & outgoing rate is again 8000 msg/s


Any tips, leads why the consumption rate settles to the incoming rate in 
this situation?

cheers,

Jelle



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20140512/b840602d/attachment.html>