[rabbitmq-discuss] Why consumer become slow if publish faster?

[Simon MacMullen]

On 07/02/2014 1:04PM, 椰风 wrote:
> Why the speed of the consumer reduced to 1200/s when speed of the
> publish grown to 4000/s?

Each queue can only use a single CPU core. Therefore as the publish rate 
goes up, the maximum consume rate goes down.

At the moment, the queue does not make any effort to prioritise 
consumers over publishers in this situation, leading to backlogs as you 
have seen.

In 3.3.0 we will be introducing a prioritisation feature so that when a 
queue is CPU-bound it will attempt to deliver messages 10% faster than 
it accepts them, leading it to tend towards being empty rather than 
growing uncontrollably.

(Historical note: we had a simpler prioritisation scheme from 1.7.0 to 
2.8.2 inclusive, which "infinitely" prioritised consuming when the queue 
was CPU bound. The trouble was you could get into situations where a 
large queue which gained some fast consumers would refuse to accept even 
a single publish until it was completely drained.)

Cheers, Simon

-- 
Simon MacMullen
RabbitMQ, Pivotal