[rabbitmq-discuss] Throttling when Fast Producer, Slow Consumer

[Emile Joubert]

Hi Paul,

On 13/03/12 22:26, Bell, Paul M. wrote:
> What are the Rabbit "best practices" ways of handling a situation
> where the consumer is v-e-r-y slow and the producer is super-fast.
> IOW: I am trying to understand the best ways to throttle the producer
> in such a case. Assume that neither queue nor its topic exchange are
> durable.
> 
> Can producer obtain information about the state of the queue, e.g.,
> the number of unACKed messages, its size in bytes or number of
> messages, etc? Or is another approach indicated?

There's no single best way to deal with mismatched rates of production
and consumption of messages. The simplest option is to ignore it,
assuming the mismatch is temporary. Queues may get very long and need to
spill over to disk, but they will shorten once producers slow down or
consumers speed up.

If the rate mismatch is a permanent feature then you ought to reconsider
the design. If producers know which queues their messages are destined
for then they could monitor the lengths of those queues. Another option
is to limit the lifetime of messages in destination queues with the
"x-message-ttl" parameter.


-Emile