[rabbitmq-discuss] Protection against slow consumers
matthias at rabbitmq.com
Mon Nov 1 21:35:34 GMT 2010
Christian Legnitto wrote:
> Nope, just the resources. I want to have tiers of service, where
> known fast-consuming consumers get no queue limits (to absorb message
> spikes), corporation employees and responsive external tools get
> fairly high limits, and random tools by the general public get fairly
> low. The public aspect is important as we are open and want anyone to
> be able to play with our data easily (without causing a DOS for
> critical applications).
Understood. The main problem with per-queue message count or TTL limits
is that they are a very poor indicator of disk usage due to variations
in message rates (in case of TTLs), message sizes and, most importantly,
the degree of sharing (i.e. the same message held by multiple queues
being written to disk only once).
Another problem with per-queue limits in general is the massive resource
underutilisation that one gets as a result of having to set the limits
on a worst case basis, i.e. "what if all consumers stopped consuming
from their queues simultaneously".
So the question is what controls could we put in place that address
these issues and bound the amount of message data held by RabbitMQ?
More information about the rabbitmq-discuss