[rabbitmq-discuss] Protection against slow consumers

[Matthias Radestock]

Christian,

Christian Legnitto wrote:
> Nope, just the resources. I want to have tiers of service, where
> known fast-consuming consumers get no queue limits (to absorb message
> spikes), corporation employees and responsive external tools get
> fairly high limits, and random tools by the general public get fairly
> low. The public aspect is important as we are open and want anyone to
> be able to play with our data easily (without causing a DOS for
> critical applications).

Understood. The main problem with per-queue message count or TTL limits 
is that they are a very poor indicator of disk usage due to variations 
in message rates (in case of TTLs), message sizes and, most importantly, 
the degree of sharing (i.e. the same message held by multiple queues 
being written to disk only once).

Another problem with per-queue limits in general is the massive resource 
underutilisation that one gets as a result of having to set the limits 
on a worst case basis, i.e. "what if all consumers stopped consuming 
from their queues simultaneously".

So the question is what controls could we put in place that address 
these issues and bound the amount of message data held by RabbitMQ?


Regards,

Matthias.