[rabbitmq-discuss] Is it possible to create/bind a queue so that it has a limit of N messages

[Matthias Radestock]

Mark,

Mark Geib wrote:
> Is it possible to create a queue so that the broker will limit the
> number of messages in the queue.? I am looking for a simple way to limit
> resource usage of a broker should things go wrong, like a very chatty
> producer... This is for a production system. I don't care if the
> messages are lost or not.

It would be fairly easy to add a per-queue message count limit to 
RabbitMQ, but it generally falls way short of constraining resources in 
a meaningful way to cope with "chatty" producers. Some reasons for that are:
- queues are not directly associated with producers - in principle a 
producer can send messages that end up in any (and any number of) queue(s).
- message counts do not constitute a good measure of resource usage; 
memory consumption would be a better measure but that is hard to 
determine with any degree of accuracy, due to the complexity of the data 
structures involved and the sharing of message data between queues

We are planning to add support for various kinds of rate limits such as 
per-connection message count and data rates. These should control 
resource usage effectively, but requires some careful design, and the 
implementation is far from trivial. So don't expect these features to 
appear any time soon.

Another option is to perform rate limiting in a proxy. That is something 
you may want to consider - writing an AMQP protocol proxy is quite 
straightforward, and with a specific use case in mind the implementation 
of the limiting may not be too complex either.

Yet another option might be to deploy a generic TCP firewall with rate 
limiting capabilities.


Regards,

Matthias.