[rabbitmq-discuss] Protection against slow consumers
clegnitto at mozilla.com
Mon Nov 1 20:41:39 GMT 2010
For my use-case it isn't exactly what I want.
If a user is listening for a very specific message (say "bug.changed.review.requested.foo at example.com") and they only have 2 items in the queue, I don't really want to delete those items because a pre-determined time has passed. On the flip-side, if I set the TTL to one day and the queue suddenly balloons to 100k within an hour, I want to limit the queue rather than waiting the one day for the messages to expire.
On Nov 1, 2010, at 1:27 PM, Alexis Richardson wrote:
> Christian, Jiri,
> What are your thoughts on per message TTLs for this?
> On Mon, Nov 1, 2010 at 6:41 PM, Christian Legnitto
> <clegnitto at mozilla.com> wrote:
>> There is not something like http://activemq.apache.org/slow-consumer-handling.html for Rabbit. You can sort of approximate it with a maintenance script that sweeps through and drains messages from a queue depending on a heuristic of your choice.
>> We are interested in something like this too (see my message titled "Missing features making me look at moving off RabbitMQ" to the list).
>> On Nov 1, 2010, at 9:33 AM, jiri at krutil.com wrote:
>>> Hi Emile
>>>> I assume you have subscribers each set up with their own queue and binding.
>>> Yes, each subscriber uses an exclusive auto-delete queue bound to the exchange.
>>>> In this case I would expect long queues associated with slow subscribers to be persisted to disk automatically. This prevents RAM pressure, which the broker attempts to relieve by blocking producers.
>>> I see what you mean, but what if the subscriber uses a large prefetch buffer and consumes without ever acking?
>>> Also, even if long queues get persisted to disk, this consumes resources and the queues can't grow forever.
>>> Is there no way how can we limit the queue size or message time to live?
>>>> This assumes that you are running version 2.0.0 or later.
>>> We are running RabbitMQ 2.1.0.
>>> rabbitmq-discuss mailing list
>>> rabbitmq-discuss at lists.rabbitmq.com
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.rabbitmq.com
More information about the rabbitmq-discuss