[rabbitmq-discuss] Protection against slow consumers

[Jiri Krutil]

In our use case, we assume that consumers will be fast and the message  
flow fairly low. So normally there should be almost no messages kept  
on the broker.

What we are concerned about is a consumer that (willingly or not)  
subscribes but does not consume, or consumes but does not ack.  
(Consumers are not under our control and belong to different clients  
that use our API.)

Of course we don't want one such funny consumer to disrupt the message  
flow to all other consumers (by draining broker memory or disk).

So anything that will prevent this should be OK - per-queue message  
TTL or per-queue limit on message count/size.

Cheers
Jiri


On 1 Nov 2010, at 21:27, Alexis Richardson <alexis at rabbitmq.com> wrote:

> Christian, Jiri,
>
> What are your thoughts on per message TTLs for this?
>
> alexis
>
>
> On Mon, Nov 1, 2010 at 6:41 PM, Christian Legnitto
> <clegnitto at mozilla.com> wrote:
>> There is not something like http://activemq.apache.org/slow-consumer-handling.html 
>>  for Rabbit. You can sort of approximate it with a maintenance  
>> script that sweeps through and drains messages from a queue  
>> depending on a heuristic of your choice.
>>
>> We are interested in something like this too (see my message titled  
>> "Missing features making me look at moving off RabbitMQ" to the  
>> list).
>>
>> Thanks,
>> Christian
>>
>> On Nov 1, 2010, at 9:33 AM, jiri at krutil.com wrote:
>>
>>> Hi Emile
>>>
>>>> I assume you have subscribers each set up with their own queue  
>>>> and binding.
>>>
>>> Yes, each subscriber uses an exclusive auto-delete queue bound to  
>>> the exchange.
>>>
>>>> In this case I would expect long queues associated with slow  
>>>> subscribers to be persisted to disk automatically. This prevents  
>>>> RAM pressure, which the broker attempts to relieve by blocking  
>>>> producers.
>>>
>>> I see what you mean, but what if the subscriber uses a large  
>>> prefetch buffer and consumes without ever acking?
>>>
>>> Also, even if long queues get persisted to disk, this consumes  
>>> resources and the queues can't grow forever.
>>>
>>> Is there no way how can we limit the queue size or message time to  
>>> live?
>>>
>>>> This assumes that you are running version 2.0.0 or later.
>>>
>>> We are running RabbitMQ 2.1.0.
>>>
>>> Cheers
>>> Jiri
>>>
>>> _______________________________________________
>>> rabbitmq-discuss mailing list
>>> rabbitmq-discuss at lists.rabbitmq.com
>>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.rabbitmq.com
>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>