[rabbitmq-discuss] Protection against slow consumers

Mon Nov 1 20:52:16 GMT 2010

Nope, just the resources. I want to have tiers of service, where known fast-consuming consumers get no queue limits (to absorb message spikes), corporation employees and responsive external tools get fairly high limits, and random tools by the general public get fairly low. The public aspect is important as we are open and want anyone to be able to play with our data easily (without causing a DOS for critical applications).

Thanks,
Christian

On Nov 1, 2010, at 1:44 PM, Alexis Richardson wrote:

> I see, ok.
> 
> Is there any reason to do this, other than conserving physical resources?
> 
> alexis
> 
> 
> 
> On Mon, Nov 1, 2010 at 8:41 PM, Christian Legnitto
> <clegnitto at mozilla.com> wrote:
>> For my use-case it isn't exactly what I want.
>> 
>> If a user is listening for a very specific message (say "bug.changed.review.requested.foo at example.com") and they only have 2 items in the queue, I don't really want to delete those items because a pre-determined time has passed. On the flip-side, if I set the TTL to one day and the queue suddenly balloons to 100k within an hour, I want to limit the queue rather than waiting the one day for the messages to expire.
>> 
>> Thanks,
>> Christian
>> 
>> 
>> On Nov 1, 2010, at 1:27 PM, Alexis Richardson wrote:
>> 
>>> Christian, Jiri,
>>> 
>>> What are your thoughts on per message TTLs for this?
>>> 
>>> alexis
>>> 
>>> 
>>> On Mon, Nov 1, 2010 at 6:41 PM, Christian Legnitto
>>> <clegnitto at mozilla.com> wrote:
>>>> There is not something like http://activemq.apache.org/slow-consumer-handling.html for Rabbit. You can sort of approximate it with a maintenance script that sweeps through and drains messages from a queue depending on a heuristic of your choice.
>>>> 
>>>> We are interested in something like this too (see my message titled "Missing features making me look at moving off RabbitMQ" to the list).
>>>> 
>>>> Thanks,
>>>> Christian
>>>> 
>>>> On Nov 1, 2010, at 9:33 AM, jiri at krutil.com wrote:
>>>> 
>>>>> Hi Emile
>>>>> 
>>>>>> I assume you have subscribers each set up with their own queue and binding.
>>>>> 
>>>>> Yes, each subscriber uses an exclusive auto-delete queue bound to the exchange.
>>>>> 
>>>>>> In this case I would expect long queues associated with slow subscribers to be persisted to disk automatically. This prevents RAM pressure, which the broker attempts to relieve by blocking producers.
>>>>> 
>>>>> I see what you mean, but what if the subscriber uses a large prefetch buffer and consumes without ever acking?
>>>>> 
>>>>> Also, even if long queues get persisted to disk, this consumes resources and the queues can't grow forever.
>>>>> 
>>>>> Is there no way how can we limit the queue size or message time to live?
>>>>> 
>>>>>> This assumes that you are running version 2.0.0 or later.
>>>>> 
>>>>> We are running RabbitMQ 2.1.0.
>>>>> 
>>>>> Cheers
>>>>> Jiri
>>>>> 
>>>>> _______________________________________________
>>>>> rabbitmq-discuss mailing list
>>>>> rabbitmq-discuss at lists.rabbitmq.com
>>>>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>>> 
>>>> _______________________________________________
>>>> rabbitmq-discuss mailing list
>>>> rabbitmq-discuss at lists.rabbitmq.com
>>>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>>> 
>> 
>> 