[rabbitmq-discuss] Server-Side Limit for a Channel's Unacknowledged Messages

[Dave Seltzer]

I just wanted to follow up on this question.

This morning I came in to find this situation:
[image: Inline image 1]

A partner who seems to be having issues has caused 2.3M messages to be
waiting for ACKs.

I'm quite sure that many of these messages should have expired, but are not
being deleted because they're waiting ACK. (I'm running RabbitMQ 3.0.1)

1) How should a message behave if it's TTL has expired and it's waiting for
ACK?
2) Is there really not that much interest setting a per-channel prefetch
queue limit?

Thanks so much!

-Dave






On Sat, Jan 12, 2013 at 3:28 PM, Dave Seltzer <dseltzer at tveyes.com> wrote:

> If you're asking me, then I think the answer might be the ability to set a
> limit on the number of un-acked messages per channel, and then the ability
> to limit the number of channels per user/vhost.
>
> Perhaps the new Policy framework in RabbitMQ 3.0+ would provide an
> interesting way to configure this behavior.
>
> I think that part of the problem is that most of the examples in the
> ClientAPI documentation don't touch on this issue, so a naive user, who
> copies and pastes an example, might find themselves with an enormous
> pre-fetch queue and never even know it. But, even if my users were more
> conscientious about setting this variable, I think there's still a
> compelling argument to be made that a broker-administrator should be able
> to limit it.
>
> Thanks for taking the time to explain the complexity!
>
> -D
>
>
> On Sat, Jan 12, 2013 at 9:34 AM, Matthias Radestock <matthias at rabbitmq.com
> > wrote:
>
>> Dave,
>>
>> On 10/01/13 13:54, Dave Seltzer wrote:
>>
>>> I was wondering if there's a way to set a policy on the broker that
>>> would effectively limit the size of the pre-fetch queue for clients?
>>>
>>
>> It is not uncommon for for novice users (and some clients) to forget to
>> ack messages, or accidentally buffer vast quantities of messages in the
>> client.
>>
>> We did consider setting basic.qos{prefetch_count=1} as a default, i.e.
>> all channels would be limited like that unless the app issues a basic.qos
>> of its own.
>>
>> Unfortunately that a) could break existing apps, and b) significantly
>> reduces throughput.
>>
>> As you say, one option would be to make this server configurable. Since
>> we try to keep the number of config settings small, this is really a last
>> resort. Still, it might be worth exploring...
>>
>> At what granularity should a prefetch count be configurable?
>>
>> a) server
>> b) user
>> c) vhost
>> d) user & vhost
>> e) peer ip
>> f) ...
>>
>>
>> Regards,
>>
>> Matthias
>>
>
>
>
> --
> Dave Seltzer <dseltzer at tveyes.com>
> Chief Systems Architect
> TVEyes
> (203) 254-3600 x222
>



-- 
Dave Seltzer <dseltzer at tveyes.com>
Chief Systems Architect
TVEyes
(203) 254-3600 x222
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130125/db0d92cd/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 3822 bytes
Desc: not available
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130125/db0d92cd/attachment.png>