[rabbitmq-discuss] Limit QueueingConsumer Memory Usage

[Ben Hood]

Lars,

On 14 Jul 2009, at 07:06, Lars George <lars at worldlingo.com> wrote:

> Hi Amit,
>
> We upgraded to 1.6.0 and I added for example channel.basicQos(30);  
> but to no avail. My test program still fills the local queue way  
> past the 30 message limit using the QueueingConsumer and noAck set  
> to false.
>
> Is this in a later version? Am I doing something wrong?
>

Interesting. Can you send a cut down version of the code that  
reproduces the behaviour and to make sure that are you are definetly  
not auto-acking anything?

Ben

> Regards,
> Lars
>
>
> amit bhatnagar wrote:
>> Check out Channel.BasicQos() and set a prefetch count.
>>
>> Have a look here for a good description of the prefetch:
>> http://hopper.squarespace.com/blog/2009/1/26/work-distribution-in-rabbitmq.html
>>
>> fta:
>>
>> "This command allows a consumer to choose a prefetch window that
>> specifies the amount of unacknowledged messages it is prepared to
>> receive. By setting the prefetch count to a non-zero value, the  
>> broker
>> will not deliver any messages to the consumer that would breach that
>> limit. To move the window forwards, the consumer has to acknowledge
>> the receipt of a message (or a group of messages). By acknowledging a
>> message, the consumer gains credit in the broker which makes it
>> eligible to receive more messages. This credit-based flow control
>> allows the broker to distribute work proportional to the individual
>> processing ability of each worker, as opposed to a simple round robin
>> mechanism."
>>
>>
>> On Jul 10, 4:32 am, Lars George <l... at worldlingo.com> wrote:
>>
>>> Hi,
>>>
>>> I would like to use a blocking provider, but the QueueingConsumer is
>>> somewhat limiting as it receives whatever the queue sends and  
>>> caches it
>>> locally, in the app servers memory. If that is a very large number  
>>> then
>>> you can quickly run out of memory and kill the Java process with  
>>> an OOME.
>>>
>>> Is there a way to implement a Consumer that say only receives N  
>>> queue
>>> items before it waits until they get processed locally? I mean  
>>> from the
>>> internal BlockingQueue and using handleDelivery() this is doable but
>>> then you would block the main loop in the AMPQ connection - and miss
>>> heart beats etc.?
>>>
>>> Is there a better way with this or do I have to go with a dumb while
>>> (true) { channel.basicGet() } loop. With that you have the issue to
>>> somehow gracefully handle the null delivery and not create a loop  
>>> that
>>> consumes all CPU cycles with no actual work being done.
>>>
>>> Thanks,
>>> Lars
>>>
>>> lars.vcf
>>> < 1KViewDownload
>>>
>>> _______________________________________________
>>> rabbitmq-discuss mailing list
>>> rabbitmq-disc... at lists.rabbitmq.comhttp://lists.rabbitmq.com/cgi- 
>>> bin/mailman/listinfo/rabbitmq-discuss
>>>
>>
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.rabbitmq.com
>> http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>
>>
> <lars.vcf>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss