[rabbitmq-discuss] Limit QueueingConsumer Memory Usage

Fri Jul 10 21:36:09 BST 2009

Hi Amit,

Thanks for the answer. So this QoS is only in trunk of the Java client. 
I will have to check this out.

Using the DefaultConsumer means extending it anyways, right? As by 
default it does not do anything. Even setting QoS to 1 would mean you 
will have to enforce ack'ing the message or else the async process will 
keep sending the next one. So a simple single local Delivery bean 
instance could hold the current message.

Saves the queue part. But then you also have to lock on that instance 
for the blocking behavior, which the QueueingConsumer has build in.

Am I on the right track here?

Regards,
Lars

amit bhatnagar wrote:
> Having said this, I am wondering if it makes sense to use a
> QueueingConsumer on a channel that has a basicQos of 1? Would I be
> safe to use a DefaultConsumer here instead?
>
> On Jul 10, 8:23 am, amit bhatnagar <brownsp... at gmail.com> wrote:
>   
>> Check out Channel.BasicQos() and set a prefetch count.
>>
>> Have a look here for a good description of the prefetch:http://hopper.squarespace.com/blog/2009/1/26/work-distribution-in-rab...
>>
>> fta:
>>
>> "This command allows a consumer to choose a prefetch window that
>> specifies the amount of unacknowledged messages it is prepared to
>> receive. By setting the prefetch count to a non-zero value, the broker
>> will not deliver any messages to the consumer that would breach that
>> limit. To move the window forwards, the consumer has to acknowledge
>> the receipt of a message (or a group of messages). By acknowledging a
>> message, the consumer gains credit in the broker which makes it
>> eligible to receive more messages. This credit-based flow control
>> allows the broker to distribute work proportional to the individual
>> processing ability of each worker, as opposed to a simple round robin
>> mechanism."
>>
>> On Jul 10, 4:32 am, Lars George <l... at worldlingo.com> wrote:
>>
>>
>>
>>     
>>> Hi,
>>>       
>>> I would like to use a blocking provider, but the QueueingConsumer is
>>> somewhat limiting as it receives whatever the queue sends and caches it
>>> locally, in the app servers memory. If that is a very large number then
>>> you can quickly run out of memory and kill the Java process with an OOME.
>>>       
>>> Is there a way to implement a Consumer that say only receives N queue
>>> items before it waits until they get processed locally? I mean from the
>>> internal BlockingQueue and using handleDelivery() this is doable but
>>> then you would block the main loop in the AMPQ connection - and miss
>>> heart beats etc.?
>>>       
>>> Is there a better way with this or do I have to go with a dumb while
>>> (true) { channel.basicGet() } loop. With that you have the issue to
>>> somehow gracefully handle the null delivery and not create a loop that
>>> consumes all CPU cycles with no actual work being done.
>>>       
>>> Thanks,
>>> Lars
>>>       
>>>  lars.vcf
>>> < 1KViewDownload
>>>       
>>> _______________________________________________
>>> rabbitmq-discuss mailing list
>>> rabbitmq-disc... at lists.rabbitmq.comhttp://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>>       
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-disc... at lists.rabbitmq.comhttp://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>     
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>   
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lars.vcf
Type: text/x-vcard
Size: 301 bytes
Desc: not available
Url : http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20090710/3ba9de13/attachment.vcf 