[rabbitmq-discuss] Garbage Collection Problem in Consumer

Wed Feb 3 05:12:52 GMT 2010

I wonder if you are running into a problem where you are fetching far too
many messages to consume, while each/some of your messages spend 2-3 seconds
in processing.

Explore the prefetch options. I ran into something similar and started
using

http://gist.github.com/293346

Pradeep
www.masplantiz.com

On Tue, Feb 2, 2010 at 6:59 PM, Seema Biradar <seemasbiradar at gmail.com>wrote:

> Thanks for the information.
>
>
> On Tue, Feb 2, 2010 at 6:42 PM, Gustavo Aquino <aquino.gustavo at gmail.com>wrote:
>
>> Seema,
>>
>> I think that your performance problem is in piece of code that you
>> dropped, this dropped code probably are reusing Rabbitmq objects references,
>> and this guys are going to Old memory and will be collected in different
>> time, this cause your feeling about "Rabbit object aren't eligible to GC".
>>
>> Take a look about how you are using Rabbit objects inside your code. Try
>> to use only the message like a TO.
>>
>> About republish message, If you are publishing it in the same queue that
>> you consume, you can just get message, process it and if OK remove it from
>> queue using ..ask instead publish it in the end of your flow.
>>
>> I don't saw any performance problems in this piece of code posted here.
>>
>>
>> On Tue, Feb 2, 2010 at 10:29 AM, Seema Biradar <seemasbiradar at gmail.com>wrote:
>>
>>> Hi ,
>>>
>>> Please find the attached code for Consumer.java.
>>>
>>> public class SingleConsumer{
>>>
>>>  public static void main(String args[]) {
>>>
>>>
>>>  ConnectionParameters params = new ConnectionParameters();
>>>
>>> params.setUsername(janusProperties.getProperty("rabbitmq_userName").trim());
>>>
>>> params.setPassword(janusProperties.getProperty("rabbitmq_password").trim());
>>>             params.setVirtualHost("/");
>>>             params.setRequestedHeartbeat(0);
>>>             ConnectionFactory factory = new ConnectionFactory(params);
>>>             com.rabbitmq.client.Connection connRMQ =
>>> factory.newConnection(janusProperties.getProperty("rabbitmq_server_ip").trim(),
>>> Integer.parseInt(janusProperties.getProperty("rabbitmq_server_port").trim()));
>>>
>>>
>>>             Channel channel = connRMQ.createChannel();
>>>
>>>             String exchangeName =
>>> janusProperties.getProperty("rabbitmq_exchangeName").trim();
>>>             String queueName =
>>> janusProperties.getProperty("rabbitmq_queueName").trim();
>>>             String routingKey =
>>> janusProperties.getProperty("rabbitmq_routingKey").trim();
>>>
>>>             QueueingConsumer consumer = new QueueingConsumer(channel);
>>>
>>>             channel.basicConsume(queueName, true, consumer);
>>>            QueueingConsumer.Delivery delivery;
>>>             while (!runInfinite)
>>>             {
>>>                       try {
>>>
>>>                      delivery = consumer.nextDelivery();
>>>
>>>                          } catch (InterruptedException ie) {
>>>                     log.debug("Problem in Deliverying the message");
>>>                     continue;
>>>                 }
>>>
>>>                   /* 1 . Do some operation on the consumed data which
>>> will take 2 -3 sec of time.
>>>                       2. if above operation is success than continue to
>>> receive next data
>>>                       3. if above operation fails than resend the message
>>> back to the queue( Consumer acting as Producer) using channel.basicPublish
>>>
>>>                   */
>>>                      channel.basicPublish(exchangeName, routingKey,
>>> true, false, MessageProperties.PERSISTENT_TEXT_PLAIN, bao.toByteArray());
>>>
>>> }
>>>
>>> }
>>>         }
>>>
>>> Memory leakage is because of operating on the data which take times. In
>>> that time the QueueingDelivery objects are created as many as the the
>>> messages sent by the Producer.
>>>
>>> Thanks,
>>> Seema
>>>   On Tue, Feb 2, 2010 at 5:01 PM, Gustavo Aquino <
>>> aquino.gustavo at gmail.com> wrote:
>>>
>>>> Seema,
>>>>
>>>> You can't control when your GC will run, if you declare this objects
>>>> into loop and don't are using hash tables or maps to reutilize it, so it
>>>> will be eligible to GC automatically, because your life cycle over in next
>>>> loop.
>>>>
>>>> Today I have a consumer in Java consuming about 1.000 m/s and never used
>>>> more than 15Mb and this objects never go throw eden memory, if you don't
>>>> forward this objects created inside consumer loop, this objects will be
>>>> automatic eligible for GC but GC don't run immediately you need to wait.
>>>>
>>>> Can you post your consumer code ?
>>>>
>>>>
>>>>
>>>>   On Tue, Feb 2, 2010 at 3:48 AM, Seema Biradar <
>>>> seemasbiradar at gmail.com> wrote:
>>>>
>>>>>   Hi All,
>>>>>
>>>>> i tried the rabbit MQ with Producer and Consumer Program.
>>>>>
>>>>> in Consumer side i am getting the all the Message, but the following
>>>>> objects
>>>>>
>>>>> com.rabbitmq.client.QueueingConsumer.Delivery
>>>>> com.rabbitmq.client.Envelop
>>>>> com.rabbitmq.client.AMQP.BasicProperties
>>>>>
>>>>> are created for each message and not garbage collected immediately
>>>>> after consuming.
>>>>>
>>>>> *in Producer.java
>>>>> *
>>>>> used basic publish as shown below.
>>>>>
>>>>> channel.basicPublish(exchangeName, routingKey, true, false,
>>>>> MessageProperties.PERSISTENT_TEXT_PLAIN, bao.toByteArray()) ;
>>>>>
>>>>>
>>>>> *Consumer.java
>>>>> *
>>>>> channel.basicConsume(queueName, true, consumer);
>>>>>
>>>>> with no-acknowledgement *true*.
>>>>>
>>>>> Please let me know, how to make the above object eligible for garbage
>>>>> collection immediately after consuming.
>>>>> for Consumer program The Heap Memory allocated is more , as this is
>>>>> performace issue
>>>>>
>>>>>
>>>>> Thanks,
>>>>> Seema
>>>>>
>>>>> _______________________________________________
>>>>> rabbitmq-discuss mailing list
>>>>> rabbitmq-discuss at lists.rabbitmq.com
>>>>> http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>>>>
>>>>>
>>>>
>>>
>>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20100203/5520aad4/attachment.htm 