[rabbitmq-discuss] Problem with RabbitMQ Java Client (Memory leak)

[Steve Powell]

Yogesh,
Sorry not to get back to you sooner.

I'm not at all sure what is causing your leak this time.

Are you using QueueingConsumer? How are your messages being consumed?

From the list of objects I am particularly interested in the ConsumerDispatcher
objects.

These ought to be created one per Channel, and can only be created when calling
the createChannel() method on the Connection interface. So please can you check
that you are not creating lots of channels.

If it is possible, can you construct a small program to demonstrate the leak,
like last time?

Steve Powell  (a happy bunny)
----------some more definitions from the SPD----------
vermin (v.) Treating the dachshund for roundworm.
chinchilla (n.) Cooling device for the lower jaw.
socialcast (n.) Someone to whom everyone is speaking but nobody likes.

On 2 Feb 2012, at 17:32, Yogesh Ketkar wrote:

> Hi Steve,
> 
> Now that I am reusing the channels, memory problem with channels is
> not an issue, but there seems to be some more problem with memory
> leak.
> I had a program which pumped 100000 messages on a queue.
> 
> Then this program kicked in and created certain queues on the fly and
> distributed these messages across them and
> eventually all the messages got consumed.
> 
> Now when all queues in the system are empty, here is the object dump
> and I again see a lot of memory used by
> these objects.
> 
> 96449 instances of class com.rabbitmq.client.AMQP$BasicProperties
> 96428 instances of class com.rabbitmq.client.Envelope
> 96399 instances of class com.rabbitmq.client.impl.ConsumerDispatcher
> $4
> 
> I created BasicProperties object like this, while routing the messages
> to appropriate queue.
>    BasicProperties bp = new BasicProperties();
>    bp.setTimestamp(Calendar.getInstance().getTime());
>    bp.setType(MessageProperties.PERSISTENT_TEXT_PLAIN.toString());
> This was then used in basicPublish method.
> 
> regards, Yogesh
> 
> On Feb 1, 10:37 pm, Steve Powell <st... at rabbitmq.com> wrote:
>> The fix (which we have coded) will be in the next release,
>> but I don't know precisely when that will be.
>> 
>> Steve Powell  (a happy bunny)
>> ----------some more definitions from the SPD----------
>> vermin (v.) Treating the dachshund for roundworm.
>> chinchilla (n.) Cooling device for the lower jaw.
>> socialcast (n.) Someone to whom everyone is speaking but nobody likes.
>> 
>> On 1 Feb 2012, at 11:21, Yogesh Ketkar wrote:
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>>> Thanks Steve.
>>> Can you give me some idea about date of next Java Client release (with
>>> this bug-fix)?
>> 
>>> regards, Yogesh
>> 
>>> On Feb 1, 3:42 pm, Steve Powell <st... at rabbitmq.com> wrote:
>>>> Yogesh,
>> 
>>>> Thank you for this problem report. First point: each connection defines a
>>>> (hidden) channel zero, so that explains why there are 20001 of some objects and
>>>> 20000 of others.
>> 
>>>> As to why these are not garbage collected, we must be holding a reference
>>>> somewhere (probably in the Connection).
>> 
>>>> We will investigate. I have raised a bug (24723).
>> 
>>>> Steve Powell  (a happy bunny)
>>>> ----------some more definitions from the SPD----------
>>>> vermin (v.) Treating the dachshund for roundworm.
>>>> chinchilla (n.) Cooling device for the lower jaw.
>>>> socialcast (n.) Someone to whom everyone is speaking but nobody likes.
>> 
>>>> On 31 Jan 2012, at 17:49, Yogesh Ketkar wrote:
>> 
>>>>> I am using rabbitmq-java-client-bin-2.7.1/rabbitmq-client.jar with
>>>>> RabbitMQ 2.7.1 Server.
>> 
>>>>> Just run this code
>> 
>>>>> public static void main(String[] argv) throws IOException {
>>>>>    ConnectionFactory f = new ConnectionFactory();
>>>>>    Connection c = f.newConnection();
>>>>>    for(int i = 0; i < 20000; ++i) {
>>>>>        Channel ch = c.createChannel();
>>>>>        ch.close();
>>>>>    }
>>>>>    System.gc();
>>>>>    while(true) {}
>>>>> }
>> 
>>>>> While program is running, observe the memory usage, it keeps on
>>>>> increasing
>>>>> Get process id and do following
>>>>> jmap -dump:format=b,file=yo.bin  <pid>
>>>>> jhat jhat -J-mx768m -stack false yo.bin
>>>>> Once jhat brings up http server, go here
>>>>> http://localhost:7000/showInstanceCounts/
>> 
>>>>> 20001 instances of class com.rabbitmq.client.impl.AMQCommand
>>>>> 20001 instances of class com.rabbitmq.client.impl.CommandAssembler
>>>>> 20000 instances of class com.rabbitmq.client.ShutdownSignalException
>>>>> 20000 instances of class com.rabbitmq.client.impl.AMQImpl$Channel
>>>>> $Close
>>>>> 20000 instances of class com.rabbitmq.client.impl.ChannelN
>>>>> 20000 instances of class com.rabbitmq.client.impl.ConsumerDispatcher
>> 
>>>>> In-spite of closing all the channels, not only channels but associated
>>>>> objects (ShutdownSignalException, ChannelClose) continue to consume
>>>>> memory.
>>>>> For sure, knowing this now, I will use channelCreate call sparingly.
>> 
>>>>> Is there a way to remedy this situation?
>>>>> What is also worrying is instances of AMQCommand, so would this be the
>>>>> case for each and every call of client library?
>> 
>>>>> regards, Yogesh
>> 
>>>>> _______________________________________________
>>>>> rabbitmq-discuss mailing list
>>>>> rabbitmq-disc... at lists.rabbitmq.com
>>>>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>> 
>>>> _______________________________________________
>>>> rabbitmq-discuss mailing list
>>>> rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>> _______________________________________________
>>> rabbitmq-discuss mailing list
>>> rabbitmq-disc... at lists.rabbitmq.com
>>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>> 
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss