[rabbitmq-discuss] RabbitMQ Client 2.7.1 Memory Leak?
Vijay Lakshmi
vijjivinay at gmail.com
Sat Jul 14 22:04:38 BST 2012
Thanks Jerry!
In which version it is fixed, can you please let me know.
Also, how do we get permission to see the bug details.
Regards,
Vinay
On Sun, Jul 15, 2012 at 2:02 AM, Jerry Kuch <jerryk at rbcon.com> wrote:
> Hi, Vijay:
>
> Initial segment of the discussion from the bug pasted here:
>
> Steve Powell <steve at rabbitmq.com> 2012-02-01 10:41:32 GMT
>
> See link, main part here:
>
> public static void main(String[] argv) throws IOException {
> ConnectionFactory f = new ConnectionFactory();
> Connection c = f.newConnection();
> for(int i = 0; i < 20000; ++i) {
> Channel ch = c.createChannel();
> ch.close();
> }
> System.gc();
> while(true) {}
> }
>
> While program is running, observe the memory usage, it keeps on increasing
>
> Get process id and do following
> jmap -dump:format=b,file=yo.bin <pid>
> jhat jhat -J-mx768m -stack false yo.bin
> Once jhat brings up http server, go herehttp://localhost:7000/showInstanceCounts/
>
> 20001 instances of class com.rabbitmq.client.impl.AMQCommand
> 20001 instances of class com.rabbitmq.client.impl.CommandAssembler
> 20000 instances of class com.rabbitmq.client.ShutdownSignalException
> 20000 instances of class com.rabbitmq.client.impl.AMQImpl$Channel$Close
> 20000 instances of class com.rabbitmq.client.impl.ChannelN
> 20000 instances of class com.rabbitmq.client.impl.ConsumerDispatcher
>
> [reply <https://bugzilla.rabbitmq.com/show_bug.cgi?id=24723#add_comment>]
> [-] <https://bugzilla.rabbitmq.com/show_bug.cgi?id=24723#>Comment 1<https://bugzilla.rabbitmq.com/show_bug.cgi?id=24723#c1>Steve
> Powell <steve at rabbitmq.com> 2012-02-01 11:09:17 GMT
>
> (In reply to comment #0 <https://bugzilla.rabbitmq.com/show_bug.cgi?id=24723#c0>)
>
> Ran sample test main under debug, and found the WorkPool in the
> ConsumerWorkService contains all the AMQChannel objects!
>
> [reply <https://bugzilla.rabbitmq.com/show_bug.cgi?id=24723#add_comment>]
> [-] <https://bugzilla.rabbitmq.com/show_bug.cgi?id=24723#>Comment 2<https://bugzilla.rabbitmq.com/show_bug.cgi?id=24723#c2>Steve
> Powell <steve at rabbitmq.com> 2012-02-01 11:58:46 GMT
>
> Problem is we never issue stopWork(channel) on the ConsumerWorkService. This
> means that the stuff in the WorkPool stays there.
>
> Difficulty is that issuing this too early may mean that the last pieces of work
> to be executed for a channel may be chopped. We may need to have a thread that
> runs after channel.close() to tidy up when the last piece of work for that
> channel is finished. This is only necessary if there is work in the pool for
> that channel, otherwise we may simply stop work immediately.
>
> Solution -- check WorkPool for work -- if there is none, there won't be any
> more later. stopWork() and that's it.
>
> If there is some work to be done, then schedule some more work which
> automatically removes the channel from the work pool.
>
> [reply <https://bugzilla.rabbitmq.com/show_bug.cgi?id=24723#add_comment>]
> [-] <https://bugzilla.rabbitmq.com/show_bug.cgi?id=24723#>Comment 3<https://bugzilla.rabbitmq.com/show_bug.cgi?id=24723#c3>Steve
> Powell <steve at rabbitmq.com> 2012-02-01 14:59:50 GMT
>
> (In reply to comment #2 <https://bugzilla.rabbitmq.com/show_bug.cgi?id=24723#c2>)> Solution -- check WorkPool for work -- if there is none, there won't be any
> > more later. stopWork() and that's it.
> > If there is some work to be done, then schedule some more work which
> > automatically removes the channel from the work pool.
>
> Actually, no, better is to hook into broadcastShutdownSignal() which calls
> handleShutdownSignal even if there are no consumers. handleShutdownSignal()
> can remove the work from the queue as it's last act.
>
> [reply <https://bugzilla.rabbitmq.com/show_bug.cgi?id=24723#add_comment>]
> [-] <https://bugzilla.rabbitmq.com/show_bug.cgi?id=24723#>Comment 4<https://bugzilla.rabbitmq.com/show_bug.cgi?id=24723#c4>Steve
> Powell <steve at rabbitmq.com> 2012-02-01 15:15:41 GMT
>
> Changeset http://hg.rabbitmq.com/rabbitmq-java-client/rev/0d949ba17115
>
> Stop channel consumer work pool after channel.close()
>
> [reply <https://bugzilla.rabbitmq.com/show_bug.cgi?id=24723#add_comment>]
> [-] <https://bugzilla.rabbitmq.com/show_bug.cgi?id=24723#>Comment 5<https://bugzilla.rabbitmq.com/show_bug.cgi?id=24723#c5>Steve
> Powell <steve at rabbitmq.com> 2012-02-01 15:48:04 GMT
>
> Repeating tests for java client -- all appears well;
> Reproducing debug run -- all appears cleared up;
> Running jmap and jhat as shown -- we get lots of unresolved instance references
> when running jhat (?), and shows:
>
> 10 instances of class
> com.rabbitmq.client.impl.LongStringHelper$ByteArrayLongString
> 4 instances of class com.rabbitmq.client.impl.CommandAssembler$CAState
> 2 instances of class com.rabbitmq.client.DefaultSaslConfig
>
> at top of instance counts list.
>
> It looks fixed.
>
>
>
> On Sat, Jul 14, 2012 at 1:00 PM, Vijay Lakshmi <vijjivinay at gmail.com>wrote:
>
>> Hi Jerry,
>>
>> Thanks for the response.
>> I can not browse the bug 24723 (it says access denied), can you please
>> provide the details.
>>
>> Regards,
>> Vinay
>>
>> On Sat, Jul 14, 2012 at 9:56 PM, Jerry Kuch <jerryk at rbcon.com> wrote:
>>
>>> Hi, Vijay:
>>>
>>> You may want to upgrade from 2.7.1. The 2.7.1 Java client had a known
>>> and now fixed memory leak that might account for what you're seeing:
>>>
>>> https://bugzilla.rabbitmq.com/show_bug.cgi?id=24723
>>>
>>> Best regards,
>>> Jerry
>>>
>>> On Sat, Jul 14, 2012 at 7:35 AM, Vijay Lakshmi <vijjivinay at gmail.com>wrote:
>>>
>>>> Hi,
>>>>
>>>> We are using RabbitMQ 2.7.1 Java Client and Server 2.7.1.
>>>> While we were running load tests using our clients we observed that our
>>>> memory has reached to 2GB.
>>>>
>>>> Below is the snapshot of heap analysis.
>>>>
>>>> We are closing the channels properly using channel.close().
>>>> Could someone suggest what could be the problem for this memory leak.
>>>> Do the later versions of RabbitMQ solve this problem?
>>>>
>>>>
>>>> Instance Counts for All Classes (excluding platform)
>>>>
>>>> 311681 instances<http://redir.aspx?C=dba9a335a30e4feb822ceaa30de7f654&URL=http%3a%2f%2flocalhost%3a7000%2finstances%2f0x77c6f06a0>
>>>> of class com.rabbitmq.client.impl.AMQCommand<http://redir.aspx?C=dba9a335a30e4feb822ceaa30de7f654&URL=http%3a%2f%2flocalhost%3a7000%2fclass%2f0x77c6f06a0>
>>>>
>>>> 311681 instances<http://redir.aspx?C=dba9a335a30e4feb822ceaa30de7f654&URL=http%3a%2f%2flocalhost%3a7000%2finstances%2f0x77c6f3228>
>>>> of class com.rabbitmq.client.impl.CommandAssembler<http://redir.aspx?C=dba9a335a30e4feb822ceaa30de7f654&URL=http%3a%2f%2flocalhost%3a7000%2fclass%2f0x77c6f3228>
>>>>
>>>> 295730 instances<http://redir.aspx?C=dba9a335a30e4feb822ceaa30de7f654&URL=http%3a%2f%2flocalhost%3a7000%2finstances%2f0x77d438380>
>>>> of class com.rabbitmq.client.impl.ChannelN<http://redir.aspx?C=dba9a335a30e4feb822ceaa30de7f654&URL=http%3a%2f%2flocalhost%3a7000%2fclass%2f0x77d438380>
>>>>
>>>> 295730 instances<http://redir.aspx?C=dba9a335a30e4feb822ceaa30de7f654&URL=http%3a%2f%2flocalhost%3a7000%2finstances%2f0x77d43ff30>
>>>> of class com.rabbitmq.client.impl.ConsumerDispatcher<http://redir.aspx?C=dba9a335a30e4feb822ceaa30de7f654&URL=http%3a%2f%2flocalhost%3a7000%2fclass%2f0x77d43ff30>
>>>>
>>>> 295715 instances<http://redir.aspx?C=dba9a335a30e4feb822ceaa30de7f654&URL=http%3a%2f%2flocalhost%3a7000%2finstances%2f0x77c60b280>
>>>> of class com.rabbitmq.client.ShutdownSignalException<http://redir.aspx?C=dba9a335a30e4feb822ceaa30de7f654&URL=http%3a%2f%2flocalhost%3a7000%2fclass%2f0x77c60b280>
>>>>
>>>> 295703 instances<http://redir.aspx?C=dba9a335a30e4feb822ceaa30de7f654&URL=http%3a%2f%2flocalhost%3a7000%2finstances%2f0x77c75a540>
>>>> of class com.rabbitmq.client.impl.AMQImpl$Channel$Close<http://redir.aspx?C=dba9a335a30e4feb822ceaa30de7f654&URL=http%3a%2f%2flocalhost%3a7000%2fclass%2f0x77c75a540>
>>>>
>>>> 271315 instances<http://redir.aspx?C=dba9a335a30e4feb822ceaa30de7f654&URL=http%3a%2f%2flocalhost%3a7000%2finstances%2f0x77d4d5590>
>>>> of class com.rabbitmq.client.QueueingConsumer<http://redir.aspx?C=dba9a335a30e4feb822ceaa30de7f654&URL=http%3a%2f%2flocalhost%3a7000%2fclass%2f0x77d4d5590>
>>>>
>>>> 60523 instances<http://redir.aspx?C=dba9a335a30e4feb822ceaa30de7f654&URL=http%3a%2f%2flocalhost%3a7000%2finstances%2f0x77cea18a0>
>>>> of class org.json.simple.parser.Yytoken<http://redir.aspx?C=dba9a335a30e4feb822ceaa30de7f654&URL=http%3a%2f%2flocalhost%3a7000%2fclass%2f0x77cea18a0>
>>>>
>>>> 31600 instances<http://redir.aspx?C=dba9a335a30e4feb822ceaa30de7f654&URL=http%3a%2f%2flocalhost%3a7000%2finstances%2f0x77c73fa58>
>>>> of class com.rabbitmq.client.AMQP$BasicProperties<http://redir.aspx?C=dba9a335a30e4feb822ceaa30de7f654&URL=http%3a%2f%2flocalhost%3a7000%2fclass%2f0x77c73fa58>
>>>>
>>>> 31210 instances<http://redir.aspx?C=dba9a335a30e4feb822ceaa30de7f654&URL=http%3a%2f%2flocalhost%3a7000%2finstances%2f0x77d4d7358>
>>>> of class com.rabbitmq.client.QueueingConsumer$Delivery<http://redir.aspx?C=dba9a335a30e4feb822ceaa30de7f654&URL=http%3a%2f%2flocalhost%3a7000%2fclass%2f0x77d4d7358>
>>>>
>>>> 31209 instances<http://redir.aspx?C=dba9a335a30e4feb822ceaa30de7f654&URL=http%3a%2f%2flocalhost%3a7000%2finstances%2f0x77d4dc1d0>
>>>> of class com.rabbitmq.client.Envelope<http://redir.aspx?C=dba9a335a30e4feb822ceaa30de7f654&URL=http%3a%2f%2flocalhost%3a7000%2fclass%2f0x77d4dc1d0>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> rabbitmq-discuss mailing list
>>>> rabbitmq-discuss at lists.rabbitmq.com
>>>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>>>
>>>>
>>>
>>> _______________________________________________
>>> rabbitmq-discuss mailing list
>>> rabbitmq-discuss at lists.rabbitmq.com
>>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>>
>>>
>>
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.rabbitmq.com
>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>
>>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120715/dc8122ab/attachment.htm>
More information about the rabbitmq-discuss
mailing list