[rabbitmq-discuss] Consumer - threw an exception for channel AMQChannel

Mon Mar 14 16:53:56 GMT 2011

Hi Rishi,

I think that new String(delivery.getBody()) will generate a String occupying about two times as much storage as the byte array, so 16 messages is about 96MB.

It is extremely unlikely that gc would fail to recover storage that was unreachable, so you cannot be losing the references to the new String() s.

It is possible that you have a reference to the String s kept around in your code, or else, the Eclipse execution environment is keeping these references for tracing/debugging purposes, which wouldn't happen in a product environment.

The details you give look OK for Eclipse, though what you say implies that the problem of OoM only happens when 'running at full throttle'.  Is that why you can give me memory patterns for beyond 16 messages?

I've run a few tests of my own in Eclipse, and as far as I can see strings are garbage collected (each takes about 6.3MB for a 3MB array), though my simple tests aren't using rabbit.

Maybe we could help if we saw more of your consumer loop.  What do you do with those String s?

Steve Powell
On 13 Mar 2011, at 11:46, RishiDev wrote:

> I am running this test program from within eclipse. My eclipse.ini
> file has below properties.
> -startup
> plugins/org.eclipse.equinox.launcher_1.1.0.v20100507.jar
> --launcher.library
> plugins/org.eclipse.equinox.launcher.win32.win32.x86_1.1.0.v20100503
> -product
> org.eclipse.epp.package.java.product
> --launcher.defaultAction
> openFile
> --launcher.XXMaxPermSize
> 256M
> -showsplash
> org.eclipse.platform
> --launcher.XXMaxPermSize
> 256m
> --launcher.defaultAction
> openFile
> -vm
> C:/programs/java/jdk1.6.0_012/bin
> -vmargs
> -Dosgi.requiredJavaVersion=1.5
> -Xms512m
> -Xmx1024m
> 
> I believe problem is coming because in while loop I am converting the
> byte[] into string using new String(byte[]).
> Below line of code is the cause:
> msgContent=new String(delivery.getBody());
> 
> GC does tries to free up memory, still I think it doesn't take care of
> string objects sitting in heap as soon as they are not needed. While
> loop is running @ full throttle.
> Here is memory (Runtime.getRuntime().freeMemory()) pattern on receipt
> of each msg:
> Intial Memory 4831168
> 0 : Memory = 4300664
> 1 : Memory = 12289864
> 2 : Memory = 12289912
> 3 : Memory = 15001624
> 4 : Memory = 12289528
> 5 : Memory = 14513344
> 6 : Memory = 14513120
> 7 : Memory = 13986864
> 8 : Memory = 14382256
> 9 : Memory = 13986760
> 10 : Memory = 16178360
> 11 : Memory = 16178336
> 12 : Memory = 16440544
> 13 : Memory = 16309256
> 14 : Memory = 16569344
> 15 : Memory = 17710224
> 16 : Memory = 16571624
> 17 : Memory = 13634552
> 18 : Memory = 13958824
> 19 : Memory = 16146296
> 20 : Memory = 17551288
> 21 : Memory = 14966368
> 22 : Memory = 17026856
> 23 : Memory = 14966496
> 24 : Memory = 15320384
> 25 : Memory = 16183312
> 26 : Memory = 16359344
> 27 : Memory = 20692472
> 28 : Memory = 23010888
> 29 : Memory = 21177504
> 30 : Memory = 21222064
> 31 : Memory = 22916800
> 32 : Memory = 22916624
> 33 : Memory = 25201640
> 34 : Memory = 28177344
> 35 : Memory = 29973712
> 36 : Memory = 28923024
> 37 : Memory = 26630152
> 38 : Memory = 26850696
> 39 : Memory = 29737008
> 40 : Memory = 30523536
> 41 : Memory = 27552984
> 42 : Memory = 31575736
> 43 : Memory = 29255120
> 44 : Memory = 29977904
> 45 : Memory = 28688760
> 46 : Memory = 30220600
> 47 : Memory = 32055928
> 48 : Memory = 31662688
> 49 : Memory = 27281776
> 50 : Memory = 32715880
> 51 : Memory = 31796376
> 52 : Memory = 29213632
> 53 : Memory = 26370440
> 54 : Memory = 26370112
> 55 : Memory = 31009408
> 56 : Memory = 31929120
> 57 : Memory = 28557568
> 58 : Memory = 29215280
> 59 : Memory = 29215360
> 60 : Memory = 31402672
> 61 : Memory = 23395400
> 62 : Memory = 26461160
> 63 : Memory = 29963944
> 64 : Memory = 30881736
> 65 : Memory = 26107536
> 66 : Memory = 28036680
> 67 : Memory = 27117232
> 68 : Memory = 26371872
> 69 : Memory = 26371648
> 70 : Memory = 27903384
> 71 : Memory = 30095208
> 72 : Memory = 26369784
> 73 : Memory = 23526928
> 74 : Memory = 25320624
> 75 : Memory = 25583208
> 76 : Memory = 24274456
> 77 : Memory = 19631280
> 78 : Memory = 24536536
> 79 : Memory = 26986152
> 80 : Memory = 26594952
> 81 : Memory = 20682104
> 82 : Memory = 17836880
> 83 : Memory = 20682152
> 84 : Memory = 20682176
> 85 : Memory = 17836736
> 86 : Memory = 17836680
> 87 : Memory = 18060384
> 88 : Memory = 17706216
> 89 : Memory = 14336712
> 90 : Memory = 14467672
> 91 : Memory = 9302712
> 92 : Memory = 6459032
> 93 : Memory = 8777240
> 94 : Memory = 5633480
> 95 : Memory = 8517688
> 96 : Memory = 7337728
> 97 : Memory = 7337984
> 
> Please advice.
> 
> Thanks,
> Rishi
> 
> On Mar 11, 9:30 pm, Rob Harrop <r... at rabbitmq.com> wrote:
>> What heap size do you have configured for your JVM? It's is entirely
>> possible that if you're using the default sizes that 16 3MB msgs will
>> cause you to start hitting the OOM exception.
>> 
>> Regards,
>> 
>> Rob
>> 
>> On 11/03/11 16:02, RishiDev wrote:
>> 
>>> Hi,
>> 
>>> I have about 10K large msgs (each ~3 MB) in a queue. Started running a
>>> java consumer client on the queue and after receiving 16 msgs I got
>>> following exception. Please help resolve/understand the issue.
>> 
>>> Consumer com.rabbitmq.client.QueueingConsumer at 15b9e68 (amq.ctag-bEjkq
>>> +5EqqmXsp1idK9rrg==) method handleDelivery for channel
>>> AMQChannel(amqp://guest@localhost:5672/,1) threw an exception for
>>> channel AMQChannel(amqp://guest@localhost:5672/,1):
>>> java.lang.OutOfMemoryError
>>> com.rabbitmq.client.AlreadyClosedException: clean connection shutdown;
>>> reason: Attempt to use closed channel
>>>    at com.rabbitmq.client.impl.AMQChannel.ensureIsOpen(AMQChannel.java:
>>> 195)
>>>    at com.rabbitmq.client.impl.AMQChannel.transmit(AMQChannel.java:290)
>>>    at com.rabbitmq.client.impl.AMQChannel.transmit(AMQChannel.java:284)
>>>    at com.rabbitmq.client.impl.ChannelN.basicAck(ChannelN.java:669)
>>>    at
>>> com.octanner.jmspoc.rabbitmq.RabbitMQConsumer.receiveMsgs(RabbitMQConsumer.java:
>>> 69)
>>>    at
>>> com.octanner.jmspoc.rabbitmq.RabbitMQConsumer.main(RabbitMQConsumer.java:
>>> 77)
>>> Exception in thread "main"
>> 
>>> Thanks,
>>> Rishi
>>> _______________________________________________
>>> rabbitmq-discuss mailing list
>>> rabbitmq-disc... at lists.rabbitmq.com
>>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>> 
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss