[rabbitmq-discuss] MulticastMain Java client causes Erlang error eheap_alloc: Cannot allocate 467078560 bytes of memory (of type "heap") (with RabbitMQ 1.7.1)

[John Apps]

Matthias,

This is rabbit.log after starting the server:
=INFO REPORT==== 26-Jan-2010::16:09:34 ===
Memory limit set to 1634MB.
=INFO REPORT==== 26-Jan-2010::16:09:34 ===
Rolling persister log to
"c:/AMQP/RabbitMQ/rabbitmq_server-1.7.0/db/rabbit-mnesi
a/rabbit_persister.LOG.previous"
=INFO REPORT==== 26-Jan-2010::16:09:35 ===
started TCP Listener on 0.0.0.0:5672
__________________
These are the parameters, which I hope match what you mean:
java -server ^
com.rabbitmq.examples.MulticastMain ^
-hlocalhost -p5672 -tdirect -eex1 -i10 -m1024 ^
-n1024 -q20 -s1000 -x1 -y1

here we go at 16:19:47.60 2010-01-26
starting consumer #0
starting producer #0
sending rate: 10376 msg/s
sending rate: 10506 msg/s
sending rate: 10141 msg/s
sending rate: 9718 msg/s
sending rate: 8954 msg/s
Exception in thread "Thread-3" java.lang.RuntimeException:
java.io.IOException
        at
com.rabbitmq.examples.MulticastMain$Producer.run(MulticastMain.java:255)
        at java.lang.Thread.run(Unknown Source)
Caused by: java.io.IOException
        at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:121)
        at
com.rabbitmq.client.impl.AMQChannel.exnWrappingRpc(AMQChannel.java:139)
        at com.rabbitmq.client.impl.ChannelN.txCommit(ChannelN.java:689)
        at com.rabbitmq.client.impl.ChannelN.txCommit(ChannelN.java:70)
        at
com.rabbitmq.examples.MulticastMain$Producer.run(MulticastMain.java:249)
all done at 16:21:17.15 2010-01-26
...
...

The good news is that it crashes very quickly now, so I do not need to wait
long;)

Which is less than before. The dump is not that big, so I have attached a
ZIPped version:

687,358 erl_crash.dump.

First few lines of the dump (the amount of memory being allocated is a good
bit less than in the last crashes):

Slogan: eheap_alloc: Cannot allocate 298930300 bytes of memory (of type
"heap").
System version: Erlang R13B03 (erts-5.7.4) [smp:2:2] [rq:2]
[async-threads:30]
Compiled: Tue Nov 24 11:12:28 2009

Cheers, John
_____________________________________________
On Mon, Jan 25, 2010 at 20:18, Matthias Radestock <matthias at lshift.net>wrote:

> John,
>
>
> John Apps wrote:
>
>> I have installed 1.7.1 in a new directory, recompiled the Java client and
>> tried this test again.
>>
>> This is the contents of rabbitmq.config
>> [
>>  {rabbit, [{vm_memory_high_watermark, 3}]},
>>  {rabbit, [{memory_alarms, true}]},
>>  {mnesia, [{dump_log_write_threshold, 1000}]},
>>  {rabbit, []}
>> ].
>> I have tried various options with the above vm_memory_high_watermark,
>> e.g., .4, .6 and so forth, the 3 above being from the last try. Perhaps
>> there more options that I missed? It would not surprise me.
>>
>
> "3" would mean 300% of system memory, which is probably not what you want.
> I'd stick with the default of "0.4" or lower. I also recommend checking what
> limit is being set by looking in the rabbit.log file for the line "Memory
> limit is set to ...", and to watch out for any memory related log messages
> in general while running your tests.
>
>
>  Running the MulticastMain (Java) test against RabbitMQ with the parameters
>> shown, leads to an Erlang crash after some time.
>> The data below is from a Windows 7 X64 machine running JDK 6 X64.
>>
>> "Cannot allocate 467078560 bytes" is the error message.
>>
>> C:\AMQP\RabbitMQ\rabbitmq-java-client-1.7.1\test\src>java -server ^
>> com.rabbitmq.examples.MulticastMain ^
>> -hlocalhost -p5672 -tdirect -eex1 -i10 -m1024 ^
>> -n1024 -q20 -r1000 -s100 -x1 -y1
>>
>
> That actually exposes an area of uncertainty in the AMQP spec: At what
> point do transactional acks affect qos? At the time the ack is received by
> the server, or at the time the commit is received? RabbitMQ currently takes
> the latter view, which means with the above settings the consumer will not
> receive more than 20 messages. I have filed a bug to figure out whether
> that's the correct answer.
>
> Anyway, let that not distract us from figuring out why rabbit runs out of
> memory for you... I tried a slightly modified version of the above, removing
> the rate limit and increasing the size to 1000, in order to put the server
> under memory pressure sooner (since at a rate limit of 1000 and a message
> size of 100 it would have take several hours). That did cause the memory
> alarms to go off and the producer to be throttled. What do you see on your
> machine when running with these settings?
>
>
> Regards,
>
> Matthias.
>



-- 
---
John Apps
(49) 171 869 1813
Sent from Traunstein, Bavaria, Germany
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20100126/822228bf/attachment.htm 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: erl_crash.zip
Type: application/zip
Size: 116926 bytes
Desc: not available
Url : http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20100126/822228bf/attachment.zip