<div>Here are two runs, one with 0.4 and the other, below, with 0.3. Both crash. The rabbit.log for the 0.3 test is attached. The 0.4 test did not log any memory watermark messages, only the 0.3 test.</div><div>The page file on this machine is 8GB with a maximum size of 16GB.</div>
<div>_________________________________________________</div><div>The client:</div><div>here we go at 19:16:49.49 2010-01-26</div><div>starting consumer #0</div><div>starting producer #0</div><div>sending rate: 11280 msg/s</div>
<div>sending rate: 10228 msg/s</div><div>sending rate: 10017 msg/s</div><div>sending rate: 10069 msg/s</div><div>sending rate: 9613 msg/s</div><div>...</div><div>...Crash</div><div>...</div><div><font class="Apple-style-span" face="arial, helvetica, sans-serif">The server with 0.4 memory:</font></div>
<div><font class="Apple-style-span" face="'courier new', monospace">eheap_alloc: Cannot allocate 298930300 bytes of memory (of type "heap").</font></div><div><div><font class="Apple-style-span" face="'courier new', monospace">=INFO REPORT==== 26-Jan-2010::19:16:46 ===</font></div>
<div><font class="Apple-style-span" face="'courier new', monospace">Memory limit set to 1634MB.</font></div><div><font class="Apple-style-span" face="'courier new', monospace"><br></font></div><div><font class="Apple-style-span" face="'courier new', monospace">=INFO REPORT==== 26-Jan-2010::19:16:46 ===</font></div>
<div><font class="Apple-style-span" face="'courier new', monospace">disk_log: repairing "c:/AMQP/RabbitMQ/rabbitmq_server-1.7.1/db/BIGPOOH-mnesia/rabbit_persister.LOG" ...</font></div><div><font class="Apple-style-span" face="'courier new', monospace"><br>
</font></div><div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">=INFO REPORT==== 26-Jan-2010::19:16:46 ===</span></font></div><div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">Repaired persister log - 1 recovered, 0 bad</span></font></div>
<div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;"><br></span></font></div><div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">=INFO REPORT==== 26-Jan-2010::19:16:46 ===</span></font></div>
<div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">Rolling persister log to "c:/AMQP/RabbitMQ/rabbitmq_server-1.7.1/db/BIGPOOH-mnesia/rabbit_persister.LOG.previous"</span></font></div>
<div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;"><br></span></font></div><div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">=INFO REPORT==== 26-Jan-2010::19:16:46 ===</span></font></div>
<div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">started TCP Listener on <a href="http://0.0.0.0:5672">0.0.0.0:5672</a></span></font></div>
<div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;"><br></span></font></div><div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">=INFO REPORT==== 26-Jan-2010::19:16:49 ===</span></font></div>
<div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">accepted TCP connection on <a href="http://0.0.0.0:5672">0.0.0.0:5672</a> from <a href="http://127.0.0.1:42109">127.0.0.1:42109</a></span></font></div>
<div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;"><br></span></font></div><div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">=INFO REPORT==== 26-Jan-2010::19:16:49 ===</span></font></div>
<div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">starting TCP connection <0.143.0> from <a href="http://127.0.0.1:42109">127.0.0.1:42109</a></span></font></div>
<div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;"><br></span></font></div><div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">=INFO REPORT==== 26-Jan-2010::19:16:49 ===</span></font></div>
<div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">accepted TCP connection on <a href="http://0.0.0.0:5672">0.0.0.0:5672</a> from <a href="http://127.0.0.1:42110">127.0.0.1:42110</a></span></font></div>
<div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;"><br></span></font></div><div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">=INFO REPORT==== 26-Jan-2010::19:16:49 ===</span></font></div>
<div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">starting TCP connection <0.149.0> from <a href="http://127.0.0.1:42110">127.0.0.1:42110</a></span></font></div>
<div><span class="Apple-style-span" style="font-size: x-small;">______________________________________________________________</span></div></div><div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">This is with 0.3 memory:</span></font></div>
<div><div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">=INFO REPORT==== 26-Jan-2010::19:22:09 ===</span></font></div><div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">Memory limit set to 1225MB.</span></font></div>
<div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;"><br></span></font></div><div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">=INFO REPORT==== 26-Jan-2010::19:22:09 ===</span></font></div>
<div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">disk_log: repairing "c:/AMQP/RabbitMQ/rabbitmq_server-1.7.1/db/BIGPOOH-mnesia/rabbit_persister.LOG" ...</span></font></div>
<div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;"><br></span></font></div><div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">=INFO REPORT==== 26-Jan-2010::19:22:09 ===</span></font></div>
<div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">Repaired persister log - 1 recovered, 0 bad</span></font></div><div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;"><br>
</span></font></div><div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">=INFO REPORT==== 26-Jan-2010::19:22:09 ===</span></font></div><div>
<font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">Rolling persister log to "c:/AMQP/RabbitMQ/rabbitmq_server-1.7.1/db/BIGPOOH-mnesia/rabbit_persister.LOG.previous"</span></font></div>
<div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;"><br></span></font></div><div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">=INFO REPORT==== 26-Jan-2010::19:22:10 ===</span></font></div>
<div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">started TCP Listener on <a href="http://0.0.0.0:5672">0.0.0.0:5672</a></span></font></div>
<div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;"><br></span></font></div><div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">=INFO REPORT==== 26-Jan-2010::19:22:13 ===</span></font></div>
<div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">accepted TCP connection on <a href="http://0.0.0.0:5672">0.0.0.0:5672</a> from <a href="http://127.0.0.1:42320">127.0.0.1:42320</a></span></font></div>
<div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;"><br></span></font></div><div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">=INFO REPORT==== 26-Jan-2010::19:22:13 ===</span></font></div>
<div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">starting TCP connection <0.143.0> from <a href="http://127.0.0.1:42320">127.0.0.1:42320</a></span></font></div>
<div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;"><br></span></font></div><div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">=INFO REPORT==== 26-Jan-2010::19:22:13 ===</span></font></div>
<div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">accepted TCP connection on <a href="http://0.0.0.0:5672">0.0.0.0:5672</a> from <a href="http://127.0.0.1:42321">127.0.0.1:42321</a></span></font></div>
<div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;"><br></span></font></div><div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">=INFO REPORT==== 26-Jan-2010::19:22:13 ===</span></font></div>
<div><font class="Apple-style-span" face="'courier new', monospace"><span class="Apple-style-span" style="font-size: x-small;">starting TCP connection <0.149.0> from <a href="http://127.0.0.1:42321">127.0.0.1:42321</a></span></font></div>
<div><font class="Apple-style-span" face="'courier new', monospace"><br></font></div><div><font class="Apple-style-span" face="'courier new', monospace"><div>___________________________________________________</div>
<div>here we go at 19:25:50.62 2010-01-26</div><div>starting consumer #0</div><div>starting producer #0</div><div>sending rate: 11216 msg/s</div><div>sending rate: 11214 msg/s</div><div>sending rate: 10332 msg/s</div><div>
sending rate: 9481 msg/s</div><div>sending rate: 7449 msg/s</div><div>sending rate: 6095 msg/s</div><div>sending rate: 3217 msg/s</div><div>sending rate: 998 msg/s</div><div>...</div><div><span class="Apple-style-span" style="font-family: arial; ">...crash</span></div>
<div><span class="Apple-style-span" style="font-family: arial; ">...</span></div><div><span class="Apple-style-span" style="font-family: arial; ">temp_alloc: Cannot allocate 197209248 bytes of memory (of type "tmp_heap").</span></div>
<div><span class="Apple-style-span" style="font-family: arial; "><br></span></div><div><span class="Apple-style-span" style="font-family: arial; ">Log file is attached.</span></div><div><span class="Apple-style-span" style="font-family: arial; "><br>
</span></div><div><span class="Apple-style-span" style="font-family: arial; ">________________________</span></div></font></div></div><i>>>>That's exactly what is supposed to happen. Rabbit has told the producer to stop sending messages since it is under memory pressure. Once memory becomes available again, e.g. when a consumer consumes enough messages, or some queues get deleted/purged, or as a result of internal garbage collection, Rabbit will tell the producer that it may resume.<br>
</i><div class="gmail_quote"><br></div><div class="gmail_quote">The funny thing is that rabbitmqctl status returned a 'node down' message. I suppose that is OK?</div><div class="gmail_quote">_______________</div><div class="gmail_quote">
<br></div><div class="gmail_quote">Cheers, John</div><div class="gmail_quote"><br></div>