<div class="gmail_quote"><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">What client are you using to publish the messages?<div class="Ih2E3d">
</div></blockquote><div><br>I'm using Aman's ruby library [1], and a simple publisher client. [2] <br><br>[1] <a href="http://github.com/tmm1/amqp">http://github.com/tmm1/amqp</a><br>[2] <a href="http://pastie.org/315607">http://pastie.org/315607</a> <br>
<br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div class="Ih2E3d">Rabbit seems to become unstable once the persister.LOG crosses the 1GB limit. Disk syncs are taking longer and longer (for obvious reasons), and memory starts to fluctuate all over the map - from 2GB to 7GB (I am seeing high memory watermark warnings). Then, without any warnings in the log file, the process falls over and everyone gets disconnected.<br>
<br></div>
Persistent messages still have copies in memory, so if you have no consumers or the rate of consumption is lower than the publishing rate eventually rabbit will run out of memory.<div class="Ih2E3d"></div></blockquote><div>
<br>So the messages don't get flushed out to disk at all? Right now, I'm running two publishers and no consumers.. I would expect RabbitMQ to keep a stack of messages in memory and overflow to disk - is this wrong? Hmm, now that I think about it, this would explain the log rollover pattern I'm seeing: .previous is removed from disk, and full log file is dumped from memory. I was surprised by this pattern, but if it's always doing a full memory flush, that would make sense. <br>
<br>- That would also mean that the number of messages we can queue is essentially limited by the amount of available RAM?<br>- Is there any way to do an incremental append to the logfile, instead of a full flush? <br>- I've seen a few mentions of RDBMS hooks for persistence, how far away are we from this?<br>
<br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div class="Ih2E3d">P.S. Running latest version of RabbitMQ from trunk (as of Nov 14th), <br>
<br></div>
The flow control in that version should tell the producers to stop sending messages when memory becomes critical. However, it does require a cooperating client. Hence my question above about what client you are using. At present the trunk version of the Java client is the only one that handles flow control correctly.<br>
</blockquote><div><br>Ah, interesting, I'll ping Aman on that. <br> </div></div>Cheers,<br>Ilya<br>