[rabbitmq-discuss] Persister crashes Rabbit
ilya at aiderss.com
Sat Nov 15 16:48:07 GMT 2008
> What client are you using to publish the messages?
I'm using Aman's ruby library , and a simple publisher client. 
Rabbit seems to become unstable once the persister.LOG crosses the 1GB
> limit. Disk syncs are taking longer and longer (for obvious reasons), and
> memory starts to fluctuate all over the map - from 2GB to 7GB (I am seeing
> high memory watermark warnings). Then, without any warnings in the log file,
> the process falls over and everyone gets disconnected.
> Persistent messages still have copies in memory, so if you have no
> consumers or the rate of consumption is lower than the publishing rate
> eventually rabbit will run out of memory.
So the messages don't get flushed out to disk at all? Right now, I'm running
two publishers and no consumers.. I would expect RabbitMQ to keep a stack of
messages in memory and overflow to disk - is this wrong? Hmm, now that I
think about it, this would explain the log rollover pattern I'm seeing:
.previous is removed from disk, and full log file is dumped from memory. I
was surprised by this pattern, but if it's always doing a full memory flush,
that would make sense.
- That would also mean that the number of messages we can queue is
essentially limited by the amount of available RAM?
- Is there any way to do an incremental append to the logfile, instead of a
- I've seen a few mentions of RDBMS hooks for persistence, how far away are
we from this?
P.S. Running latest version of RabbitMQ from trunk (as of Nov 14th),
> The flow control in that version should tell the producers to stop sending
> messages when memory becomes critical. However, it does require a
> cooperating client. Hence my question above about what client you are using.
> At present the trunk version of the Java client is the only one that handles
> flow control correctly.
Ah, interesting, I'll ping Aman on that.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the rabbitmq-discuss