[rabbitmq-discuss] Persister crashes Rabbit

[Matthias Radestock]

Ilya,

Ilya Grigorik wrote:
> I'm doing some load testing on Rabbit and I'm bumping into the following 
> scenario:
> 
> Fanout exchange > Durable queue
> 
> I'm pushing ~300~350 message/s into Rabbit, each ~10kb/s.

What client are you using to publish the messages?

> Rabbit seems to become unstable once the persister.LOG crosses the 1GB 
> limit. Disk syncs are taking longer and longer (for obvious reasons), 
> and memory starts to fluctuate all over the map - from 2GB to 7GB (I am 
> seeing high memory watermark warnings). Then, without any warnings in 
> the log file, the process falls over and everyone gets disconnected.

Persistent messages still have copies in memory, so if you have no 
consumers or the rate of consumption is lower than the publishing rate 
eventually rabbit will run out of memory.

> P.S. Running latest version of RabbitMQ from trunk (as of Nov 14th), 

The flow control in that version should tell the producers to stop 
sending messages when memory becomes critical. However, it does require 
a cooperating client. Hence my question above about what client you are 
using. At present the trunk version of the Java client is the only one 
that handles flow control correctly.


Regards,

Matthias.