[rabbitmq-discuss] Flow Control in RabbitMQ 1.5.3

Wed Apr 1 00:22:10 BST 2009

Chris,

Chris Pettitt wrote:
> What I'm seeing is usually some variation of the following:
> 
> 1. Start broker clean
> 2. Start 1 consumer
> 3. Start 10 producers that publish as fast as they can (we're trying
> to stress the system)
> 4. Once system memory reaches the high water mark throttling occurs
> (I've tried settings from about 40% - 95%, the rest of the
> observations will refer to measurements at 70%).
> 5. Throttling toggles on and off a few times, between 3 - 5 times, and
> then all clients (including the consumer) get disconnected.
> 6. Memory sores to over 90%.
> 6. Sometimes the erlang process crashes at this point, other times all
> producers and consumers reconnect and within about 30 seconds the
> erlang process crashes. In most cases the producers never produce
> after the reconnect, but on some occasions the consumer does receive
> messages before dying.

I have tried to reproduce the above by running the 
com.rabbitmq.examples.MulticastMain test with the args "-a -x 10 -s 
1024", and while I can make RabbitMQ crash with the default 95% limit, 
once I lowered the limit to 70% it kept going.

That's on an old-ish machine with only 4GB of memory though. It's 
conceivable that a faster machine with more memory requires more of a 
margin. 40%, which is the lowest limit you tried, is quite low, but it 
may be worth trying even lower limits.

Can you send us the code for your tests? I'd like to gain a better 
understanding of exactly what parts of RabbitMQ get stressed, and how 
your test differs from what MulticastMain with the above params does.

Also, what version of Erlang are you running?

Regards,

Matthias.