[rabbitmq-discuss] Configure VM memory high watermark

[Matthew Sackman]

Hi Sylvain,

On Tue, Aug 16, 2011 at 10:27:20AM +0200, CORDIER-AKKA Sylvain (MORPHO) wrote:
> Our RabbitMQ host has got 64GB installed RAM and RabbitMQ server
> process uses in normal use about 5GB of RAM.  The normal use point is
> just above the 8% of installed RAM (8% of 64GB is 5.12GB) and a small
> variation of the use point makes the RabbitMQ starts paging messages
> and causes a final system slowdown.
> 
> If the vm_memory_high_watermark is not the good solution, what are the
> available solutions?
> Increase the amount of installed RAM to 96GB for the system starts
> paging at 7.68GB?
> Isn't there another way to delay RabbitMQ to start swapping?

Well. The thing that Rabbit is trying to ensure is that should RAM
continue to be eaten up, by the time it gets to around 26GB used,
everything should be safely on disk and that the transition to putting
things onto disk is a smooth one. Obviously, unless you have insanely
fast disks, writing out 26GB of messages to disk takes some time, which
is why Rabbit starts writing out quite early on.

The use of 0.4 as the default highwater mark is based on the fear that
GC by the Erlang VM could cause a tempory spike of near double the
amount of RAM in the machine. As the amount of RAM you have installed
gets greater and greater, I'd have thought the odds of there being such
a spike reduces. Thus maybe in such machines, a much higher highwater
mark would be reasonable.

If you have the time, what I'd suggest is that you try with the
watermark much higher (say, 0.8) and then stress test it and see if you
can get Rabbit/Erlang to fail (most likely it'll be a a "cannot allocate
X bytes" malloc-style failure). Reduce the watermark and repeat until
you have confidence that for your use case, there will be no fatal
spikes and you still have utilisation of RAM that you're happy with.

Matthew