[rabbitmq-discuss] limit number of messages buffered in memory in new persister

[Matthew Sackman]

On Fri, Mar 05, 2010 at 09:04:08AM +0000, Matthias Radestock wrote:
> transactional and persistent:
> $ sh build/dist/runjava.sh com.rabbitmq.examples.MulticastMain -m 1 -a 
> -f persistent -i 2 -z 1
> starting consumer #0
> starting producer #0
> recving rate avg: 9 msg/s
> sending rate avg: 9 msg/s

Note this test was done on a recent ubuntu, which is running ext4 on its
fs, with barriers turned on. On my machine, Debian Sid, which runs ext3
with data=ordered, I get

recving rate avg: 487 msg/s
sending rate avg: 487 msg/s

On my machine on an SSD, which is mounted ext4 barrier=1,
data=writeback, I get

recving rate avg: 1138 msg/s
sending rate avg: 1138 msg/s

On another ubuntu karmic box, with ext3, we get

recving rate avg: 220 msg/s
sending rate avg: 220 msg/s

all on the same test.

Also it's a good idea to make sure the disk schedular is set to
deadline, not cfq, as it normally is by default. You also really want to
stop anything else from writing to disk, and turn of CPU Frequency
Scaling. Basically, give Rabbit its own disk, and very carefully tune
and test. You may have good results with the latest btrfs too, depends
how bleeding edge you want to be. As you can see, SSD performance is
astonishing, because fsyncs on SSDs are constant time and an order of
magnitude faster than rotating media.

Matthew