[rabbitmq-discuss] Does SSD improve rabbitmq's performance

[Graeme N]

Quick update for this, I've got some initial numbers on a box with 8x10k
RPM SAS disks in RAID5, with 128GiB of RAM:

 msg_store_file_size_limit (MiB) queues msgs / queue msg size (MB) total
msgs total msg size (MB) runtime (s) runtime (s) avg throughput MB/s 161001500
1 150000150000 449458 330.76 32100 15001 150000150000 425418 355.87 641001500
1 150000150000 401400 374.53 128100 15001 150000150000 385390 387.10 2561001500
1 150000150000 383385 390.63
I get some reasonable gains by bumping msg_store_file_size_limit up to 128
MiB, 256 MiB seems to be only marginal returns. This may simply be because
it can't batch up more data than that in RAM prior to writing, given the
RAM in my test system. Note that the underlying FS itself is capable of
delivering 780MB/s in block writes, so this is still less than I expected
to see, but demonstrates there's some minor advantages to be had by using
larger store sizes.

My next theory was that 100 queues would mean a minimum of 100 on disk
persistent files, and that I'd see better block throughput with less queues
and more messages. However, testing with lower numbers of queues and higher
messages per queue (to keep total messages delivered constant) revealed
some weird behaviour:

msg_store_file_size_limit (MiB)queuesmsgs / queuemsg size (MB)total msgstotal
msg size (MB)runtime (s)runtime (s)avg throughput MB/s12810015001150000
150000385390387.101285030001150000150000373376400.531282560001150000150000
409413364.9612810150001150000150000463442331.491285300001150000150000465448
328.59
Looks like queue contention becomes a bigger bottleneck than IO throughput
once you get below 50 queues, which is a result I didn't expect. In any
case, looks like max storage size ~= RAM hits some sort of
buffering/contention sweet spot, at least with larger messages, so that's
something which should be easy enough to automate.

Graeme




On Wed, Nov 20, 2013 at 2:01 PM, Graeme N <graeme at sudo.ca> wrote:

> On Wed, Nov 20, 2013 at 12:56 PM, Matthias Radestock <
> matthias at rabbitmq.com> wrote:
>
>> The msg_store has a gc hat has a hard upper bound on the amount of
>> garbage left around. See the essay near the beginning of
>>
>> http://hg.rabbitmq.com/rabbitmq-server/file/default/src/rabbit_msg_store.erlfor the gory details.
>>
>
> This is extremely informative. Definitely helping me get a better
> understanding of what Rabbit's IO patterns are. Thanks!
>
> Graeme
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20131121/d5fd57cb/attachment.htm>