[rabbitmq-discuss] Throughput observation with RabbitMQ-3.1.3 and Erlang R16B01 - Single Node and Cluster Node

Thu Aug 1 20:27:34 BST 2013

Priyanki Vashi:

> 1) Why there is no linear increase in throughput with DISK type of node as it's seen with RAM type of node ?

Because disks are much slower than RAM and performing multiple I/O operations in parallel often won't yield
linear increase in I/O throughput (depends on your disks and how they are set up; try with a striping array of SSDs ;))

With RAM access, it is more likely that you will see nearly linear increase.

> 3) What can be done to improve throughput in both the Tests ?

It's hard to give you good advice without seeing the code.

> 4) Since I have VM with 20 cores dedicated for rabbitMQ execution, how can I load the CPU to it's limit ? with the current tests I can load CPU maximum to 800% with    above mentioned throughput. currently the limiting factor seems to be server latency so how to overcome that ?

I suspect that the limiting factor is disk and network I/O and not CPU.
If Erlang runtime does not need more cores, it will not use more cores.

Try running iostat or vmstat alongside your benchmarks to see what % of load your disk has.
Monitoring network throughput is another thing to do. Finally, TCP stack configuration matters.
For example, does Pika disable Nagle's algorithm [1] on the sockets it uses? I'm not sure.

1. http://boundary.com/blog/2012/05/02/know-a-delay-nagles-algorithm-and-you
--
MK

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130801/82bab552/attachment.pgp>