[rabbitmq-discuss] |Spam| |Spam| Rabbitmq performence test

[Matthew Sackman]

On Wed, Mar 16, 2011 at 11:31:11AM +0800, yaohui wrote:
> >>On Tue, Mar 15, 2011 at 08:50:43AM +0800, yaohui wrote:
> >>    I do a performence test on my machine using java client with the script runperftest.sh , the message rate is only 4500msg/s. 
> >>   cpu  4 core 3.0G
> >>   memory 4G 
> 
> >runperftest.sh does not attempt to run balanced tests. They merely exist
> >to measure changes between releases. If you construct a more balanced
> >test (i.e. ingress and egress rates are equal) with something like
> >MulticastMain, then you should be able to get much higher throughput.
>   
>  thanks , but the resuslt of MulticastMain is the same. is that the highest throughput?

MulticastMain takes many options, and should be carefully adjusted to
match the performance of your system. The fact that in your run, latency
does not increase suggests that Rabbit is not overloaded, but MCM is
not able to publish messages as fast as it should be able to.

For example on my desktop at home (Q9450 @ 2.6GHz, 4GB, 2.6.32), I get:

> sh build/dist/runjava.sh com.rabbitmq.examples.MulticastMain 
starting consumer #0
starting producer #0
recving rate: 1785 msg/s, min/avg/max latency: 1442/29312/60038 microseconds
sending rate: 26664 msg/s
recving rate: 10155 msg/s, min/avg/max latency: 52340/300171/615841 microseconds
sending rate: 24188 msg/s
recving rate: 14501 msg/s, min/avg/max latency: 614822/986783/1240903 microseconds

I.e. the sending rate is too high, so latency increases rapidly. If I
limit that, I can get it stable at 16.5kHz:

> sh build/dist/runjava.sh com.rabbitmq.examples.MulticastMain -r 16500
starting consumer #0
starting producer #0
recving rate: 9004 msg/s, min/avg/max latency: 347/173694/338583 microseconds
sending rate: 16501 msg/s
recving rate: 14708 msg/s, min/avg/max latency: 332890/419962/452755 microseconds
sending rate: 16501 msg/s
recving rate: 16891 msg/s, min/avg/max latency: 416239/432187/445180 microseconds
sending rate: 16501 msg/s
recving rate: 16645 msg/s, min/avg/max latency: 392858/405681/419339 microseconds
sending rate: 16501 msg/s
recving rate: 16741 msg/s, min/avg/max latency: 393234/414833/431492 microseconds
sending rate: 16501 msg/s

Note how after an initial spike, the receiving rate stays consistently
above the sending rate and the latency starts to drop.

Finally, if I add -a to stop the consumer having to send back acks:

> sh build/dist/runjava.sh com.rabbitmq.examples.MulticastMain -r 24000 -a
starting consumer #0
starting producer #0
recving rate: 12173 msg/s, min/avg/max latency: 377/151754/325200 microseconds
sending rate: 24000 msg/s
recving rate: 25855 msg/s, min/avg/max latency: 294027/343956/388849 microseconds
sending rate: 23977 msg/s
recving rate: 26798 msg/s, min/avg/max latency: 176395/222305/295340 microseconds
sending rate: 23977 msg/s
recving rate: 27465 msg/s, min/avg/max latency: 31190/111042/178470 microseconds
sending rate: 23977 msg/s
recving rate: 24411 msg/s, min/avg/max latency: 1926/13614/32481 microseconds
sending rate: 24000 msg/s
recving rate: 24126 msg/s, min/avg/max latency: 2419/8967/18779 microseconds
sending rate: 23984 msg/s

It's stable at 24kHz. It's actually stable at just below 25kHz, but the
trace is much longer to show that. Finally, if I do multiple consumers,
the outbound rates multiply so you can get silly numbers:

> sh build/dist/runjava.sh com.rabbitmq.examples.MulticastMain -r 4800 -a -e e -t fanout -y 10
starting consumer #0
starting consumer #1
starting consumer #2
starting consumer #3
starting consumer #4
starting consumer #5
starting consumer #6
starting consumer #7
starting consumer #8
starting consumer #9
starting producer #0
recving rate: 19908 msg/s, min/avg/max latency: 374/213387/449689 microseconds
sending rate: 4801 msg/s
recving rate: 43505 msg/s, min/avg/max latency: 159054/457996/891523 microseconds
sending rate: 4801 msg/s
recving rate: 48327 msg/s, min/avg/max latency: 133656/460732/1393276 microseconds
sending rate: 4796 msg/s
recving rate: 49496 msg/s, min/avg/max latency: 105081/409588/1873754 microseconds
sending rate: 4801 msg/s
recving rate: 51456 msg/s, min/avg/max latency: 100556/388283/2161097 microseconds
sending rate: 4801 msg/s
recving rate: 50709 msg/s, min/avg/max latency: 12505/458151/2138811 microseconds
sending rate: 4801 msg/s
recving rate: 47776 msg/s, min/avg/max latency: 2074/365075/1354416 microseconds
sending rate: 4801 msg/s
recving rate: 48505 msg/s, min/avg/max latency: 23075/354988/964308 microseconds
sending rate: 4801 msg/s
recving rate: 48564 msg/s, min/avg/max latency: 7803/307815/625464 microseconds
sending rate: 4801 msg/s

Now, we never quote these numbers because they're incredibly misleading
if you just generally say "Rabbit can do 50kHz". It can, but in this
case, in a 1-in 10-out scenario. In a 1-in, 1-out, it can't do 50kHz.

Anywho, I don't know why you're not geting the same performance figures.

I'm not really sure where to start looking to tune things. Probably try
to use various system monitoring tools to try to figure out where the
bottlenecks are.

Matthew