[rabbitmq-discuss] performance

[Matthias Radestock]

Ryan,

Ryan Pratt wrote:
> On Fri, Aug 29, 2008 at 5:29 PM, Matthias Radestock <matthias at lshift.net 
> <mailto:matthias at lshift.net>> wrote:
>     Instead of pulling messages one-by-one using basicGet, create a
>     subscription with basicConsume. That will give better results.
> 
> I can't for the use case I am testing for. The case is multiple servers 
> receiving on the same queues. I want less active servers to receive more 
> messages than the active ones for load balancing. My understanding is 
> that consumers are simply round robin, even if some of the servers in 
> the mix are having trouble keeping up.

The server does in fact detects slower consumers and sends them fewer 
messages. This throttling logic is quite crude though and probably too 
coarse-grained for your requirements.

Finer-grained flow control for subscriptions will be possible once we 
implement basic.qos. TBH though I doubt it's going to perform any better 
than basicGet.

>      > The machine is a dual quad core 2.66 GHz Xeon
> 
>     What CPU utilisation are you seeing?
> 
> About  60-70%

How is that distributed between your Java clients and the RabbitMQ broker?

> I have tried adding more processes to the mix (more queues as well as 
> same queue) but get basically the same results.

How many connections and channels are your tests using?

Try running the MulticastMain example that ships with the Java client. 
By default, i.e. w/o setting any of the command line flags/options, this 
will create one producer and one consumer in non-auto-ack mode, send 
messages as fast as it can and report on throughput and latency. You can 
use the "-r" flag to limit the sending rate and prevent messages from 
backing up at the broker.

While this test uses basicConsume rather than basicGet, it will 
nevertheless give you a good idea about the performance and, more 
importantly, I can compare the results against what we are seeing on 
some of our test machines.

>     Also, what OS are you on and what version of Erlang/OTP are you running?
> 
> I am on CentOS 5 running erlang 5.6.3

Some testing we did in the past indicates that generally a clustered 
broker - with one node per core and smp disabled for the Erlang VM - 
performs significantly better than a single smp-enabled node.


Matthias.