[rabbitmq-discuss] how does clustering improve performance?

Tue Apr 19 11:01:18 BST 2011

Hi Pierre-Marie,

On Fri, Apr 15, 2011 at 03:14:55PM +0200, PADIOU Pierre-Marie (MORPHO) wrote:
> I have been reading the documentation about clustering in rabbitmq. My
> goal is to achieve maximum performance, not availability (I am not
> using any persistence btw).
> 
> In a previous post on this mailing list ("some questions about
> rabbitmq clustering, answered"), Alexis Richardson wrote:
> 
> "A message arriving at a given node will be routed to appropriate
> queues via AMQP bindings and exchange routing rules.  For queues on
> that node, this means 'zero hops', and for queues on other nodes, this
> means 'one hop'."
> 
> In other words, if you've got a two nodes cluster, and a bunch of
> clients that randomly connects to one of the nodes, it means that for
> each message sent from one client to a queue there is an equal chance
> of having zero or one 'hop'. From the point of view of a rabbit node,
> the throughput will be exactly the same than with a standalone broker
> (the output being equally divided between local clients and the other
> node forming the cluster), *and* the global network traffic will
> increase by 50%.
> 
> Can someone explain what I didn't understand?

You understood everything correctly.

The point of a cluster is to increase _scale_, not straight-line
performance.

Queues, connections, channels etc all have a memory and CPU cost.
Spreading these costs across multiple machines rather than just one will
allow the broker as a logical whole to be able to cope with more queues,
connections, channels, messages etc than just a single machine could.

However, the single threaded items are channels and queues: if you want
a single queue to go faster then you have to use a CPU which is faster
for a single thread.

Matthew