[rabbitmq-discuss] 3 node rabbitmq cluster behind the load balancer

Sun Apr 27 12:02:39 BST 2014

On 27 April 2014 at 08:12:14, sagu prf (sagu.prf1 at gmail.com) wrote:
> > How effieciently, we can setup 6 node rabbitmq cluster with  
> HA mirror
> queues behinde the Load balancer?

It depends on what exactly you mean by “efficiently” but one good
idea in a 6 node cluster is to not use ha-mode = “all” for your queues.

Use ha-mode = “exactly” and mirror to 2 or 3 nodes (depending on how valuable
their contents is for your system). Mirroring to more nodes will only make
RabbitMQ do more work and increase intra-cluster traffic, at no real benefit
for the described setup.

See http://www.rabbitmq.com/ha.html.

> Loadbalancer will place the publishing and consuming request  
> across
> the node and how does the rabbitmq works ?

In a RabbitMQ cluster, clients can connect to and work with any node.
However, queues have a master node that all requests are routed to.
You will reduce intra-cluster traffic and thus improve throughput
and reduce latency if you publish on a node that hosts a cluster for
the queue(s) of interest. The same is true for consumers.

There are multiple load balancing strategies available in modern
tools but they are unaware of the above point. Which brings us to my
(very general) recommendation: if you don’t have extreme throughput
requirements, you can use virtually any load balancing strategy. 
With HAproxy, leastconn [1] is a good idea if you expect a lot of client
connections.

If you do have extreme throughput requirements, you need to infer queue
masters via HTTP API and make your clients connect to master nodes,
and reconnect when it fails.

1. http://haproxy.1wt.eu/download/1.5/doc/configuration.txt, search for “balance <algorithm>"
--  
MK  

Software Engineer, Pivotal/RabbitMQ