[rabbitmq-discuss] Clustering setup - Inputs
simon at rabbitmq.com
Thu Feb 13 15:29:48 GMT 2014
On 13/02/14 06:25, Arun Rao wrote:
> 1. If I had 10 servers, will I achieve more throughput having 5 clusters
> (2 nodes each) compared to 2 clusters (5 nodes each)? I get into the
> grey area when i think if I had 5 servers in a cluster and all servers
> are taking traffic, then there could be more load syncing traffic across
The key question is really: "Where are your queues?" The question of how
you arrange your nodes into clusters is actually quite secondary to this.
Any given queue can only be on one node at once (in the case of an HA
queue the master can only be on one node and the slaves contribute
greater reliability but not greater throughput). And cross cluster
traffic is not free.
So if you only have one queue, you will get the best results if all
publishers and consumers connect directly to that queue.
If you have more than one queue, you'll benefit from sharing the queues
across nodes. It's still always best if your consumers and producers
connect to the same node as the queue is on. Obviously this can't always
be arranged :-) but it should be followed as much as possible.
> 2. Will I benefit if I had publishers publishing to 3 servers out of 4
> servers (in a cluster) and consumers consuming from all 4 servers. This
> way the 4th server not in publisher's radar will be more effective in
> draining clusters queues?
Again, it depends where the queues are.
> 3. Higher QoS Prefetch count and multiple acknowledgements.
At the moment, turning off prefetch (either don't use basic.qos or set
prefetch_count to 0) is much faster, especially when consuming across a
cluster (i.e. with the connection of a different node to the queue).
This is because qos is shared between consuming queues on a channel, so
we need to do a bit of coordination between cluster nodes just to figure
out if we can deliver a message.
We will have an improvement to this in 3.3.0.
Acknowledgements with multiple=true will help somewhat, if only by
reducing the amount of work your consuming connection / channel have to
do. But the queue will have to do nearly as much work, so they are no
Hope this helps.
More information about the rabbitmq-discuss