[rabbitmq-discuss] Clustering setup - Inputs

Arun Rao arunrao.seattle at gmail.com
Thu Feb 13 17:07:46 GMT 2014

Thank you so much, Simon, as always!

I learnt from the documentation that queues are single threaded resource.
What does this mean to delivering and acknowledging messages? If

Cross cluster traffic are they asynchronous?


On Thu, Feb 13, 2014 at 7:29 AM, Simon MacMullen <simon at rabbitmq.com> wrote:

> On 13/02/14 06:25, Arun Rao wrote:
>> 1. If I had 10 servers, will I achieve more throughput having 5 clusters
>> (2 nodes each) compared to 2 clusters (5 nodes each)? I get into the
>> grey area when i think if I had 5 servers in a cluster and all servers
>> are taking traffic, then there could be more load syncing traffic across
>> clusters?
> The key question is really: "Where are your queues?" The question of how
> you arrange your nodes into clusters is actually quite secondary to this.
> Any given queue can only be on one node at once (in the case of an HA
> queue the master can only be on one node and the slaves contribute greater
> reliability but not greater throughput). And cross cluster traffic is not
> free.
> So if you only have one queue, you will get the best results if all
> publishers and consumers connect directly to that queue.
> If you have more than one queue, you'll benefit from sharing the queues
> across nodes. It's still always best if your consumers and producers
> connect to the same node as the queue is on. Obviously this can't always be
> arranged :-) but it should be followed as much as possible.
>  2. Will I benefit if I had publishers publishing to 3 servers out of 4
>> servers (in a cluster) and consumers consuming from all 4 servers. This
>> way the 4th server not in publisher's radar will be more effective in
>> draining clusters queues?
> Again, it depends where the queues are.
>  3. Higher QoS Prefetch count and multiple acknowledgements.
> At the moment, turning off prefetch (either don't use basic.qos or set
> prefetch_count to 0) is much faster, especially when consuming across a
> cluster (i.e. with the connection of a different node to the queue). This
> is because qos is shared between consuming queues on a channel, so we need
> to do a bit of coordination between cluster nodes just to figure out if we
> can deliver a message.
> We will have an improvement to this in 3.3.0.
> Acknowledgements with multiple=true will help somewhat, if only by
> reducing the amount of work your consuming connection / channel have to do.
> But the queue will have to do nearly as much work, so they are no panacea.
> Hope this helps.
> Cheers, Simon
> --
> Simon MacMullen
> RabbitMQ, Pivotal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20140213/0708758b/attachment.html>

More information about the rabbitmq-discuss mailing list