[rabbitmq-discuss] Clustering setup - Inputs

[Arun Rao]

Thank you so much, Simon, as always!

I learnt from the documentation that queues are single threaded resource.
What does this mean to delivering and acknowledging messages? If
prefetch_count=10.

Cross cluster traffic are they asynchronous?

Thanks!


On Thu, Feb 13, 2014 at 7:29 AM, Simon MacMullen <simon at rabbitmq.com> wrote:

> On 13/02/14 06:25, Arun Rao wrote:
>
>> 1. If I had 10 servers, will I achieve more throughput having 5 clusters
>> (2 nodes each) compared to 2 clusters (5 nodes each)? I get into the
>> grey area when i think if I had 5 servers in a cluster and all servers
>> are taking traffic, then there could be more load syncing traffic across
>> clusters?
>>
>
> The key question is really: "Where are your queues?" The question of how
> you arrange your nodes into clusters is actually quite secondary to this.
>
> Any given queue can only be on one node at once (in the case of an HA
> queue the master can only be on one node and the slaves contribute greater
> reliability but not greater throughput). And cross cluster traffic is not
> free.
>
> So if you only have one queue, you will get the best results if all
> publishers and consumers connect directly to that queue.
>
> If you have more than one queue, you'll benefit from sharing the queues
> across nodes. It's still always best if your consumers and producers
> connect to the same node as the queue is on. Obviously this can't always be
> arranged :-) but it should be followed as much as possible.
>
>
>  2. Will I benefit if I had publishers publishing to 3 servers out of 4
>> servers (in a cluster) and consumers consuming from all 4 servers. This
>> way the 4th server not in publisher's radar will be more effective in
>> draining clusters queues?
>>
>
> Again, it depends where the queues are.
>
>
>  3. Higher QoS Prefetch count and multiple acknowledgements.
>>
>
> At the moment, turning off prefetch (either don't use basic.qos or set
> prefetch_count to 0) is much faster, especially when consuming across a
> cluster (i.e. with the connection of a different node to the queue). This
> is because qos is shared between consuming queues on a channel, so we need
> to do a bit of coordination between cluster nodes just to figure out if we
> can deliver a message.
>
> We will have an improvement to this in 3.3.0.
>
> Acknowledgements with multiple=true will help somewhat, if only by
> reducing the amount of work your consuming connection / channel have to do.
> But the queue will have to do nearly as much work, so they are no panacea.
>
> Hope this helps.
>
> Cheers, Simon
>
>
> --
> Simon MacMullen
> RabbitMQ, Pivotal
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20140213/0708758b/attachment.html>