[rabbitmq-discuss] RabbitMQ Cluster On AWS VPC

[Simon MacMullen]

On 14/12/2012 5:27PM, Laing, Michael P. wrote:
> We have not yet seen any performance issues. All traffic between
> instances in different availability zones within a region is charged:
> $.01/GB in us-east-1. So there is a cost, and the volume of traffic
> through your HA queues has an impact.
>
> In us-east-1, the increase in latency by going across zones is < 10ms,
> in our small sample. We will add some continuous testing of this in the
> future across all zones.
>
> Regions may vary in the amount of latency. And latency may vary (jitter).
>
> *Rather then LAN vs WAN, it would be useful to know the design envelope
> for rabbitmq clusters in terms of latency and jitter, as these are
> measurable. Also if there is tuning that can be done.

We talk about LAN vs WAN because of the different probability of 
partitions rather than because of performance.

Having said that:

Resource creation and deletion (exchanges / queues / bindings / users / 
vhosts / permissions) is synchronous across the cluster (in fact a 2PC). 
Messaging (publish / confirm / deliver / ack) is async (although there 
were some synchronous bits publishing in mirrored queues in 2.x). And 
it's the synchrony that hurts performance as latency goes up obviously.

> All that said, I may choose to use federation rather than clustering
> across zones, because the rabbitmq response to a net split is so
> catastrophic, apparently, and I am not sure I want to live with that
> uncertainty in production. We are architected in such a way that we can
> do either with minor modifications. We would run a cluster in each zone
> and federate the clusters. It's more expensive though and more moving parts.

I would recommend that you federate across zones.

> We use DynamoDB and Riak as reliable KV stores, as well as s3. Riak
> seems to have good tolerance of net splits. (*Why is it better than
> rabbitmq in this respect?)

Based on a quick reading of their docs, Riak clusters are eventually 
consistent, like RabbitMQ federation.

Cheers, Simon