[rabbitmq-discuss] cluster write/read performance

Thu May 3 10:51:05 BST 2012

On 02/05/12 17:49, Žilvinas Šaltys wrote:
> Let's say I have 3 remote rabbitmq brokers around the world and I need
> to consume messages from them to a central cluster. As I understand from
> the documentation all write operations happen on the master and then the
> master replicates to slaves.

That's how *mirrored queues* work. Mirrored queues are a feature that 
you can use when clustering, but they're not the only thing.

> That means that the cluster's write/read performance is only as fast as
> the master? In that case what happens if my master can no longer keep up
> and starts lagging further and further behind from the remote brokers?

Just to be clear here, when you say "remote brokers" you are planning to 
connect them with federation / shovel to a central cluster? Clustering 
over a WAN is not recommended - see http://www.rabbitmq.com/distributed.html

> Does it mean I need to set up another independent cluster with it's own
> master to deal with it?

I'm not sure what this means.

> If I'm still on the right path there is no real advantage to have more
> than 2 nodes in a cluster if all you need to do is consume from the
> cluster and have a failover if something goes wrong as all consumes
> happen only on the master ?

Yes, if you're just aiming to have redundancy in your queues. But you 
might want to have a cluster with lots of non-mirrored queues 
distributed around the cluster - in that case you would add more nodes 
to get more capacity.

(You can also have mirrored queues on a subset of cluster nodes to 
achieve the same goal, but the mechanism for doing that is currently a 
bit awkward.)

Cheers, Simon

-- 
Simon MacMullen
RabbitMQ, VMware