[rabbitmq-discuss] Distributing Tasks to Worker Queues Across Federated Exchanges

Simon MacMullen simon at rabbitmq.com
Tue Dec 18 15:10:40 GMT 2012

On 18/12/12 13:55, Chris wrote:
> I have a federation of brokers (let's say two: A and B).  They are
> federated because they are across WAN, so I know clustering will not
> work.  I would like to setup worker queues so that if I submit a job to
> an exchange/queue, it will be processed by one and only one consumer
> across the federated brokers.  Ideally, it should also prefer local
> consumers before going to federated consumers.
> For example:
> Broker A: Workers listening on queues with routing keys X & Y
> Broker B: Workers listening on queues with routing keys Y & Z
> 1) Submit job with routing key X from Broker A: Should go to Consumer of
> X on Broker A
> 2) Submit job with routing key Y from Broker A: Should go ONLY to
> Consumer of Y on Broker A
> 3) Submit job with routing key Z from Broker A: Should go to Consumer of
> Z on Broker A

I think that the alternate-exchange mechanism 
(http://www.rabbitmq.com/ae.html) might help you here. Publish to a 
non-federated direct exchange X1. Declare X1 with an AE of a federated 
direct exchange: X2.

Your workers will need to bind to both X1 and X2 unfortunately, but 
messages should get routed the way you want - if there's a local binding 
then the message never hits the federated exchange, but if there isn't 
it then goes out over federation.

What this *won't* do is ensure the message only gets delivered to *one* 
other broker over federation. Aha! But you could do that by making each 
X1 be federated, but have its upstream set to a single remote X2, with 
the broker federation links arranged in a ring. Then messages go round 
the ring, at each node either being delivered locally or forwarded on by 
federation. You'd need to set max_hops to the ring size at each point, 
so that unroutable messages go round the ring once and then die.

Does this make sense?

Cheers, Simon

Simon MacMullen
RabbitMQ, VMware

More information about the rabbitmq-discuss mailing list