[rabbitmq-discuss] Distributing Tasks to Worker Queues Across Federated Exchanges
simon at rabbitmq.com
Tue Dec 18 15:10:40 GMT 2012
On 18/12/12 13:55, Chris wrote:
> I have a federation of brokers (let's say two: A and B). They are
> federated because they are across WAN, so I know clustering will not
> work. I would like to setup worker queues so that if I submit a job to
> an exchange/queue, it will be processed by one and only one consumer
> across the federated brokers. Ideally, it should also prefer local
> consumers before going to federated consumers.
> For example:
> Broker A: Workers listening on queues with routing keys X & Y
> Broker B: Workers listening on queues with routing keys Y & Z
> 1) Submit job with routing key X from Broker A: Should go to Consumer of
> X on Broker A
> 2) Submit job with routing key Y from Broker A: Should go ONLY to
> Consumer of Y on Broker A
> 3) Submit job with routing key Z from Broker A: Should go to Consumer of
> Z on Broker A
I think that the alternate-exchange mechanism
(http://www.rabbitmq.com/ae.html) might help you here. Publish to a
non-federated direct exchange X1. Declare X1 with an AE of a federated
direct exchange: X2.
Your workers will need to bind to both X1 and X2 unfortunately, but
messages should get routed the way you want - if there's a local binding
then the message never hits the federated exchange, but if there isn't
it then goes out over federation.
What this *won't* do is ensure the message only gets delivered to *one*
other broker over federation. Aha! But you could do that by making each
X1 be federated, but have its upstream set to a single remote X2, with
the broker federation links arranged in a ring. Then messages go round
the ring, at each node either being delivered locally or forwarded on by
federation. You'd need to set max_hops to the ring size at each point,
so that unroutable messages go round the ring once and then die.
Does this make sense?
More information about the rabbitmq-discuss