[rabbitmq-discuss] Distributing Tasks to Worker Queues Across Federated Exchanges
stuff at moesel.net
Tue Dec 18 17:13:20 GMT 2012
Thanks for the suggestions. This definitely gets me closer to what I'm
looking for! I didn't know about the alternate-exchange, so that's very
helpful to know (and could be helpful in other scenarios too). The
ring-configuration sounds like it could work too, but then I guess I am
giving up some high availability-- in that a single broken link breaks the
chain and potentially prevents communication to other brokers that are
still running fine.
Definitely something to think about. Thanks, Simon!
On Tue, Dec 18, 2012 at 10:10 AM, Simon MacMullen <simon at rabbitmq.com>wrote:
> On 18/12/12 13:55, Chris wrote:
>> I have a federation of brokers (let's say two: A and B). They are
>> federated because they are across WAN, so I know clustering will not
>> work. I would like to setup worker queues so that if I submit a job to
>> an exchange/queue, it will be processed by one and only one consumer
>> across the federated brokers. Ideally, it should also prefer local
>> consumers before going to federated consumers.
>> For example:
>> Broker A: Workers listening on queues with routing keys X & Y
>> Broker B: Workers listening on queues with routing keys Y & Z
>> 1) Submit job with routing key X from Broker A: Should go to Consumer of
>> X on Broker A
>> 2) Submit job with routing key Y from Broker A: Should go ONLY to
>> Consumer of Y on Broker A
>> 3) Submit job with routing key Z from Broker A: Should go to Consumer of
>> Z on Broker A
> I think that the alternate-exchange mechanism (http://www.rabbitmq.com/ae.
> **html <http://www.rabbitmq.com/ae.html>) might help you here. Publish to
> a non-federated direct exchange X1. Declare X1 with an AE of a federated
> direct exchange: X2.
> Your workers will need to bind to both X1 and X2 unfortunately, but
> messages should get routed the way you want - if there's a local binding
> then the message never hits the federated exchange, but if there isn't it
> then goes out over federation.
> What this *won't* do is ensure the message only gets delivered to *one*
> other broker over federation. Aha! But you could do that by making each X1
> be federated, but have its upstream set to a single remote X2, with the
> broker federation links arranged in a ring. Then messages go round the
> ring, at each node either being delivered locally or forwarded on by
> federation. You'd need to set max_hops to the ring size at each point, so
> that unroutable messages go round the ring once and then die.
> Does this make sense?
> Cheers, Simon
> Simon MacMullen
> RabbitMQ, VMware
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the rabbitmq-discuss