[rabbitmq-discuss] Federated Topic Exchanges

[Richard Raseley]

Simon,

Thank you so much for your detailed response - it was very helpful to me
and answered all of my questions.

Regards,

Richard


On Wed, Feb 27, 2013 at 8:19 AM, Simon MacMullen <simon at rabbitmq.com> wrote:

> On 26/02/13 23:41, Richard Raseley wrote:
>
>> I am contemplating the use of topic exchanges along with the federation
>> plugin to route messages between geographically separated brokers.
>>
>> 1) Will someone confirm my understanding of the configuration is valid?
>>
>> In this situation there would be topic exchange named "B1.TE" "B2.TE"
>> that would be configured within brokers "B1" and "B2" respectively.
>> B1.TE would have B2.TE configured as an upstream exchange and B2.TE
>> would have B1.TE configured as an upstream exchange.
>>
>> Broker B1 would have queues "B1.Q1" and "B1.Q2" configured and bound to
>> exchange B1.TE with routing keys "B1.Q1" and "*.Q2" respectively.
>>
>> Broker B2 would have queues "B2.Q1" and "B2.Q2" configured and bound to
>> exchange B2.TE with routing keys "B2.Q1" and "*.Q2" respectively.
>>
>
> It is valid, certainly. Note that the exchanges and queues in B1 and B2 do
> not need to have distinct names (but maybe you were just doing that to be
> clearer?)
>
>
>  2) Assuming the answer to question #1 above is "yes, it is valid", would
>> someone confirm my understanding of how the following messages would be
>> routed?
>>
>> A message sent to exchange B1.TE with routing key "B2.Q1" would be
>> received, forwarded to exchange B2.TE, placed in queue B2.Q1,
>>
>
> Yes.
>
>
>  and then acknowledgement would be sent back to the client.
>>
>
> No.
>
> Assuming the client is publishing in confirm mode, the confirm will be
> sent back as soon as B1 has taken responsibility for the message. Confirms
> mean "the broker you published to took responsibility for the message".
> Otherwise as your federation grows larger and less-reliable network links
> come into play, you could be waiting a long time for those confirms!
>
>
>  A message sent to exchange B1.TE with routing key "B1.Q2" would be
>> received, a copy placed in queue B1.Q2, a copy forwarded to exchange
>> B2.TE, placed in queue B2.Q2, and then acknowledgement would be sent
>> back to the client.
>>
>> A message sent to exchange B2.TE with routing key "B2.Q2" would be
>> received, a copy placed in queue B2.Q2, a copy forwarded to exchange
>> B1.TE, placed in queue B1.Q2, and then acknowledgement would be sent
>> back to the client.
>>
>
> Again, yes to the routing, no to when confirmation happens.
>
>
>  3) My understanding is that an exchange has knowledge of the routing
>> topology of any upstream queues and therefore won't send a message
>> upstream that wouldn't end up in a queue. Firstly, is that understanding
>> correct?
>>
>
> I think you mean "downstream" in both cases, but yes.
>
>
>  Secondly (if yes to the previous question), what type of
>> overhead are we talking about when using this configuration with a topic
>> exchange?
>>
>
> Effectively the bindings propagate upstream, so each federated exchange
> will end up with all the bindings for all of its downstreams.
>
>
>  My worry is that the processing power required to deliver each message
>> will increase exponentially as I add additional brokers in a "mesh"
>> style federated topology as it seems that the routing table on each
>> upstream exchange has to be examined for each message delivery.
>>
>
> Well, it won't be exponential. But the routing table will get larger.
> Topic routing is approximately O(log n) in the number of bindings.
>
> The federation mechanism doesn't try to trade routing complexity for
> unnecessarily-routed messages by amalgamating topic keys since
>
>  (a) it would have to know about topics for that, it's supposed to be a
>      generic mechanism
>
>  (b) it can't know what the distribution of messages across your
>      routing key space is
>
> However, *you* know that distribution. So you could federate topic
> exchanges and then use e2e bindings from them to local topic exchanges,
> with relatively "large" routing keys that will cover a decent sized range.
> Then apps bind queues to the local topics. That does require a bit more
> up-front setup though - but it kind of has to since it requires knowledge
> of how your routing keys are distributed.
>
> Cheers, Simon
>
> --
> Simon MacMullen
> RabbitMQ, VMware
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130227/fe3ba383/attachment.htm>