[rabbitmq-discuss] Architecture Question: 2 Brokers, or Not 2 Brokers ?

[dnsplus]

simone_b wrote:
> 
> Hi dns, I'll try to bring my own experience on the subject. When you
> federate exchanges between brokers you are in the loose sense having the
> downstream exchange acting as a consumer for the upstream exchange. I may
> have misunderstood you but I think that having 8000 brokers rather 8000
> remote consumers on the central broker doesn't solve the consistency
> problem you are describing.
> 
> Let's make an example with federated brokers on the happy path:
> 
>    1. One message is published on the upstream exchange and reaches the
>    downstream exchange, and in turn the queue and then the consumer on the
>    downstream broker
>    2. The downstream broker confirms the message to the upstream broker so
>    the upstream broker can forget about it
>    3. The connection between the brokers goes down
>    4. The consumer on the downstream broker processes the message,
>    acknowledges it and everyone's happy
> 
> Here's the sad path with federated brokers:
> 
>    1. same as above
>    2. The connection between the brokers is lost before the downstream can
>    confirm the message to the upstream (unlikely, but can definitely
> happen)
>    3. The consumer on the downstream broker processes the message and
>    acknowledges it to its local broker
>    4. The connection between the two brokers is restored, and thus the
>    upstream will resend the unconfirmed messages once again
>    5. The consumer on the downstream will receive the same message it has
>    already consumed, therefore it will need a way to deduplicate it
> 
> Now the safe path with just the central broker:
> 
>    1. One message is published on the central broker and reaches a remote
>    consumers
>    2. The consumer processes the message but the connection to the central
>    broker will go down before it can ack the message
>    3. Once the connection is restored the consumer will receive the
> unacked
>    message again, so also in this case it will need to deduplicate it
> 
> Some observations:
> 
>    - in any case your consumer will need to be somewhat aware of the fact
>    that it is quite distant from the original broker and cope with what
> this
>    implies in some way (message duplication, for one)
>    - with the central broker the remote consumers will have to handle
>    connection failures themselves, something which is less likely to
> happen
>    with a local broker, although not impossible of course. This is handled
> by
>    the federation plugin in case of federated brokers instead
>    - Having 8000 brokers rather than 8000 consumers is going to require
>    quite some maintenance, and I'm not sure how well the federation plugin
>    RabbitMQ can handle that, I am under the impression that it was not
>    designed for this many federated exchanges, but I may be wrong
>    - You mention synchronicity between the central location and the remote
>    consumers. Consider that even if you federate brokers and you see all
>    messages delivered and acked on the federation queues, it doesn't tell
> you
>    anything about whether the remote consumers have processed the messages
> or
>    not
> 
> 

Thank you for all this. If I were inclined to use central and remote
exchanges/brokers, then SHOVEL is the missing piece I have been looking for.
However ... oh, the overhead with maintaining it all !

I suppose I am inclined to go with a Central Broker and have remote
consumers, and was wondering if someone might convince me otherwise.
However, this has not yet happened. 

Seeing as how my application is more of a "task distribution and management
system" where all sites are processing the same tasks (with RabbitMQ making
sure that the tasks are received), it has occurred to me that I might want a
database component at the central data center as well. I am working on a
conceptual workflow to flush this idea out.

I keep coming back around to keeping as much intelligence as possible at
Central, and keeping the consumer side as lean as I can. 

Thoughts ?


-- 
View this message in context: http://old.nabble.com/Architecture-Question%3A-2-Brokers%2C-or-Not-2-Brokers---tp33189279p33191832.html
Sent from the RabbitMQ mailing list archive at Nabble.com.