<font face="trebuchet ms,sans-serif">Hi dns, I'll try to bring my own experience on the subject.�</font><span style="font-family:'trebuchet ms',sans-serif">When you federate exchanges between brokers you are in the loose sense having the downstream exchange acting as a consumer for the upstream exchange. I may have misunderstood you but I think that having 8000 brokers rather 8000 remote consumers on the central broker doesn't solve the consistency problem you are describing.</span><div>
<font face="trebuchet ms,sans-serif"><br></font></div><div><font face="trebuchet ms,sans-serif">Let's make an example with federated brokers on the happy path:</font></div><div><ol><li><span style="font-family:'trebuchet ms',sans-serif">One message is published on the upstream exchange and reaches the downstream exchange, and in turn the queue and then the consumer on the downstream broker</span></li>
<li><span style="font-family:'trebuchet ms',sans-serif">The downstream broker confirms the message to the upstream broker so the upstream broker can forget about it</span></li><li><span style="font-family:'trebuchet ms',sans-serif">The connection between the brokers goes down</span></li>
<li><span style="font-family:'trebuchet ms',sans-serif">The consumer on the downstream broker processes the message, acknowledges it and everyone's happy</span></li></ol><div><font face="'trebuchet ms', sans-serif">Here's the sad path with federated brokers:</font></div>
<div><ol><li><font face="'trebuchet ms', sans-serif">same as above</font></li><li><font face="'trebuchet ms', sans-serif">The connection between the brokers is lost before the downstream can confirm the message to the upstream (unlikely, but can definitely happen)</font></li>
<li><font face="'trebuchet ms', sans-serif">The consumer on the downstream broker processes the message and acknowledges it to its local broker</font></li><li><font face="'trebuchet ms', sans-serif">The connection between the two brokers is restored, and thus the upstream will resend the unconfirmed messages once again</font></li>
<li><font face="'trebuchet ms', sans-serif">The consumer on the downstream will receive the same message it has already consumed, therefore it will need a way to deduplicate it</font></li></ol><div><font face="'trebuchet ms', sans-serif">Now the safe path with just the central broker:</font></div>
</div><div><ol><li><font face="'trebuchet ms', sans-serif">One message is published on the central broker and reaches a remote consumers</font></li><li><font face="'trebuchet ms', sans-serif">The consumer processes the message but the connection to the central broker will go down before it can ack the message</font></li>
<li><font face="'trebuchet ms', sans-serif">Once the connection is restored the consumer will receive the unacked message again, so also in this case it will need to deduplicate it</font></li></ol><div><font face="'trebuchet ms', sans-serif">Some observations:</font></div>
<div><ul><li><span style="font-family:'trebuchet ms',sans-serif">in any case your consumer will need to be somewhat aware of the fact that it is quite distant from the original broker and cope with what this implies in some way (message duplication, for one)</span></li>
<li><span style="font-family:'trebuchet ms',sans-serif">with the central broker the remote consumers will have to handle connection failures themselves, something which is less likely to happen with a local broker, although not impossible of course. This is handled by the federation plugin in case of federated brokers instead</span></li>
<li><span style="font-family:'trebuchet ms',sans-serif">Having 8000 brokers rather than 8000 consumers is going to require quite some maintenance, and I'm not sure how well the federation plugin RabbitMQ can handle that, I am under the impression that it was not designed for this many federated exchanges, but I may be wrong</span></li>
<li><span style="font-family:'trebuchet ms',sans-serif">You mention synchronicity between the central location and the remote consumers. Consider that even if you federate brokers and you see all messages delivered and acked on the federation queues, it doesn't tell you anything about whether the remote consumers have processed the messages or not</span></li>
</ul></div></div><br><div class="gmail_quote">On Mon, Jan 23, 2012 at 19:54, dnsplus <span dir="ltr"><<a href="mailto:developer@dnsplus.net">developer@dnsplus.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im"><br>
<br>
<br>
Jerry Kuch wrote:<br>
><br>
> Hi, dnsplus:<br>
><br>
> Could you perhaps clarify a bit what you mean by "remote?" �Are you<br>
> talking about systems that are geographically separated from your<br>
> "central broker" for some reason external to your application? �Or<br>
> are you just talking about having 8,000 consumers? �The latter<br>
> shouldn't be a daunting number at all for even fairly modest Rabbit<br>
> deployments.<br>
><br>
> Best regards,<br>
> Jerry<br>
><br>
><br>
<br>
</div>Its a little complicated, but I'll give it my best shot:<br>
<br>
Jerry, for you ... Definition of a Site: �A retail store location, from 1 to<br>
~1800 miles away from the data center where the Central Broker runs).<br>
<br>
I am working with a large environment which has no guarantee of a stable<br>
network, or guaranteed bandwidth to the remote sites. The typical CIR is<br>
384Kbps, with a 1.544Mbps MAX (yep, fractional T1s). To mitigate the<br>
possibility that network connectivity to a site, or region, can be<br>
unavailable for an undetermined period of time, at any time, and for any<br>
reason ... we think guaranteed message delivery with RabbitMQ is the<br>
foundation we need for this 'application' to work.<br>
<br>
The message payload for this application will be well formatted XML which<br>
contains instructions that will be parsed, interpreted, and executed via a<br>
consumer at each client site, implemented in PERL.<br>
<br>
So ... should I be using a Central Broker, which forwards a message to a<br>
queue at a remote broker, which is consumed by the client process?<br>
<br>
Or, do I leverage the Central Broker with a Queue for each remote site, and<br>
configure the consumers to talk to the Central Broker for message collection<br>
and processing.<br>
<br>