HI Matthias,<div><br></div><div>Could you direct me to an example of how to set up a "stand alone" shovel?</div><div>The only example[1] i found is demonstrating an embedded shovel as rabbitmq plugin. This seems to be my degenerate case all over again. No?</div>
<div><br></div><div>[1] <a href="http://hopper.squarespace.com/blog/2008/6/22/introducing-shovel-an-amqp-relay.html">http://hopper.squarespace.com/blog/2008/6/22/introducing-shovel-an-amqp-relay.html</a></div><div> <br clear="all">
Peter Fitzgibbons<br>(847) 859-9550<br>Email: <a href="mailto:peter.fitzgibbons@gmail.com">peter.fitzgibbons@gmail.com</a><br>IM GTalk: peter.fitzgibbons<br>IM AOL: <a href="mailto:peter.fitzgibbons@gmail.com">peter.fitzgibbons@gmail.com</a><br>
<br><br><div class="gmail_quote">On Sat, Feb 6, 2010 at 12:42 PM, Matthias Radestock <span dir="ltr"><<a href="mailto:matthias@lshift.net">matthias@lshift.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Peter,<br>
<br>
Peter Fitzgibbons wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">
I'm thinking about how to handle client-connection failover. By this<br>
I mean clientA has a producerP that needs to have reliable and<br>
tolerant connections to rabbitA and rabbitB. This is so that if the<br>
connection/pid of rabbitA goes down, producerP sends it's next message<br>
to rabbitB with as little handling (and wall-clock time) as possible.<br></div>
[...]<div class="im"><br>
The best solution I can think of right now is to have a rabbitP on the<br>
same machine as producerP, which is clustered to the "real" cluster,<br>
so if rabbitA goes down, the built-in clustering failover will handle<br>
the proper interaction. My issue with this is considering how this is<br>
configured and maintained when the farm of servers with producerP gets<br>
to be 1000+. Even if this doesn't scale to the google-farm level,<br>
what about farm = 10, 30 ?<br>
</div></blockquote>
<br><div><div></div><div class="h5">
Having a local broker is indeed a common approach to get higher resilience, particularly since it allows producers to offload messages even when the connection to the main brokers is down for prolonged periods of time, e.g. in the event the main brokers are across a WAN and there is a WAN failure.<br>
<br>
As you suspect, making all these brokers part of a cluster would indeed be problematic for high node counts, even more so when going across a WAN. Besides, RabbitMQ's clustering is really designed for scaling, not improved reliability.<br>
<br>
But you do not need to cluster the brokers. Instead you could stick our experimental shovel (see <a href="http://www.lshift.net/blog/2010/02/01/rabbitmq-shovel-message-relocation-equipment" target="_blank">http://www.lshift.net/blog/2010/02/01/rabbitmq-shovel-message-relocation-equipment</a>) onto each local node and configure it to shovel messages to the main broker pair. The shovel has built-in re-connection and failover logic for that.<br>
<br>
Of course having a local broker is no panacea for reliability, since that broker itself becomes a point of failure. Whether that is a problem depends on how high you place the reliability bar.<br>
<br>
An alternative approachh is to make the clients themselves deal with re-connection and failover. Check out the experimental <a href="http://hg.rabbitmq.com/rabbitmq-java-messagepatterns/" target="_blank">http://hg.rabbitmq.com/rabbitmq-java-messagepatterns/</a> and <a href="http://hg.rabbitmq.com/rabbitmq-dotnet-messagepatterns/" target="_blank">http://hg.rabbitmq.com/rabbitmq-dotnet-messagepatterns/</a> libraries which do that for a specific type of messaging (point-to-point, though most of the logic should be easily ported to other types).<br>
<br>
<br>
Regards,<br>
<br>
Matthias<br>
</div></div></blockquote></div><br></div>