Thank you.<br><br><div class="gmail_quote">On Sun, Jun 16, 2013 at 11:24 PM, Jason McIntosh <span dir="ltr"><<a href="mailto:mcintoshj@gmail.com" target="_blank">mcintoshj@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Queues exist (definition wise) on all nodes in a cluster. Messages only existing unless the queue is mirrored on a single node. You can mirror a queue's messages to N number of cluster nodes. Admin console is an overall view of the cluster, so you'll see all the available nodes/queues/exchanges/etc. <div>
<br></div><div>One reason you'd NOT have queues have their messages automatically replicated to other nodes is scaling. If you had a cluster of 10 machines, and all the machines mirrored, then your message would have to hit all 10 nodes before being acknowledged (publish confirm situation as I recall). You CAN instead say a queue must be mirrored on N number of nodes, allowing a failover capacity of N. </div>
<div><br></div><div>Rabbit is also a Master-slave write system. Only ONE node in a cluster is the "master". Other nodes would be there for failover/scaling support. i.e. queues could be created on any one of the nodes in a cluster to distribute load. In the event of a failure on the master, a slave node would take over the masters position. When the old master comes on line, it becomes a slave, but unless you have 3.1 and auto healing type stuff, it would only get messages as of the time it has recovered, leaving an indeterminate amount of time before it goes back into sync.</div>
<div><br></div><div>Further, if clients do auto queue creation, it can get even more complex/diverse. RabbitMQ makes messaging pretty simple but architecting it right and true understanding is a much bigger issue. </div>
<div><br></div><div>Jason</div></div><div class="gmail_extra"><br><br><div class="gmail_quote"><div><div class="h5">On Fri, Jun 14, 2013 at 10:37 AM, Girish Gangadharan <span dir="ltr"><<a href="mailto:girish@giri.sh" target="_blank">girish@giri.sh</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5">Thanks for your reply. Please see my response in red.<br><br><div class="gmail_quote">
<div>On Fri, Jun 14, 2013 at 4:03 AM, Emile Joubert <span dir="ltr"><<a href="mailto:emile@rabbitmq.com" target="_blank">emile@rabbitmq.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
Hi,<br>
<br>
On 13/06/13 20:41, Girish Gangadharan wrote:<br>
> For example, if a publisher writes a message to an exchange that ends<br>
> up in a specific queue bound to that exchange, it immediately shows<br>
> up in the same queue on the other node.<br>
<br>
There is only one queue on one node in this case. The cluster offers a<br>
consistent view of all resources in the cluster and allows producers and<br>
consumers to connect to any node. Messages are transparently routed<br>
across the cluster.<br></blockquote><div><br></div></div><div><font color="#cc0000">Does this mean that the queues actually exist only in one node to whose exchange the message was originally written to? And the fact that I can see them in the admin console of the other node is just to provide a consistent view across the cluster? So any action that I do with the original queue on the other nodes' admin consoles basically makes the changes to the queue on the original node and not to the copy of that queue on the other nodes? In fact, if I understand you correctly, there IS no copy of the queue on the other nodes? Did I get that right?</font></div>
<div><font color="#cc0000"><br></font></div><div><font color="#cc0000">Not sure I understand the consistent view explanation either? What are the main benefits of having multiple nodes in a cluster if they can't really share anything? I would think Mirrored Queues should be an automatic by-product of clustering, thus offering HA out of the box. Can you explain to me why somebody would create a cluster of nodes and NOT have mirrored queues set up?</font></div>
<div><div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
> But then why does the article on Mirrored Queues say that *you have<br>
> to explicitly set up mirrored queues by policy*?<br>
<br>
If a queue exists on only one node and that node crashes then the queue<br>
becomes unavailable. Mirrored queues provides resilience by maintaining<br>
copies of the queue that can take over in case a node crashes. See<br>
<a href="http://www.rabbitmq.com/ha.html" target="_blank">http://www.rabbitmq.com/ha.html</a><br>
<br>
> So if one node goes down, the load balancer would automatically<br>
> redirect all the traffic just to the node that is up and running. And<br>
> the clients will not be affected. Would that work? Is that the<br>
> recommended approach?<br>
<br>
That is a common approach. Another is to provide clients with reconnect<br>
logic that allows them to select the alternative node when one node crashes.<br>
<br>
> I was thinking about using Shoveling technique to push messages from<br>
> a logical broker sitting on a cluster in our local HA servers to a<br>
> different logical broker to the DR server. In this case, a copy of<br>
> the messages will keep getting pumped into the DR with no clients<br>
> processing it. What will happen to those messages?<br>
<br>
Nothing will happen to them. They will accumulate in the DR server<br>
unless you take action to remove them, or unless they expire.<br>
<br>
> Should I have a workflow to go purge them every day manually or via a<br>
> script (after I make sure those messages aren't needed anymore since<br>
> the clients have already process them from the main HA cluster).<br>
<br>
That depends on the specifics of your disaster scenario, your messaging<br>
pattern and how you plan to switch over to your DR environment. Purging<br>
queues after a day sounds reasonable.<br>
<br>
> If the main HA cluster goes down and I have to bring the DR server into<br>
> action, what happens to the messages that did get processed already by<br>
> the clients? Should the clients have logic built in to ignore duplicates<br>
> to handle this specific scenario?<br>
<br>
Performing deduplication in clients is probably the simplest solution,<br>
or you could mirror the effects of consumers removing messages from<br>
queues in your DR environment.<br>
<span><font color="#888888"><br>
<br>
<br>
-Emile<br>
<br>
<br>
<br>
<br>
</font></span></blockquote></div></div></div><br>
<br></div></div>_______________________________________________<br>
rabbitmq-discuss mailing list<br>
<a href="mailto:rabbitmq-discuss@lists.rabbitmq.com" target="_blank">rabbitmq-discuss@lists.rabbitmq.com</a><br>
<a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a><br>
<br></blockquote></div><span class="HOEnZb"><font color="#888888"><br><br clear="all"><div><br></div>-- <br>Jason McIntosh<br><a href="http://mcintosh.poetshome.com/blog/" target="_blank">http://mcintosh.poetshome.com/blog/</a><br>
<a href="tel:573-424-7612" value="+15734247612" target="_blank">573-424-7612</a>
</font></span></div>
</blockquote></div><br>