<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 12 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">I am looking for a little help around the topic of setting up a shovel in a rabbitmq clustered environment. We have an application that will need to send messages across two different datacenters. Each datacenter will have a RabbitMQ
cluster. I have set up a shovel in our lab to verify that this will meet the applications requirements. It is fairly straightforward to set up the shovel between standalone RabbitMQ nodes. However, I have ran into issues implementing the shovel between
RabbitMQ clusters, with the error occurring when trying to failover the shovel from one node to another within the cluster. The problem is mostly around the queues the shovel is using. The configuration I am testing with is as follows:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">[<o:p></o:p></p>
<p class="MsoNormal"> {rabbitmq_shovel,<o:p></o:p></p>
<p class="MsoNormal"> [{shovels,<o:p></o:p></p>
<p class="MsoNormal"> [{test_shovel,<o:p></o:p></p>
<p class="MsoNormal"> [{sources, [{brokers,<o:p></o:p></p>
<p class="MsoNormal"> ["amqp://guest:guest@ron-rabbitmq:5672",<o:p></o:p></p>
<p class="MsoNormal"> "amqp://guest:guest@ron-rabbitmq1:5672"] },<o:p></o:p></p>
<p class="MsoNormal"> {declarations,<o:p></o:p></p>
<p class="MsoNormal"> [{'queue.declare',<o:p></o:p></p>
<p class="MsoNormal"> [{queue, <<"shovel_out_queue">>},<o:p></o:p></p>
<p class="MsoNormal"> durable]},<o:p></o:p></p>
<p class="MsoNormal"> {'exchange.declare',<o:p></o:p></p>
<p class="MsoNormal"> [{exchange, <<"shovel_exchange">>},<o:p></o:p></p>
<p class="MsoNormal"> {type, <<"direct">>},<o:p></o:p></p>
<p class="MsoNormal"> durable]},<o:p></o:p></p>
<p class="MsoNormal"> {'queue.bind',<o:p></o:p></p>
<p class="MsoNormal"> [{exchange, <<"shovel_exchange">>},<o:p></o:p></p>
<p class="MsoNormal"> {queue, <<"shovel_out_queue">>},<o:p></o:p></p>
<p class="MsoNormal"> {routing_key, <<"shovel_rk">>}]}<o:p></o:p></p>
<p class="MsoNormal"> ]}]},<o:p></o:p></p>
<p class="MsoNormal"> {destinations, [{broker, "amqp://guest:guest@ron-rabbitmq2:5672"},<o:p></o:p></p>
<p class="MsoNormal"> {declarations,<o:p></o:p></p>
<p class="MsoNormal"> [{'queue.declare',<o:p></o:p></p>
<p class="MsoNormal"> [{queue, <<"shovel_in_queue">>},<o:p></o:p></p>
<p class="MsoNormal"> durable]},<o:p></o:p></p>
<p class="MsoNormal"> {'exchange.declare',<o:p></o:p></p>
<p class="MsoNormal"> [{exchange, <<"shovel_exchange">>},<o:p></o:p></p>
<p class="MsoNormal"> {type, <<"direct">>},<o:p></o:p></p>
<p class="MsoNormal"> durable]},<o:p></o:p></p>
<p class="MsoNormal"> {'queue.bind',<o:p></o:p></p>
<p class="MsoNormal"> [{exchange, <<"shovel_exchange">>},<o:p></o:p></p>
<p class="MsoNormal"> {queue, <<"shovel_in_queue">>},<o:p></o:p></p>
<p class="MsoNormal"> {routing_key, <<"shovel_rk">>}]}<o:p></o:p></p>
<p class="MsoNormal"> ]}]},<o:p></o:p></p>
<p class="MsoNormal"> {queue, <<"shovel_out_queue">>},<o:p></o:p></p>
<p class="MsoNormal"> {ack_mode, no_ack},<o:p></o:p></p>
<p class="MsoNormal"> {publish_properties, [{delivery_mode, 2}]},<o:p></o:p></p>
<p class="MsoNormal"> {publish_fields, [{exchange, <<"shovel_exchange">>},<o:p></o:p></p>
<p class="MsoNormal"> {routing_key, <<"shovel_rk">>}]},<o:p></o:p></p>
<p class="MsoNormal"> {reconnect_delay, 5}<o:p></o:p></p>
<p class="MsoNormal"> ]}<o:p></o:p></p>
<p class="MsoNormal"> ]<o:p></o:p></p>
<p class="MsoNormal"> }]<o:p></o:p></p>
<p class="MsoNormal"> }<o:p></o:p></p>
<p class="MsoNormal">].<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The error I am enountering is:<o:p></o:p></p>
<p class="MsoNormal">** Reason for termination ==<o:p></o:p></p>
<p class="MsoNormal">** {{shutdown,<o:p></o:p></p>
<p class="MsoNormal"> {server_initiated_close,404,<o:p></o:p></p>
<p class="MsoNormal"> <<"NOT_FOUND - home node 'RABBIT1@ron-rabbitmq' of durable queue 'shovel_out_queue' in vhost '/' is down or inaccessible">>}},<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">It is evident that it can’t access the queue because it was on the other cluster member that is down. I have considered having the queues be transient, but there are additional risks associated with that, such as loosing messages during
a node failure. I have also thought about mirroring the queues, but additional overhead may be a concern due to the high volume of traffic. Any thoughts or suggestions would be appreciated.
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><b><i><span style="color:#0070C0">Ron Jaeger</span></i></b><b><i><span style="font-size:12.0pt;font-family:"Times New Roman","serif";color:#0070C0"><o:p></o:p></span></i></b></p>
<p class="MsoNormal"><b><span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#F79646">Thomson Reuters</span></b><span style="color:#F79646">
</span><span style="font-size:10.0pt;color:#F79646"><o:p></o:p></span></p>
</div>
</body>
</html>