<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 12 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
        {font-family:Calibri;
        panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
        {margin:0in;
        margin-bottom:.0001pt;
        font-size:11.0pt;
        font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
        {mso-style-priority:99;
        color:blue;
        text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
        {mso-style-priority:99;
        color:purple;
        text-decoration:underline;}
span.EmailStyle17
        {mso-style-type:personal-compose;
        font-family:"Calibri","sans-serif";
        color:windowtext;}
.MsoChpDefault
        {mso-style-type:export-only;
        font-size:10.0pt;}
@page WordSection1
        {size:8.5in 11.0in;
        margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
        {page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">I am looking for a little help setting up a shovel between two rabbitmq clusters. We have an application that will need to send messages between two different datacenters. Each datacenter will have a RabbitMQ cluster. I have set up
a shovel in our lab to verify that this will meet the applications requirements. It is fairly straightforward to set up the shovel between standalone RabbitMQ nodes. However, I have ran into issues implementing the shovel with RabbitMQ clusters. The
shovel encounters an error when trying to failover the shovel from one node to another within the source cluster. The problem appears to be around the source queue the shovel is using. The configuration I am testing with is as follows:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">[<o:p></o:p></p>
<p class="MsoNormal"> {rabbitmq_shovel,<o:p></o:p></p>
<p class="MsoNormal"> [{shovels,<o:p></o:p></p>
<p class="MsoNormal"> [{test_shovel,<o:p></o:p></p>
<p class="MsoNormal"> [{sources, [{brokers,<o:p></o:p></p>
<p class="MsoNormal"> ["amqp://guest:guest@ron-rabbitmq:5672",<o:p></o:p></p>
<p class="MsoNormal"> "amqp://guest:guest@ron-rabbitmq1:5672"] },<o:p></o:p></p>
<p class="MsoNormal"> {declarations,<o:p></o:p></p>
<p class="MsoNormal"> [{'queue.declare',<o:p></o:p></p>
<p class="MsoNormal"> [{queue, <<"shovel_out_queue">>},<o:p></o:p></p>
<p class="MsoNormal"> durable]},<o:p></o:p></p>
<p class="MsoNormal"> {'exchange.declare',<o:p></o:p></p>
<p class="MsoNormal"> [{exchange, <<"shovel_exchange">>},<o:p></o:p></p>
<p class="MsoNormal"> {type, <<"direct">>},<o:p></o:p></p>
<p class="MsoNormal"> durable]},<o:p></o:p></p>
<p class="MsoNormal"> {'queue.bind',<o:p></o:p></p>
<p class="MsoNormal"> [{exchange, <<"shovel_exchange">>},<o:p></o:p></p>
<p class="MsoNormal"> {queue, <<"shovel_out_queue">>},<o:p></o:p></p>
<p class="MsoNormal"> {routing_key, <<"shovel_rk">>}]}<o:p></o:p></p>
<p class="MsoNormal"> ]}]},<o:p></o:p></p>
<p class="MsoNormal"> {destinations, [{broker, "amqp://guest:guest@ron-rabbitmq2:5672"},<o:p></o:p></p>
<p class="MsoNormal"> {declarations,<o:p></o:p></p>
<p class="MsoNormal"> [{'queue.declare',<o:p></o:p></p>
<p class="MsoNormal"> [{queue, <<"shovel_in_queue">>},<o:p></o:p></p>
<p class="MsoNormal"> durable]},<o:p></o:p></p>
<p class="MsoNormal"> {'exchange.declare',<o:p></o:p></p>
<p class="MsoNormal"> [{exchange, <<"shovel_exchange">>},<o:p></o:p></p>
<p class="MsoNormal"> {type, <<"direct">>},<o:p></o:p></p>
<p class="MsoNormal"> durable]},<o:p></o:p></p>
<p class="MsoNormal"> {'queue.bind',<o:p></o:p></p>
<p class="MsoNormal"> [{exchange, <<"shovel_exchange">>},<o:p></o:p></p>
<p class="MsoNormal"> {queue, <<"shovel_in_queue">>},<o:p></o:p></p>
<p class="MsoNormal"> {routing_key, <<"shovel_rk">>}]}<o:p></o:p></p>
<p class="MsoNormal"> ]}]},<o:p></o:p></p>
<p class="MsoNormal"> {queue, <<"shovel_out_queue">>},<o:p></o:p></p>
<p class="MsoNormal"> {ack_mode, no_ack},<o:p></o:p></p>
<p class="MsoNormal"> {publish_properties, [{delivery_mode, 2}]},<o:p></o:p></p>
<p class="MsoNormal"> {publish_fields, [{exchange, <<"shovel_exchange">>},<o:p></o:p></p>
<p class="MsoNormal"> {routing_key, <<"shovel_rk">>}]},<o:p></o:p></p>
<p class="MsoNormal"> {reconnect_delay, 5}<o:p></o:p></p>
<p class="MsoNormal"> ]}<o:p></o:p></p>
<p class="MsoNormal"> ]<o:p></o:p></p>
<p class="MsoNormal"> }]<o:p></o:p></p>
<p class="MsoNormal"> }<o:p></o:p></p>
<p class="MsoNormal">].<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The error I am enountering is:<o:p></o:p></p>
<p class="MsoNormal">** Reason for termination ==<o:p></o:p></p>
<p class="MsoNormal">** {{shutdown,<o:p></o:p></p>
<p class="MsoNormal"> {server_initiated_close,404,<o:p></o:p></p>
<p class="MsoNormal"> <<"NOT_FOUND - home node 'RABBIT1@ron-rabbitmq' of durable queue 'shovel_out_queue' in vhost '/' is down or inaccessible">>}},<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">It seems that it can’t access the queue because it doesn’t exist on the source cluster node it failed over to. I have considered having the source queue be transient, but there is the potential to lose orphaned messages with that approach.
I have also thought about mirroring the queue, but additional overhead may be a concern due to the high volume of traffic. Any thoughts or suggestions would be appreciated.
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><b><i><span style="color:#0070C0">Ron Jaeger</span></i></b><b><i><span style="font-size:12.0pt;font-family:"Times New Roman","serif";color:#0070C0"><o:p></o:p></span></i></b></p>
<p class="MsoNormal"><b><span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#F79646">Thomson Reuters</span></b><span style="color:#F79646">
</span><span style="font-size:10.0pt;color:#F79646"><o:p></o:p></span></p>
</div>
</body>
</html>