[rabbitmq-discuss] Question regarding setting up a shovel between two clusters

Fri Oct 4 17:23:12 BST 2013

I am looking for a little help setting up a shovel between two rabbitmq clusters.  We have an application that will need to send messages between two different datacenters.   Each datacenter will have a RabbitMQ cluster.   I have set up a shovel in our lab to verify that this will meet the applications requirements.  It is fairly straightforward to set up the shovel between standalone RabbitMQ nodes.  However,  I have ran into issues implementing  the shovel with RabbitMQ clusters.  The  shovel encounters an  error when trying to failover the shovel from one node to another within the source cluster.   The problem appears to be around the source queue the shovel is using.  The configuration I am testing with is as follows:

[
    {rabbitmq_shovel,
       [{shovels,
         [{test_shovel,
            [{sources, [{brokers,
                          ["amqp://guest:guest@ron-rabbitmq:5672",
                           "amqp://guest:guest@ron-rabbitmq1:5672"] },
                         {declarations,
                            [{'queue.declare',
                                    [{queue, <<"shovel_out_queue">>},
                                      durable]},
                            {'exchange.declare',
                                    [{exchange, <<"shovel_exchange">>},
                                     {type, <<"direct">>},
                                     durable]},
                            {'queue.bind',
                              [{exchange, <<"shovel_exchange">>},
                               {queue, <<"shovel_out_queue">>},
                               {routing_key, <<"shovel_rk">>}]}
                            ]}]},
            {destinations, [{broker, "amqp://guest:guest@ron-rabbitmq2:5672"},
                         {declarations,
                            [{'queue.declare',
                                    [{queue, <<"shovel_in_queue">>},
                                      durable]},
                            {'exchange.declare',
                                    [{exchange, <<"shovel_exchange">>},
                                     {type, <<"direct">>},
                                      durable]},
                            {'queue.bind',
                                    [{exchange, <<"shovel_exchange">>},
                                     {queue, <<"shovel_in_queue">>},
                                     {routing_key, <<"shovel_rk">>}]}
                            ]}]},
            {queue, <<"shovel_out_queue">>},
            {ack_mode, no_ack},
            {publish_properties, [{delivery_mode, 2}]},
            {publish_fields, [{exchange, <<"shovel_exchange">>},
                            {routing_key, <<"shovel_rk">>}]},
            {reconnect_delay, 5}
            ]}
           ]
         }]
      }
].

The error I am enountering is:
** Reason for termination ==
** {{shutdown,
        {server_initiated_close,404,
            <<"NOT_FOUND - home node 'RABBIT1 at ron-rabbitmq' of durable queue 'shovel_out_queue' in vhost '/' is down or inaccessible">>}},

It seems that it can't access the queue because it doesn't exist on the source  cluster node it failed over to.   I have considered having the source queue be transient, but there is the potential to lose orphaned messages with that approach.  I have also thought about mirroring the queue, but additional overhead may be a concern due to the high volume of traffic.  Any thoughts or suggestions would be appreciated.

Ron Jaeger
Thomson Reuters
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20131004/f2c35d79/attachment.htm>