[rabbitmq-discuss] Shovel plugin behavior with durable queues...

[Sean Heaton]

Steve,

Ah!  That was it!  Thank you very much for that and sorry for my ignorance.  When I had all the machines up and the messages made it to Machine A, if I restarted it, all of the messages were still in place, so I assumed that the messages were persistant.  As you indicated, this was not the case.  Once I added the delivery_mode setting you suggested to my publish call, I was able to restart Machine C and the messages were still present.  Apply palm to forehead and problem solved.

Thank you again for your help with this.

Regards,
Sean

From: Steve Powell [mailto:steve at rabbitmq.com]
Sent: Wednesday, April 04, 2012 7:25 AM
To: Sean Heaton
Cc: rabbitmq-discuss at lists.rabbitmq.com
Subject: Re: [rabbitmq-discuss] Shovel plugin behavior with durable queues...

Sean,

Please check that the messages are 'persistent' messages (published with
BasicProperties containing delivery_mode=2). If not they won't survive a
restart; only persistent messages in durable queues survive a restart.

I note that you have set persistent in the shovel configuration, but
this only overrides the properties of the original message. If they
weren't persistent originally and shovel doesn't re-publish them, they
will be lost on restart.

Steve Powell  (a happy bunny)
----------some more definitions from the SPD----------
chinchilla (n.) Cooling device for the lower jaw.
socialcast (n.) Someone to whom everyone is speaking but nobody likes.
literacy (n.) A textually transmitted disease usually contracted in childhood.

On 3 Apr 2012, at 17:46, Sean Heaton wrote:


Steve,

Thank you very much for helping me out.  I actually stop Machine A.  That prevents messages from being shoveled from Machine C  to A.  While A is still stopped, I restart the RabbitMQ service Machine C.

When I do this, I figured since the messages are in a durable queue on Machine C, they would still be in the durable queue on C waiting to be shoveled to A.  This isn't the case though-they are simply gone.  They don't make it to A (since A is stopped), but they aren't on C anymore either.  I was thinking I probably have something configured wrong or that there is a setting I can tweak to make them persist on C.

Thank you again for your help on this...

Regards,
Sean


From: Steve Powell [mailto:steve at rabbitmq.com]
Sent: Tuesday, April 03, 2012 7:44 AM
To: Sean Heaton
Cc: rabbitmq-discuss at lists.rabbitmq.com<mailto:rabbitmq-discuss at lists.rabbitmq.com>
Subject: Re: [rabbitmq-discuss] Shovel plugin behavior with durable queues...

Hi Sean,

Did you mean to say you restarted Machine C or Machine A?

In any case, the messages in a durable queue will only survive a server
stop/start if the messages are persistent.

If you restarted Machine A, then the shovel ought to reconnect (from C
to A) and the messages in the queue should then be sent. Since the queue
on Machine A is durable it ought to still exist when you restart it.

Hope this gives some clue. If you still have problems, please come back.

Steve Powell  (a happy bunny)
----------some more definitions from the SPD----------
chinchilla (n.) Cooling device for the lower jaw.
socialcast (n.) Someone to whom everyone is speaking but nobody likes.
literacy (n.) A textually transmitted disease usually contracted in childhood.

On 2 Apr 2012, at 22:33, Sean Heaton wrote:



Hello,

I am new to the list and RabbitMQ in general, so I hope this is not a stupid question, but I have seen some behavior that is strange to me when using the RabbitMQ shovel plugin.  I am using RabbitMQ v 2.7.1 running on Centos 6.0.  I have three machines in my setup.  Machines A and B are both clustered together.  Machine C is not part of the cluster.  It has the shovel plugin installed.  The plugin is set up to shovel messages from a durable queue on Machine C to a durable queue on Machine A.  In case it helps, my /etc/rabbitmq/rabbitmq.conf file looks like this:

[{rabbitmq_shovel,
  [{shovels,
    [{myShovel,
      [{sources,      [{broker,"amqp://MachineC"}]},
       {destinations, [{broker, "amqp://MachineA"}]},
       {queue, <<"myQueue">>},
       {ack_mode, on_confirm},
       {publish_properties, [{delivery_mode, 2}]},
       {reconnect_delay, 5}
      ]
    }]
 }]
}].

With this configuration, everything seems to work fine.  When I send messages to the queue on Machine C, they are shoveled automatically out to A.

So here is the weird behavior I mentioned earlier...  If I shut down the RabbitMQ service on Machine A, messages stack up in the durable queue on Machine C.  This is expected since Machine A is down.  However, if I then restart the RabbitMQ service on Machine C, all of the messages waiting in the durable queue get nuked.

Has anyone else seen this behavior?  Do I have a setting wrong or is there something else I can change so that this doesn't happen?

Thanks in advance for your help!

Regards,
Sean

_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss at lists.rabbitmq.com<mailto:rabbitmq-discuss at lists.rabbitmq.com>
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120404/2406f30e/attachment.htm>