[rabbitmq-discuss] How to handle 'RabbitMQ cluster unaccessible' issues? Is using shovels the best solution?

[Tim Watson]

Hi

On 02/04/2013 07:54 AM, Arizanov wrote:
> Hi guys,
>
> We are doing some realtime replication of critical data and there may 
> be cases when the RabbitMQ cluster is unaccessible. E.g. the cluster 
> is being patched or there just a network connectivity issue. On one 
> side we can have local rabbitmq installations inside the webserver and 
> shovels, which transmit the data to the cluster. On the other side, we 
> can develop our own database and a service over it, which is scheduled 
> to transmit the messages to the cluster when it becomes available. So 
> is using the 'shovel' the best practise? Does the statement
> "A shovel behaves like a well-written client application, which 
> connects to its source and destination, reads and writes messages, and 
> copes with connection failures." from 
> http://www.rabbitmq.com/shovel.html means so?
>

Reading from ...,

"Whereas federation aims to mirror messages from one exchange to another 
(with optimisations to ensure that messages are not forwarded unless 
needed), the shovel simply consumes messages from a queue on one broker, 
and forwards them to an exchange on another. "

So with the Shovel, because you "Choose Availability and Partition 
Tolerance from the CAP theorem 
<http://en.wikipedia.org/wiki/CAP_theorem>", there is the possibility 
that you'll end up with duplicate messages. You can tweak the 
'republish' semantics to compensate for this if you like, and of course 
yes the Shovel will attempt to reconnect (and this is also configurable) 
on connection failure.

You might want to consider Federation if you don't need to low level of 
control that the Shovel offers.

Cheers,
Tim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130204/68b82885/attachment.htm>