[rabbitmq-discuss] Backing up persisted messages

[Beth Jones]

I'll have a look at exchange federation - thanks for the suggestion.

I'm surprised that this isn't something that's supported out of the box - is it on the TODO list?

Beth


On 31 Jan 2014, at 09:32, Alvaro Videla wrote:

> Hi,
> 
> What if you replicate the messages to a node on different
> administrative domain, using exchange federation?
> http://www.rabbitmq.com/federation.html
> 
> That way you can prevent people from connecting there and deleting messages.
> 
> Regards,
> 
> Alvaro
> 
> On Fri, Jan 31, 2014 at 10:24 AM, Beth Jones <bj2 at sanger.ac.uk> wrote:
>> The scenario I'm looking at, is someone accidentally / inadvertently / deliberately deleting the messages from the queue.  I want to be able to take a snapshot of the messages that are in a particular queue and put them somewhere else on disk outside of RabbitMQ.  Then if someone/something does delete the messages, I want to be able to restore them.
>> 
>> Having the messages replicated on multiple nodes goes someway towards mitigating the risk, but it doesn't stop someone accidentally clearing out multiple queues.
>> 
>> Beth
>> 
>> 
>> On 30 Jan 2014, at 18:34, Tim Watson wrote:
>> 
>>> On 30 Jan 2014, at 17:58, Beth Jones wrote:
>>>> Specifically I have a queue that is accumulating messages and I'm worried about those messages being deleted accidentally.  I want to be able to back up that queue on a regular basis (say daily or hourly) to a location on disk.
>>>> 
>>>> I read about copying the mnesia directory.  Does that copy all the data for all of the queue on that node?  Is there something similar that is queue specific?
>>>> 
>>> 
>>> The queue specific thing would be to cluster two or more RabbitMQ brokers and set a High Availability policy on the queue so that all its messages are replicated to the other node(s). If the primary node crashes, one of the replicas will take over and you won't loose any messages (that have already been written to disk). Take a look at http://www.rabbitmq.com/ha.html for more details.
>>> 
>>> If your network is potentially unreliable (i.e., network partitions are quite likely) then you might want to consider using a federated exchange instead. This will perform the replication for you (i.e., copy messages from the exchange to another node(s) as per your configuration/policy) and the receiving node will effectively have a backup of all the data sent to the original exchange. I'd go for HA if you've got a reliable network though, since for your use-case it is ideal.
>>> 
>>> Cheers,
>>> Tim
>>> _______________________________________________
>>> rabbitmq-discuss mailing list
>>> rabbitmq-discuss at lists.rabbitmq.com
>>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>> 
>> 
>> 
>> --
>> The Wellcome Trust Sanger Institute is operated by Genome Research
>> Limited, a charity registered in England with number 1021457 and a
>> company registered in England with number 2742969, whose registered
>> office is 215 Euston Road, London, NW1 2BE.
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.rabbitmq.com
>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss



-- 
 The Wellcome Trust Sanger Institute is operated by Genome Research 
 Limited, a charity registered in England with number 1021457 and a 
 company registered in England with number 2742969, whose registered 
 office is 215 Euston Road, London, NW1 2BE.