[rabbitmq-discuss] Abrupt Termination

Thu Sep 4 12:28:21 BST 2008

Tsuraan

On Thu, Sep 4, 2008 at 4:14 AM, tsuraan <tsuraan at gmail.com> wrote:
> Second post, but I figure this is probably more polite than having a
> multi-topic thread...
>
> How well do RabbitMQ persistent queues handle abrupt termination, such
> as from a power loss?

Well, "how well" is subjective.  We think it does a good job and is
properly architected for reliability, scale and speed.

> Will entire queues be corrupted, or just a
> possible loss of messages that were inserted but not yet written to
> disk?

RabbitMQ does not notify observers outside the transaction of work
that is completed until that work is properly committed.  For
persistent messages this means nothing and nobody can see them until
the transaction completes any in memory work, and any subsequent
logging has been flushed to disk from the buffer.  This is correct
behaviour.

I.e.: for an external observer, the broker losing work that has not
been committed to disk, should be no different than losing it on the
network before it got to the broker.

To answer your qn, this means
* Queues will not be corrupted
* Messages not committed to disk may be lost
* But nobody will believe a message has been committed to disk when in
fact it has not been
* Everything committed to disk is recoverable properly
* Operations complete correcty, or fail correctly

Two years ago we found a bug in the windows (but not any *nix) version
of OTP where sync() did not flush correctly which was then fixed.  As
far as we can tell this is all now as it should be.  DO PLEASE tell us
immediately if you see any other behaviour.

> If multiple messages were accepted and dispatched in a
> transaction, is the entire transaction guaranteed to be fully
> committed or fully lost in the event of system failure?

This is as above, for RabbitMQ, and correctly implemented.  A detailed
reading of the 0-8/0-9 specs reveals the interaction between TX and
mandatory/immediate to have been poorly expressed in your 'multiple
messages' case, at least with many queues.  But RabbitMQ does the
right thing.

Comments, thoughts, questions are welcomed on this subtle area!

alexis