[rabbitmq-discuss] Exactly Once Delivery

Sat Aug 7 22:25:27 BST 2010

On Sat, Aug 07, 2010 at 08:46:50PM +0100, Matthias Radestock wrote:
> I believe Matthew was simply trying to point out that many of the
> supposed guarantees of messaging systems are a lot softer than most
> people think.

Well, in my mind, if you "guarantee" something, then you can offer a
proof in some branch of maths that the property you are guaranteeing can
never be violated except in the circumstances you qualify the statement
with.

That is why, as far as I'm concerned, no messaging system, regardless of
whether it's implemented in a computer or not, can ever guarantee
exactly once semantics: I can prove (or rather, other people have
proven) that provided any individual message can be lost, you can either
achieve "at least once" semantics or "at most once" semantics.

Quantum mechanics may offer a way to avoid messages ever being lost, but
I suspect that's probably not going to be sorted out in time for
RabbitMQ 3.0.

> In reality a "guarantee" is little more than an
> increase in the probability that the right thing will happen.

Indeed. And importantly, under different circumstances, that probability
can vary wildly. Once you start combining different systems, and the
error rates compound, the probability that the whole thing works end to
end can end up looking very shaky. I suspect the insurance industry will
do well here.

> > What has not been touched on in this little discussion so far is
> > the question of transactions. .. Updating DBs from queues and
> > posting the results of those updates to queues should be atomic; and
> > if I want my debit/credit to happen once rather than many times or
> > not at all, then a combination of transactions and guaranteed
> > delivery becomes very attractive both to the designer and the
> > developers. Yes, ACID comes to mind here...and it is indeed what I
> > am referring to.

Distributed transactions rely on 2-phase or 3-phase commit or some
variant thereof (eg there's an improved 3-phase build on paxos). Whilst
the protocols themselves are sound, my understanding is (and please
correct me if I'm wrong) that they all rely on, once everyone's agreed
there is no problem committing the transaction, no participant will
renege on that promise. But of course, no participant can actually
guarantee that. Thus distributed transactions can never guarantee ACID.

All you can get is increasing probability that the outcome will be
atomic, consistent and isolated. But you can never guarantee it.

Matthew