[rabbitmq-discuss] Exactly Once Delivery

Tue Aug 10 08:52:08 BST 2010

Alexis Richardson wrote:

> We've often found customers whose requirements include "please break
> the laws of physics, and cure cancer".  I'm sure you know what I mean.
>  If you could advise us on how to best help in such cases, we're all
> ears.
> 
> In the meantime we are stuck in world where "guaranteed" has no fixed
> connotation.

Yes. People tend to have unrealistic expectations.

It kind of reminds me of requirements for "security". Yes, application 
can be made more resistant to attacks, however, "security" is not a 
purely technical issue. It's a technical and _operational_ issue. To 
achieve "security" you have to use good software _and_ introduce a sane 
security policy in your organisation.

Same with guaranteed delivery. While applications can do all kinds of 
tricks to improve reliability at some point you'll have to load the data 
to USB key and ride across the country to deliver it.

So the goal of designing "guaranteed delivery" is twofold IMO. First, 
it's doing all the tweaking of the software necessary to get as much 
nines of reliability as possible. Second, its promoting sane 
organisational patterns (patterns that would prevent message lost or 
duplication).

Let me give a simple example:

1. Messages are confirmed by simple ack. There's no replay 
functionality. This guarantees no duplicates.
2. Sender has a timeout for getting an ack. When the ack does not 
arrive, the message is moved to dead letter queue. (Note that messages 
in the DLQ are "dubious", i.e. they may have been delivered or not, we 
don't know).
3. There's a person in the organisation responsible for the dead letter 
queue. He goes through the queue once a day and tries to find out what's 
the actual state of the dubious messages is (using personal 
conversation, phone, checking the production system etc.) Delivered 
messages can be simply deleted from the DLQ. Lost messages can be 
resubmitted in different ways (doing a new transaction, reading the 
details of the transaction over the phone etc.)

My 2c.
Martin