[rabbitmq-discuss] Exactly Once Delivery

[Mike Petrusis]

Greetings,

In reviewing the mailing list archives, I see various threads which state that ensuring "exactly once" delivery requires deduplication by the consumer.  For example the following:

"Exactly-once requires coordination between consumers, or idempotency,
even when there is just a single queue. The consumer, broker or network
may die during the transmission of the ack for a message, thus causing
retransmission of the message (which the consumer has already seen and
processed) at a later point."  http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2009-July/004237.html

In the case of competing consumers which pull messages from the same queue, this will require some sort of shared state between consumers to de-duplicate messages (assuming the consumers are not idempotent).   

Our application is using RabbitMQ to distribute tasks across multiple workers residing on different servers, this adds to the cost of sharing state between the workers. 

Another message in the email archive mentions that "You can guarantee exactly-once delivery if you use transactions, durable queues and exchanges, and persistent messages, but only as long as any failing node eventually recovers."

>From the way I understand it, the transaction only affects the publishing of the message into RabbitMQ and prevents the message from being queued until the transaction is committed.  If this is correct, I don't understand how the transaction will prevent a duplicate message in the previously mentioned scenarios that will cause a retransmission.  Can anybody clarify?  

On a more practical level:

What's the recommended way to deal with the potential of duplicate messages?  
What do people generally do? 
Is this a rare enough edge case that most people just ignore it?


Thanks,

Mike