[rabbitmq-discuss] Message durability and partition tolerence

[Jonathan Oliver]

Since the introduction of better clustering support and message queue 
failover in v2.6, my company is starting to consider moving to RabbitMQ. 
 But we had a few questions about the roadmap.

We are looking to ensure that a message is only considered received by an 
exchange from a producing app when at least two physical instances of 
Rabbit acknowledge the receipt of the message--similar to Riak's method. 
 Furthermore, we are looking to ensure that a message is considered 
consumed (ack'd) when when at least one physical instance receives the ack, 
which may result in a message being delivered more than once.  Essentially, 
we'd like to propose a configuration where Rabbit can fall on the AP side 
of CAP instead of the CA side.

I should mention that we're fully aware that the above scenario may cause a 
message to be pushed to consumers more than once and how messages may get 
out of order.  In general, however, these concepts are very much a part of 
message queuing system guarantees--that of *at-least-once* delivery where 
order is not strictly guaranteed.  If we needed *at-most-once* delivery and 
strict ordering, we'd use RPC.  In many regards, this is similar to the 
Amazon SQS architecture with a few key differences--one of the most 
important being stateful connections and "push" capabilities between the 
server and connected consumers which is where Amazon's SQS offering falls 
apart--that and message size restrictions.

I can see that many systems really want to ensure at-most-once delivery (or 
close to it) and how the requirement to re-order messages makes things 
difficult, but by relaxing the constraints (at least in some 
configurations), it enables so much more.  For example, if an instance of 
Rabbit goes offline under the current HA configuration, when it rejoins the 
cluster the queues are wiped clean.  Yuck.  So much for guaranteed 
delivery.  While under some kind of partition-tolerant configuration, the 
queues wouldn't need to be cleaned.  Instead, the consuming application 
would be responsible for message idempotency.

At the same time, I realize that you guys might be bumping against 
restrictions in the AMQP standard, for example multiple consumers receiving 
copies of the message where one consumer ack's the message, while another 
nack's the message.  But that's something where the messaging system can't 
determine the proper course of action without knowing the business context. 
 In that case, I would imaging it should present that nack back to the 
producer as it currently does.

The bottom line in my question and proposal really has to do with message 
loss.  If the idea is to *not lose messages*, I can see a 
partition-tolerant configuration bringing a lot to the table and where the 
consuming application must be resilient to duplicate messages and ordering, 
thus freeing up RabbitMQ from having to worry so much about that.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20111108/08c1f831/attachment.htm>