[rabbitmq-discuss] Option for surviving connection failure feature suggestion

[Wayne Brantley]

Disclaimer:  I am new to RabbitMQ.  :-)

When processing long running messages, if the connection drops or gets 
interrupted - the message will be handed to another consumer.
Meanwhile, the original consumer is still doing the work.

I would propose a new option on a queue.

If the connection is dropped, that would not make the message available to 
another consumer.
The message would have a 'processing timeout'.  Only after that time has 
expired would the message become available to other consumers.

There would be several different options that could be set on the message 
for how to determine to deliver the message to another consumer.

1)  You could have an 'absolute processing timeout'.  
    -  RabbitMQ would keep track of how long the consumer has had the 
message.
    -  The client could send 'keep alive' acknowledgements to reset the the 
time RabbitMQ is tracking to zero.
    -  All that matters is has the message been processing longer than the 
timeout value.  (where timeout can be reset by client).  A dropped 
connection would not matter. 

2)  You could have a 'broken connection timeout'
    -  RabbitMQ would keep track of how long the consumer has had the 
message, AFTER the connection is dropped.
    -  The client would NOT ever need to send 'keep alive' acknowledgements.
    -  If a consumer connection is re-established - the consumer should 
somehow indicate they are still working on the message.
    -  Potentially - when the consumer is working on the message, they send 
'keep alive' every so often.  This way if the connection broke and was 
reconnected, the next time a 'keep alive' is sent, it would let RabbitMQ 
know you are still working on the message.

The above options would probably have default values at the queue level, 
and each message could override (for example if you know the message takes 
a long time to process.)

This makes it where any intermittent connection from consumer to publisher 
would not necessarily mean the consumer 'failed' and the message should be 
passed to another consumer.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20131205/95630e1e/attachment.html>