[rabbitmq-discuss] Consumer crash, redelivery and prefetch

[Karl Nilsson]

Hi Thomas,

It is a great shame that a mature message broker such as RabbitMQ is so
lacking in sensible poison message handling (or any strategies regarding
redelivery). I raised the issue here a few months ago and was advised the
priority of planned feature [basic / deliver / 01] had been raised but
wasn't given any indication of when it might be delivered.

http://comments.gmane.org/gmane.comp.networking.rabbitmq.general/27733

After that I wrote a blog entry summarising my investigations which
includes some of the suggested workarounds (such as using multiple queues),
none of which I consider adequate.

http://kjnilsson.github.io/blog/2014/01/30/spread-the-poison/

There is an argument for not designing systems where losing a message is a
big deal but I think that is a topic of a different discussion. :)

Cheers
Karl


On 13 March 2014 19:31, Thomas Riccardi <riccardi at systran.fr> wrote:

> Consumer crash, redelivery and prefetch
>
> Hi,
>
> I'm having issues with consumer crash, redelivery and prefetch:
>
> The messages ack strategy I've chosen is to ack the message only after
> the consumer has finished working on the message. I need this because I
> cannot aford any lost message, even in case of consumer crash during the
> work on the message.
>
> For performance reason my consumer have a non 0 prefetch, so when my
> consumer is handling a message (doing some work related to the message),
> other messages are pushed to the client so that it has them already
> available when it's done working on the previous message.
>
> With this, when a message makes the consumer crash, the handled message
> *and* the prefetched ones are re-queued, with the flag "redelivered" ==
> 1.
>
> I currently have no other choice than rejecting all messages that have
> the "redelivered" flag set to true, because I have no way to distinguish
> the message that cause the crash from the prefetched ones that did
> nothing wrong.
> Indeed if I retry to work on a redelivered message, the consumer will
> crash again on the poison message, in an infinite loop. However I would
> like to not reject the previously prefetched messages since they are
> probably OK.
> This reasoning also applies to transient crashes that are not
> systematically reproduced by the message.
>
>
> Ideally the solution would be to be able to distinguish the first non
> ack'd message from the other ones when the consumer crashes, or
> alternatively having two types of ack : one "I'm starting to work on
> this message", and a second one "I've finished working on this message".
>
> Otherwise a redelivery count would do the trick (even if it means
> working multiple times on a message that did make the consumer crash,
> which is not efficient if we know crashes happen mostly systematically).
>
> The redelivery count is a "planned" feature according to
> http://www.rabbitmq.com/specification.html, but when will it be really
> implemented?
> Having a parameter per queue like the TTL to tell a maximum redelivery
> count before dead lettering the message would be much helpful in my case
> (a RPC above rabbitmq, which is probably not a highly specific and rare
> use-case).
>
>
> Thanks,
> Thomas
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>



-- 
*Karl Nilsson*
twitter: @kjnilsson
blog: coderkarl.wordpress.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20140314/df2237f7/attachment.html>