[rabbitmq-discuss] Lack of Ack, Failure and Re-delivery

[Simone Busoli]

Hi Paul, I'll throw my 2c too

On Wed, Jan 25, 2012 at 15:05, Bell, Paul M. <pbell at syncsort.com> wrote:

> Hello again,****
>
> ** **
>
> I’m working my way through several recovery scenarios. I think we’ve
> established the following:****
>
> ** **
>
> **a.       **If broker bounces, durable messages in the durable queue
> that have been consumed – but not acked – will be redelivered. ****
>
> **b.      **Such redelivered messages will have the “redelivered” flag
> set.****
>
> **c.       **If consumer bounces, both (a) and (b) still hold. ****
>
> ** **
>
> In re (a): is redelivery accomplished by placing another copy of the
> message in the queue (thus, two copies of same message are in the  queue –
> one consumed but not acked, the other brand new); OR does the broker simply
> know that it’s OK to redeliver the original message?****
>
> ** **
>
> In re (b): the redelivered flag seems to enable the creation of what Hohpe
> calls an “idempotent receiver.” That is, the consumer can use this flag to
> decide how to dispose of the redelivered message. Consider a scenario
> wherein the consumers, all pulling from a single queue, dispatch the work
> to a lower layer service. If there are multiple consumer nodes (even
> multiple consumer threads), they will need some means of coordinating the
> disposition of the redelivered message. For example, NodeA consumes (but
> doesn’t ack) message 1. But it does dispatch the work represented by
> message 1 to a lower layer service. If the broker bounces and message 1 is
> redelivered, it might be consumed by NodeB. But the message 1 task is
> already in-flight on NodeA. So, depending on both the idempotency of the
> task and application specific functionality, NodeB might want to discard
> the redelivered message 1. Such intra-consumer/intra-consumer node
> coordination suggests that the first consumer of the message needs to
> record something, probably in the persistence layer, that says “I’ve
> started this particular task (using a unique request ID). Does this seem
> right?
>

Yes, according to my experience message deduplication is something that you
need to carry out at the application level as it depends on how you consume
your messages within your application. The broker is only able to make sure
that messages reached a consumer in the application, what the application
does with the messages is completely up to the application itself.


> ****
>
> ** **
>
> In re (c): does redelivery after consumer bounce work because the broker
> detects the loss of connection?
>

That's just one of the scenarios in which messages are redelivered. Take
this with a grain of salt as I'm not completely sure of all the cases in
which this happens with these exact semantics but in general whenever
messages are not acked and, loosely speaking, a consumer goes away and a
new one comes on the same queue, those messages will be redelivered.

Off the top of my mind:

   - connection consumer-broker bounces
   - broker bounces (with durable ex and queues and persistent messages)
   - consumer is cancelled and another basic consume is issued
   - consumer channel is closed an another channel with a new consumer on
   the same queue is created
   - consumer issues a recover command to the broker
   - consumer rejects or nacks messages with requeue flag



>
> ****
>
> Please confirm, clarify, or correct as you see fit!****
>
> ** **
>
> Thanks for your help.****
>
> ** **
>
> -Paul****
>
> ** **
>
> *From:* rabbitmq-discuss-bounces at lists.rabbitmq.com [mailto:
> rabbitmq-discuss-bounces at lists.rabbitmq.com] *On Behalf Of *Bell, Paul M.
> *Sent:* Saturday, January 21, 2012 5:39 PM
> *To:* Simone Busoli
>
> *Cc:* rabbitmq-discuss at lists.rabbitmq.com
> *Subject:* Re: [rabbitmq-discuss] Lack of Ack, Failure and Re-delivery****
>
> ** **
>
> Thank you both.****
>
> ** **
>
> -Paul
>
> On Jan 21, 2012, at 5:24 PM, "Simone Busoli" <simone.busoli at gmail.com>
> wrote:****
>
> Also in case 2 the broker will redeliver all messages that were not acked,
> although it doesn't necessarily mean that the consumer didn't really
> consume them already. For that you should look at the redelivered flag.***
> *
>
> On Jan 21, 2012 11:20 PM, "Bell, Paul M." <pbell at syncsort.com> wrote:****
>
> I apologize. I should have stated up front that the queues and messages
> are durable.
>
> What happens in case 2, where the consumer bounces?
>
> Thanks, Jason.
>
> -Paul
>
> ________________________________________
> From: Jason J. W. Williams [jasonjwwilliams at gmail.com]
> Sent: Saturday, January 21, 2012 5:05 PM
> To: Bell, Paul M.
> Cc: rabbitmq-discuss at lists.rabbitmq.com
> Subject: Re: [rabbitmq-discuss] Lack of Ack, Failure and Re-delivery
>
> Hi Paul,
>
> If the messages were published with the durable flag and the queues are
> durable, the messages will survive the broker restart and be re-presented
> to consumers. If the messages or the queues were not durable they will
> disappear after restart.
>
> -J
>
> Sent via iPhone
>
> Is your email Premiere?
>
> On Jan 21, 2012, at 14:57, "Bell, Paul M." <pbell at syncsort.com> wrote:
>
> > Hi,
> >
> > Suppose I have a producer on NodeP, a broker on NodeB, and a consumer on
> NodeC. Suppose further that explicit acks are required (i.e., if I've
> understood the docs, that "noack" is not in effect) and that after
> consuming a message from the queue, NodeC.consumer dispatches some other
> work in order to process that message. IOW: NodeC.consumer doesn't respond
> immediately with an ack.
> >
> > So, from NodeB.broker's point of view, the message has been moved from
> the exchange to the queue, consumed, but not yet ack-ed.
> >
> > What will happen under these two scenarios:
> >
> > 1. NodeB.broker bounces - when broker restarts what will he do with
> un-acked messages that have been delivered to queue and haven't been
> ack-ed; e.g., will these be again delivered to the queue?
> >
> >
> > 2. NodeC.consumer bounces - when consumer restarts is there any way he
> can again consume the un-acked message, and begin again the work that this
> message represents? (I suppose that NodeB.broker might be implicated here
> because it might detect the loss of connection to the NodeC.consumer....).
> >
> >
> > When I say "bounces," I mean at a minimum that the application (broker,
> consumer) crashed and restarted. But I am also interested in the case where
> the node's OS is for whatever reason rebooted. Perhaps it looks no
> different.
> >
> > Thanks for your help.
> >
> > -Paul
> >
> >
> >
> > ATTENTION: -----
> >
> > The information contained in this message (including any files
> transmitted with this message) may contain proprietary, trade secret or
> other  confidential and/or legally privileged information. Any pricing
> information contained in this message or in any files transmitted with this
> message is always confidential and cannot be shared with any third parties
> without prior written approval from Syncsort. This message is intended to
> be read only by the individual or entity to whom it is addressed or by
> their designee. If the reader of this message is not the intended
> recipient, you are on notice that any use, disclosure, copying or
> distribution of this message, in any form, is strictly prohibited. If you
> have received this message in error, please immediately notify the sender
> and/or Syncsort and destroy all copies of this message in your possession,
> custody or control.
> > _______________________________________________
> > rabbitmq-discuss mailing list
> > rabbitmq-discuss at lists.rabbitmq.com
> > https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss****
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120125/357e7360/attachment.htm>