[rabbitmq-discuss] Batch Acknowledge

Wed Oct 20 11:15:11 BST 2010

On 19/10/10 18:30, Davide Maestroni wrote:
> What I want to achieve is a queue in which all the messages persist
> until a "consumer" (not sure if it is really a consumer actually)  do
> some batch processing on N messages, like updating a separate DB, and,
> only when everything has completed correctly, send an acknowledge so
> that the queue can release the last delivered N messages.
> As per my understanding I can "declare" a queue and then start reading
> from it using basicGet() API. Then, after processing the desired number
> of messages, I call basicAck(deliveryTag, true) passing the tag of the
> last read message.
> In my case it is both important to keep the message ordering and not to
> lose any message even if the consumer or the broker crash.

Note that we only guarantee message ordering when messages are not 
redelivered. If your client dies with unacked messages about, those 
messages will be reinserted into the queue.

Example: the queue contains messages ABCDEF. You client consumes ABC, 
does not ack them, and dies. The queue will now contain DEFABC.

> In order to
> achieve that, I am going to publish messages "mandatory" and
> "persistent" and then send an ack in the consumer after successfully
> securing the data in some other place.
> What I want to be clear here is: is it correct to say that if the
> consumer crashes after M basicGet() (but before sending the ack), at
> restart the same M messages will be returned calling basicGet()? I.e.
> are the messages removed from the queue only after receiving the
> acknowledge?

They are only finally removed when the server sees the ack, but see 
above about ordering.

> If it is not so, do I have to call basicRecover()?

No. basic.recover allows you to say "pretend I crashed". It's not 100% 
obvious why this is useful, but it's in the spec. Note that you don't 
have to call recover after recovering from crashing...

> Can I do it even if I don't call basicConsume()?

Yes, the rules are exactly the same for consume vs get.

> And most importantly, which is the order of
> the messages when calling basicRecover() after a crash and restart of
> the consumer node?

Hopefully explained above.

> A final question: if a message is flagged as "mandatory", how long the
> queue wait for a consumer to be ready before returning the message to
> the sender?

"Mandatory" just means the message must get routed to a queue, not that 
anything must be listening on the queue. I think you mean "immediate". 
And in the immediate case, it really is immediate; there must be a 
consumer on the queue (no basic.get!), no messages already in the queue 
and the consumer must not be blocked for basic.qos or channel.flow reasons.

Cheers, Simon
-- 
Simon MacMullen
Staff Engineer, RabbitMQ
SpringSource, a division of VMware