[rabbitmq-discuss] Fwd: Batch Acknowledge

[Simon MacMullen]

For your case it doesn't make much difference. The usual use case for 
consuming with transactions is some sort of message transformer that 
needs to consume and publish atomically.

Cheers, Simon

On 20/10/10 15:56, Davide Maestroni wrote:
> BTW, does it make sense to use transactions in the consumer?
>
> Davide
>
> ---------- Forwarded message ----------
> From: *Davide Maestroni* <davide.maestroni at gmail.com
> <mailto:davide.maestroni at gmail.com>>
> Date: Wed, Oct 20, 2010 at 2:58 PM
> Subject: Re: [rabbitmq-discuss] Batch Acknowledge
> To: Simon MacMullen <simon at rabbitmq.com <mailto:simon at rabbitmq.com>>
>
>
> Hi Simon,
>
> thank you very much for the quick reply!
> Now I have a better understanding on how the message flow works. Still I
> cannot find a way to achieve correct message ordering even after a crash.
> My point here is: as soon as I read a message from the queue, the order
> of that message is virtually lost, even if I try immediately to write
> the message somewhere else. So if a crash happens after reading
> something, either through "basic.get" or "basic.consume", it does not
> matter if the acknowledge was sent or not, in any case I have no way to
> get the same message(s) as the first one(s) after the recovery.
> Is there no method to just "peek" into the queue? or, can you suggest me
> a way to ensure ordering AND not lose any message at the same time?
>
> Thanks,
>
> Davide
>
>
> On Wed, Oct 20, 2010 at 12:15 PM, Simon MacMullen <simon at rabbitmq.com
> <mailto:simon at rabbitmq.com>> wrote:
>
>     On 19/10/10 18:30, Davide Maestroni wrote:
>
>         What I want to achieve is a queue in which all the messages persist
>         until a "consumer" (not sure if it is really a consumer
>         actually)  do
>         some batch processing on N messages, like updating a separate
>         DB, and,
>         only when everything has completed correctly, send an acknowledge so
>         that the queue can release the last delivered N messages.
>         As per my understanding I can "declare" a queue and then start
>         reading
>         from it using basicGet() API. Then, after processing the desired
>         number
>         of messages, I call basicAck(deliveryTag, true) passing the tag
>         of the
>         last read message.
>         In my case it is both important to keep the message ordering and
>         not to
>         lose any message even if the consumer or the broker crash.
>
>
>     Note that we only guarantee message ordering when messages are not
>     redelivered. If your client dies with unacked messages about, those
>     messages will be reinserted into the queue.
>
>     Example: the queue contains messages ABCDEF. You client consumes
>     ABC, does not ack them, and dies. The queue will now contain DEFABC.
>
>
>         In order to
>         achieve that, I am going to publish messages "mandatory" and
>         "persistent" and then send an ack in the consumer after successfully
>         securing the data in some other place.
>         What I want to be clear here is: is it correct to say that if the
>         consumer crashes after M basicGet() (but before sending the ack), at
>         restart the same M messages will be returned calling basicGet()?
>         I.e.
>         are the messages removed from the queue only after receiving the
>         acknowledge?
>
>
>     They are only finally removed when the server sees the ack, but see
>     above about ordering.
>
>
>         If it is not so, do I have to call basicRecover()?
>
>
>     No. basic.recover allows you to say "pretend I crashed". It's not
>     100% obvious why this is useful, but it's in the spec. Note that you
>     don't have to call recover after recovering from crashing...
>
>
>         Can I do it even if I don't call basicConsume()?
>
>
>     Yes, the rules are exactly the same for consume vs get.
>
>
>         And most importantly, which is the order of
>         the messages when calling basicRecover() after a crash and
>         restart of
>         the consumer node?
>
>
>     Hopefully explained above.
>
>
>         A final question: if a message is flagged as "mandatory", how
>         long the
>         queue wait for a consumer to be ready before returning the
>         message to
>         the sender?
>
>
>     "Mandatory" just means the message must get routed to a queue, not
>     that anything must be listening on the queue. I think you mean
>     "immediate". And in the immediate case, it really is immediate;
>     there must be a consumer on the queue (no basic.get!), no messages
>     already in the queue and the consumer must not be blocked for
>     basic.qos or channel.flow reasons.
>
>     Cheers, Simon
>     --
>     Simon MacMullen
>     Staff Engineer, RabbitMQ
>     SpringSource, a division of VMware
>
>     _______________________________________________
>     rabbitmq-discuss mailing list
>     rabbitmq-discuss at lists.rabbitmq.com
>     <mailto:rabbitmq-discuss at lists.rabbitmq.com>
>     https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
>
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss


-- 
Simon MacMullen
Staff Engineer, RabbitMQ
SpringSource, a division of VMware