BTW, does it make sense to use transactions in the consumer?<div><br></div><div>Davide<br><br><div class="gmail_quote">---------- Forwarded message ----------<br>From: <b class="gmail_sendername">Davide Maestroni</b> <span dir="ltr"><<a href="mailto:davide.maestroni@gmail.com">davide.maestroni@gmail.com</a>></span><br>
Date: Wed, Oct 20, 2010 at 2:58 PM<br>Subject: Re: [rabbitmq-discuss] Batch Acknowledge<br>To: Simon MacMullen <<a href="mailto:simon@rabbitmq.com">simon@rabbitmq.com</a>><br><br><br>Hi Simon,<div><br></div><div>thank you very much for the quick reply!</div>
<div>Now I have a better understanding on how the message flow works. Still I cannot find a way to achieve correct message ordering even after a crash.</div>
<div>My point here is: as soon as I read a message from the queue, the order of that message is virtually lost, even if I try immediately to write the message somewhere else. So if a crash happens after reading something, either through "basic.get" or "basic.consume", it does not matter if the acknowledge was sent or not, in any case I have no way to get the same message(s) as the first one(s) after the recovery.</div>
<div>Is there no method to just "peek" into the queue? or, can you suggest me a way to ensure ordering AND not lose any message at the same time?</div><div><br></div><div>Thanks,</div><div><br></div><font color="#888888"><div>
Davide</div></font><div><div></div><div class="h5">
<div><br><br><div class="gmail_quote">On Wed, Oct 20, 2010 at 12:15 PM, Simon MacMullen <span dir="ltr"><<a href="mailto:simon@rabbitmq.com" target="_blank">simon@rabbitmq.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>On 19/10/10 18:30, Davide Maestroni wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
What I want to achieve is a queue in which all the messages persist<br>
until a "consumer" (not sure if it is really a consumer actually) do<br>
some batch processing on N messages, like updating a separate DB, and,<br>
only when everything has completed correctly, send an acknowledge so<br>
that the queue can release the last delivered N messages.<br>
As per my understanding I can "declare" a queue and then start reading<br>
from it using basicGet() API. Then, after processing the desired number<br>
of messages, I call basicAck(deliveryTag, true) passing the tag of the<br>
last read message.<br>
In my case it is both important to keep the message ordering and not to<br>
lose any message even if the consumer or the broker crash.<br>
</blockquote>
<br></div>
Note that we only guarantee message ordering when messages are not redelivered. If your client dies with unacked messages about, those messages will be reinserted into the queue.<br>
<br>
Example: the queue contains messages ABCDEF. You client consumes ABC, does not ack them, and dies. The queue will now contain DEFABC.<div><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
In order to<br>
achieve that, I am going to publish messages "mandatory" and<br>
"persistent" and then send an ack in the consumer after successfully<br>
securing the data in some other place.<br>
What I want to be clear here is: is it correct to say that if the<br>
consumer crashes after M basicGet() (but before sending the ack), at<br>
restart the same M messages will be returned calling basicGet()? I.e.<br>
are the messages removed from the queue only after receiving the<br>
acknowledge?<br>
</blockquote>
<br></div>
They are only finally removed when the server sees the ack, but see above about ordering.<div><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
If it is not so, do I have to call basicRecover()?<br>
</blockquote>
<br></div>
No. basic.recover allows you to say "pretend I crashed". It's not 100% obvious why this is useful, but it's in the spec. Note that you don't have to call recover after recovering from crashing...<div>
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Can I do it even if I don't call basicConsume()?<br>
</blockquote>
<br></div>
Yes, the rules are exactly the same for consume vs get.<div><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
And most importantly, which is the order of<br>
the messages when calling basicRecover() after a crash and restart of<br>
the consumer node?<br>
</blockquote>
<br></div>
Hopefully explained above.<div><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
A final question: if a message is flagged as "mandatory", how long the<br>
queue wait for a consumer to be ready before returning the message to<br>
the sender?<br>
</blockquote>
<br></div>
"Mandatory" just means the message must get routed to a queue, not that anything must be listening on the queue. I think you mean "immediate". And in the immediate case, it really is immediate; there must be a consumer on the queue (no basic.get!), no messages already in the queue and the consumer must not be blocked for basic.qos or channel.flow reasons.<br>
<br>
Cheers, Simon<br>
-- <br>
Simon MacMullen<br>
Staff Engineer, RabbitMQ<br>
SpringSource, a division of VMware<br>
<br>
_______________________________________________<br>
rabbitmq-discuss mailing list<br>
<a href="mailto:rabbitmq-discuss@lists.rabbitmq.com" target="_blank">rabbitmq-discuss@lists.rabbitmq.com</a><br>
<a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a><br>
</blockquote></div><br></div>
</div></div></div><br></div>