Hi Simone,<br><br><font face="'trebuchet ms', sans-serif">| Can you describe why you think publisher confirms are not appropriate in your scenario?<br><br>I think if you read the paragraph that starts with "At the heart of things" it describes the scenario in enough detail. In a follow up, Simon MacMullen appears to agree that publisher confirms aren't ideal for us.<br>
<br>In short, when we publish a message, we must know that the broker safely has it. With transactions, OR a blocking publisher-confirm model, the client is assured it has it.<br><br>HOWEVER, all the publisher-confirm descriptions/examples I've seen assume an async model: Publish a message and some time later you'll be told that the message is safely in the broker's hands. However, that window between those two events is the crux of the problem in our environment.<br>
<br>Assume we went with the async publisher-confirm model as suggested. What if our client dies and is not around to re-send the message? Or what if the client dies, and milliseconds later the broker dies? No publisher confirm would be forthcoming, and hence, lost messages.<br>
<br>Top level problem for us: Our state machine, which persists every critical change of state to a database, will be incorrect. We will have recorded "Yup, that message was sent", when in reality it might not have made it to broker, or the broker died before the confirm could be sent.<br>
</font><font face="'trebuchet ms', sans-serif"><br>By not recording that we've sent the message until AFTER a transaction/publisher-confirm completes, we can guarantee the integrity of our state-machine. However, as we've all seen, transactions impose a limit on message throughput. </font><font face="'trebuchet ms', sans-serif"> I'm simply trying to understand
if there's any non-trivial message rate difference expected between
using a transaction vs. waiting in my publish routine for the confirm.</font><br><font face="'trebuchet ms', sans-serif"><br><br>| If you happen to be using the
.NET API this is already implemented, the WaitForConfirms and
WaitForConfirmsOrDie on the IModel interface provide this behavior.<br><br>Unfortunately, I'm not on a .NET client. I'm using Python/Pika. I'd consider going to the c-library (wrapped from Python) if there's some non-trivial benefit to using a synchronous publisher-confirm.<br>
<br>Thanks,<br><br>Matt<br></font><br><div class="gmail_quote">On Thu, Feb 16, 2012 at 1:08 PM, Simone Busoli <span dir="ltr"><<a href="mailto:simone.busoli@gmail.com">simone.busoli@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<font face="trebuchet ms,sans-serif">Hi Matt, I've gone through the old thread very quickly so forgive me if I'm missing something. Your first sentence leaves me with some doubts:<br></font><br><div class="gmail_quote">
<div class="im">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="font-size:14px;font-family:Calibri,sans-serif;word-wrap:break-word"><div>As I've detailed in other posts to this DL (<a href="http://bit.ly/zf6sKC" target="_blank">http://bit.ly/zf6sKC</a>), we have situation where straightforward publisher-confirms aren't appropriate in our scenario. We need to know the broker has received the message, and we have to assume our client can die at any moment, so retransmitting messages is not an option for us.</div>
</div></blockquote><div><br></div></div><div><font face="'trebuchet ms', sans-serif">Publisher confirms guarantee that the broker has taken care of the message and it has been either delivered to a queue or to a consumer. This is a strong delivery guarantee, even if the client dies. Coupled with persistent messages and durable exchange/queues, it guarantees at-least-once delivery. Can you describe why you think publisher confirms are not appropriate in your scenario?</font></div>
<div class="im">
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="font-size:14px;font-family:Calibri,sans-serif;word-wrap:break-word"><div></div><div>What I'd like to know is: Is there any benefit to creating our own "blocking publisher-confirm publishing"? That is, we'd do the following:</div>
<div><br></div><div>Publish a single message</div><div>Block on an event 'X'</div><div>When the publisher confirm comes in, signal 'X'</div><div>Return from the publish call</div></div></blockquote><div><br>
</div></div><div><font face="'trebuchet ms', sans-serif">If you happen to be using the .NET API this is already implemented, the WaitForConfirms and WaitForConfirmsOrDie on the IModel interface provide this behavior.</font></div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im"><div style="font-size:14px;font-family:Calibri,sans-serif;word-wrap:break-word"><div>In a mirrored queue scenario, is there any advantage to doing this vs. using straight-up transactions?</div>
<span><font color="#888888"><div><br></div><div>Matt</div></font></span><div><br></div><div>p.s. I realize in the above pseudocode that multiple threads would be necessary - One to block and the other to listen for the publisher-confirm.</div>
<div><br></div></div>
<br></div>_______________________________________________<br>
rabbitmq-discuss mailing list<br>
<a href="mailto:rabbitmq-discuss@lists.rabbitmq.com" target="_blank">rabbitmq-discuss@lists.rabbitmq.com</a><br>
<a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a><br>
<br></blockquote></div><br>
</blockquote></div><br>