Thanks, Emile, for your explanation. That was helpful.<div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
An alternative approach would be to declare a return queue for the consumers to report their progress to the publisher. The publisher can subscribe to this queue and use this to wait until the subscribers are ready before proceeding. The publisher would need to know how many subscribers to expect replies from.<br>
</blockquote><div><br></div><div>This is one of the main reasons I wanted to use pub/sub. I don't want the publisher to have to know how many subscribers exist.</div><div>�</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
If your design allows it you should avoid coupling producers and consumers as tightly as you seem to need here. What is the reason for forcing producers and consumers in lockstep? </blockquote><div><br></div><div>Here's a quick explanation of my design. I have a system with multiple machines, each with several worker threads. Each worker takes tasks from a shared work queue one at a time, does that task and returns for another task. That part of the system already works well. The new part I am trying to create is a control system. The control system will send pause and resume messages to a worker manager that runs on each worker machine. The worker manager is the subscriber. I would like the worker manager to wait until all of its workers are paused before acknowledging the pause message to the publisher. Sometimes pausing a worker can take up to a minute.�</div>
<div><br></div><div>Since each worker manager will wait to send its ack to the publisher until all of its workers are paused, the publisher will know that all of the workers throughout the system are paused and can safely continue. Later the publisher will send a resume message to all of the worker managers and work can resume.</div>
<div><br></div><div>Does this seem like a reasonable design? In the future it would probably be nice to have a system that has more holistic view of its workers, but for now I am not concerned if the publisher sends messages to one machine or 100.</div>
<div>�</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Can the same aim be achieved in another way by taking a larger view of the system?</blockquote>
<div><br></div><div>I'm not sure. Do you have any suggestions?</div><div><br></div><div>Thanks,</div><div>Nick</div><div><br><br><div class="gmail_quote">On Thu, Nov 1, 2012 at 3:42 AM, Emile Joubert <span dir="ltr"><<a href="mailto:emile@rabbitmq.com" target="_blank">emile@rabbitmq.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
Hi Nick,<br>
<br>
This page has more detail and you may find some of your answers:<br>
<a href="http://www.rabbitmq.com/confirms.html" target="_blank">http://www.rabbitmq.com/<u></u>confirms.html</a><div class="im"><br>
<br>
On 31/10/12 22:10, Nick Martin wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Wait until all messages published since the last call have been<br>
either ack'd or nack'd by the broker.<br>
</blockquote>
<br></div>
A message being ack'd by the broker is different from being ack'd by a consumer. If the broker acknowledges a messages it means that the broker has taken responsibility for it by writing it to disk or successfully handed it off. Handing off could mean returning the messages to the publisher (unroutable mandatory msg) or successfully delivering to consumer. The publisher cannot choose how the broker discharges this responsibility. In your case you would like a broker acknowledgement only when the message is successfully delivered to a consumer, but the broker may decide to write the message to disk and acknowledge when the OS guarantees that the disk write succeeded.<div class="im">
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
It seems to me that the publisher is<br>
supposed to block until all of the messages are ack'd via its call to<br>
waitForConfirmsOrDie(). Is this what this is really supposed to do?<br>
</blockquote>
<br></div>
Yes, waitForConfirmsOrDie() will block until an ack is received from the broker.<div class="im"><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
This example code seemed like it matched up perfectly with what I was<br>
trying to do. But, it doesn't seem to work the way I thought it did. In<br>
fact, if, in the consumer thread, I turn off auto-acking messages, then<br>
waitForConfirmsOrDie() still returns immediately.<br>
</blockquote>
<br></div>
This is because the broker has taken responsibility for the message by writing it to disk.<div class="im"><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
So what does waitForConfirmsOrDie() actually do? When would it block?<br>
</blockquote>
<br></div>
It could block e.g. as a result of attempting to perform an fsync operation.<div class="im"><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
If waitForConfirmsOrDie doesn't do what I want, is there a way to make a<br>
publisher wait until all subscribers ack a message before proceeding?<br>
</blockquote>
<br></div>
An alternative approach would be to declare a return queue for the consumers to report their progress to the publisher. The publisher can subscribe to this queue and use this to wait until the subscribers are ready before proceeding. The publisher would need to know how many subscribers to expect replies from.<br>
<br>
If your design allows it you should avoid coupling producers and consumers as tightly as you seem to need here. What is the reason for forcing producers and consumers in lockstep? Can the same aim be achieved in another way by taking a larger view of the system?<span class="HOEnZb"><font color="#888888"><br>
<br>
<br>
<br>
<br>
-Emile<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
</font></span></blockquote></div><br></div>