[rabbitmq-discuss] Pub/Sub -- block publisher until all acks received from subscribers?

[Emile Joubert]

Hi Nick,

This page has more detail and you may find some of your answers:
http://www.rabbitmq.com/confirms.html

On 31/10/12 22:10, Nick Martin wrote:
> Wait until all messages published since the last call have been
> either ack'd or nack'd by the broker.

A message being ack'd by the broker is different from being ack'd by a 
consumer. If the broker acknowledges a messages it means that the broker 
has taken responsibility for it by writing it to disk or successfully 
handed it off. Handing off could mean returning the messages to the 
publisher (unroutable mandatory msg) or successfully delivering to 
consumer. The publisher cannot choose how the broker discharges this 
responsibility. In your case you would like a broker acknowledgement 
only when the message is successfully delivered to a consumer, but the 
broker may decide to write the message to disk and acknowledge when the 
OS guarantees that the disk write succeeded.

> It seems to me that the publisher is
> supposed to block until all of the messages are ack'd via its call to
> waitForConfirmsOrDie(). Is this what this is really supposed to do?

Yes, waitForConfirmsOrDie() will block until an ack is received from the 
broker.

> This example code seemed like it matched up perfectly with what I was
> trying to do. But, it doesn't seem to work the way I thought it did. In
> fact, if, in the consumer thread, I turn off auto-acking messages, then
> waitForConfirmsOrDie() still returns immediately.

This is because the broker has taken responsibility for the message by 
writing it to disk.

> So what does waitForConfirmsOrDie() actually do? When would it block?

It could block e.g. as a result of attempting to perform an fsync operation.

> If waitForConfirmsOrDie doesn't do what I want, is there a way to make a
> publisher wait until all subscribers ack a message before proceeding?

An alternative approach would be to declare a return queue for the 
consumers to report their progress to the publisher. The publisher can 
subscribe to this queue and use this to wait until the subscribers are 
ready before proceeding. The publisher would need to know how many 
subscribers to expect replies from.

If your design allows it you should avoid coupling producers and 
consumers as tightly as you seem to need here. What is the reason for 
forcing producers and consumers in lockstep? Can the same aim be 
achieved in another way by taking a larger view of the system?




-Emile