[rabbitmq-discuss] acks from temporary queues

[gneeri]

Hi Alex,

No problem at all, and again, thanks for the clarification. I really
do appreciate the help. That said, I think I now have a decent sense
of how to proceed. I'll let you know if I run into any perplexities,
but will try not to take much more of your time.

Thanks again,
Jonathan

On Sep 26, 12:28 pm, Alexandru Scvorţov <alexan... at rabbitmq.com>
wrote:
> Hi Jonathan,
>
> > This is a point of confusion for me then. In a previous note where I
> > mentioned setting autoAck to false and manually using basicAck,
> > Matthias replied as follows:
>
> Uf, I forgot about the persister in the last message.  Sorry.
>
> > If in the fanout context I could have used consumer acks to hold off
> > broker acks to the publisher, I would have expected that I could
> > effectively block confirms until after I sent basicAcks from the
> > consumer to the broker and then on up to the publisher. From
> > Matthias's note (and my tests), however, it appears that the broker
> > sends an ack back to the publisher as soon as it takes control of the
> > message and so issues a confirm without waiting on anything from the
> > consumer.
>
> He's right.  I was hoping the immediate flag will hold off the confirm,
> but, of course, it won't.
>
> > In the direct exchange context, however, I took your notes
> > to be indicating that the broker there will wait for an ack from the
> > consumer, such that if the consumer died while in process, but before
> > sending the ack, the broker would return a basic.return along with the
> > confirm. If this is not the case, then, my sense is that I will have
> > to have the consumers publish a response directly back to original
> > publisher as Matthias notes--while using the fanout broadcast to keep
> > track of the number of expected responses (as you indicate).
>
> You will have to have consumers send something to the original
> publisher and a fanout to keep track of responses.  You'll only need a
> timeout of some sort, so that if any of the consumers takes to long to
> respond, you react somehow (that probably means the consumer died
> processing the message).
>
> You'll still need the mandatory, immediate flags to at least see that
> the consumer was alive and was consuming from the queue at the time the
> message was sent.
>
> > Does that make sense? Basically, it still orbits around a question of
> > how and/or if consumer acks can be made to effect a producer. I am
> > guessing that my implementation will have to require the return
> > publications (responses), but any clarification you could add to the
> > consumer ack, confirm chain would be greatly appreciated.
>
> You'll need to publish back.  The only things confirms will give you in
> this case is the guarantee that basic.returns happen before confirms.
> So, once you receive the confirm, you know the message has reached the
> consumer.
>
> There's also no need to delay acknowledgement on the consumer side.
>
> Sorry for the confusion.
>
> Alex
>
>
>
>
>
>
>
> On Mon, Sep 26, 2011 at 09:04:06AM -0700, gneeri wrote:
> > Hi Alex,
>
> > This is a point of confusion for me then. In a previous note where I
> > mentioned setting autoAck to false and manually using basicAck,
> > Matthias replied as follows:
>
> > "Confirms mean one thing and one thing only: that the broker has
> > accepted
> > responsibility for the message.
>
> > "They concern the interaction between producers and the broker, and
> > have
> > nothing to do with consumer acks."
>
> > If in the fanout context I could have used consumer acks to hold off
> > broker acks to the publisher, I would have expected that I could
> > effectively block confirms until after I sent basicAcks from the
> > consumer to the broker and then on up to the publisher. From
> > Matthias's note (and my tests), however, it appears that the broker
> > sends an ack back to the publisher as soon as it takes control of the
> > message and so issues a confirm without waiting on anything from the
> > consumer. In the direct exchange context, however, I took your notes
> > to be indicating that the broker there will wait for an ack from the
> > consumer, such that if the consumer died while in process, but before
> > sending the ack, the broker would return a basic.return along with the
> > confirm. If this is not the case, then, my sense is that I will have
> > to have the consumers publish a response directly back to original
> > publisher as Matthias notes--while using the fanout broadcast to keep
> > track of the number of expected responses (as you indicate).
>
> > Does that make sense? Basically, it still orbits around a question of
> > how and/or if consumer acks can be made to effect a producer. I am
> > guessing that my implementation will have to require the return
> > publications (responses), but any clarification you could add to the
> > consumer ack, confirm chain would be greatly appreciated.
>
> > Thanks,
> > Jonathan
>
> > On Sep 26, 11:34 am, Alexandru Scvorţov <alexan... at rabbitmq.com>
> > wrote:
> > > Hi Jonathan,
>
> > > > "In addition, when processing "please log out this user" messages, a
> > > > machine should *not* acknowledge that it has received the message
> > > > until after it publishes its response (this way, if the machine dies
> > > > while processing, the sender will get a basic.return for it)."
>
> > > > Am I correct in understanding from this that, in the direct exchange
> > > > context, I can control when the consumer machine sends back its ack,
> > > > by calling something like basicAck at some point in the consumption/
> > > > delivery code?
>
> > > The exchange type has nothing to do with it.  When you consume from a
> > > queue, you specify the autoAck parameter.  If this is set to true
> > > (which it is, by default), then the server will not wait for the
> > > consumer to acknowledge receipt of the message (so, it's effectively
> > > auto-ack'ing all messages).  If it's set to false, the broker will not
> > > consider the message delivered to the consumer until it explicitly ack's
> > > the message.  See the API guide for an example (I assume you're using
> > > the Java client, but the other clients have very similar APIs):
> > >  http://www.rabbitmq.com/api-guide.html#consuming
>
> > > > If so, I would expect that I could skip the return
> > > > publishing (i.e., the response) from the consumer to the producer and
> > > > simply check confirms and confirms + basic.returns for each message
> > > > sent directly to each machine. Moreover, if, prior to publishing, I
> > > > associate the machine (or direct queue corresponding to it) to the
> > > > message to be sent, I should be able to find out which machines
> > > > present which form of response (i.e., confirm or confirm +
> > > > basic.return). Does this make sense or am I still reading too much
> > > > into the consumer ack (such that, for example, the confirmation
> > > > received by the publisher just represents the handoff of the message
> > > > to the queue and not its uptake by the consumer)?
>
> > > Oh.  I missed that.  Yes, I think that would work.
>
> > > Cheers,
> > > Alex
>
> > > On Mon, Sep 26, 2011 at 08:04:28AM -0700, gneeri wrote:
> > > > Hi Alex,
>
> > > > Thanks so much for the detailed reply. I had begun thinking along
> > > > these lines, but really appreciate the level of detail regarding the
> > > > specific types of publish requests, queues and exchanges. This is
> > > > really helpful and will likely save me many hours of trial and error.
> > > > I do have just one quick question though regarding the following:
>
> > > > "In addition, when processing "please log out this user" messages, a
> > > > machine should *not* acknowledge that it has received the message
> > > > until after it publishes its response (this way, if the machine dies
> > > > while processing, the sender will get a basic.return for it)."
>
> > > > Am I correct in understanding from this that, in the direct exchange
> > > > context, I can control when the consumer machine sends back its ack,
> > > > by calling something like basicAck at some point in the consumption/
> > > > delivery code? If so, I would expect that I could skip the return
> > > > publishing (i.e., the response) from the consumer to the producer and
> > > > simply check confirms and confirms + basic.returns for each message
> > > > sent directly to each machine. Moreover, if, prior to publishing, I
> > > > associate the machine (or direct queue corresponding to it) to the
> > > > message to be sent, I should be able to find out which machines
> > > > present which form of response (i.e., confirm or confirm +
> > > > basic.return). Does this make sense or am I still reading too much
> > > > into the consumer ack (such that, for example, the confirmation
> > > > received by the publisher just represents the handoff of the message
> > > > to the queue and not its uptake by the consumer)?
>
> > > > Thanks again, really, I very much appreciate your help.
>
> > > > Cheers,
> > > > Jonathan
>
> > > > On Sep 26, 5:54 am, Alexandru Scvorţov <alexan... at rabbitmq.com> wrote:
> > > > > Hi Jonathan,
>
> > > > > > 1. Do consumer acks have more significance in transactions, such that
> > > > > > I could use those to effect the functionality I'm looking for?
>
> > > > > Not really, no.  I don't think there's anyway to get what you want
> > > > > automatically.
>
> > > > > > 2. If not, is there any way (apart from the admin HTTP API) for
> > > > > > determining the number of consumers (e.g., bound queues) for a given
> > > > > > exchange? This way I could test the number of reply messages as you
> > > > > > suggest; otherwise, with needing to register and account for a dynamic
> > > > > > array of servers, I would lose a good deal of what I had hoped would
> > > > > > be the benefit of a fanout broadcast.
>
> > > > > There's no AMQP way of finding how many queues are bound to an exchange.
> > > > > As you say, there's the management API, but using that is likely to lead
> > > > > to races (what happens if you query the number of queues, then one of
> > > > > the machines dies before it sends its acknowledgement?).
>
> > > > > You could kind of get what you want with AMQP.  Use a fanout exchange
> > > > > like you want for broadcast messages, but use a direct exchange for
> > > > > direct communication between machines.  You could use the fanout
> > > > > exchange to make sure that all the machines know about each other (in
> > > > > particular, that each knows the queues the others bound to the direct
> > > > > exchange, so that they can communicate directly).  And you could
>
> ...
>
> read more »