[rabbitmq-discuss] acks from temporary queues

Mon Sep 26 17:04:06 BST 2011

Hi Alex,

This is a point of confusion for me then. In a previous note where I
mentioned setting autoAck to false and manually using basicAck,
Matthias replied as follows:

"Confirms mean one thing and one thing only: that the broker has
accepted
responsibility for the message.

"They concern the interaction between producers and the broker, and
have
nothing to do with consumer acks."

If in the fanout context I could have used consumer acks to hold off
broker acks to the publisher, I would have expected that I could
effectively block confirms until after I sent basicAcks from the
consumer to the broker and then on up to the publisher. From
Matthias's note (and my tests), however, it appears that the broker
sends an ack back to the publisher as soon as it takes control of the
message and so issues a confirm without waiting on anything from the
consumer. In the direct exchange context, however, I took your notes
to be indicating that the broker there will wait for an ack from the
consumer, such that if the consumer died while in process, but before
sending the ack, the broker would return a basic.return along with the
confirm. If this is not the case, then, my sense is that I will have
to have the consumers publish a response directly back to original
publisher as Matthias notes--while using the fanout broadcast to keep
track of the number of expected responses (as you indicate).

Does that make sense? Basically, it still orbits around a question of
how and/or if consumer acks can be made to effect a producer. I am
guessing that my implementation will have to require the return
publications (responses), but any clarification you could add to the
consumer ack, confirm chain would be greatly appreciated.

Thanks,
Jonathan

On Sep 26, 11:34 am, Alexandru Scvorţov <alexan... at rabbitmq.com>
wrote:
> Hi Jonathan,
>
> > "In addition, when processing "please log out this user" messages, a
> > machine should *not* acknowledge that it has received the message
> > until after it publishes its response (this way, if the machine dies
> > while processing, the sender will get a basic.return for it)."
>
> > Am I correct in understanding from this that, in the direct exchange
> > context, I can control when the consumer machine sends back its ack,
> > by calling something like basicAck at some point in the consumption/
> > delivery code?
>
> The exchange type has nothing to do with it.  When you consume from a
> queue, you specify the autoAck parameter.  If this is set to true
> (which it is, by default), then the server will not wait for the
> consumer to acknowledge receipt of the message (so, it's effectively
> auto-ack'ing all messages).  If it's set to false, the broker will not
> consider the message delivered to the consumer until it explicitly ack's
> the message.  See the API guide for an example (I assume you're using
> the Java client, but the other clients have very similar APIs):
>  http://www.rabbitmq.com/api-guide.html#consuming
>
> > If so, I would expect that I could skip the return
> > publishing (i.e., the response) from the consumer to the producer and
> > simply check confirms and confirms + basic.returns for each message
> > sent directly to each machine. Moreover, if, prior to publishing, I
> > associate the machine (or direct queue corresponding to it) to the
> > message to be sent, I should be able to find out which machines
> > present which form of response (i.e., confirm or confirm +
> > basic.return). Does this make sense or am I still reading too much
> > into the consumer ack (such that, for example, the confirmation
> > received by the publisher just represents the handoff of the message
> > to the queue and not its uptake by the consumer)?
>
> Oh.  I missed that.  Yes, I think that would work.
>
> Cheers,
> Alex
>
>
>
>
>
>
>
>
>
> On Mon, Sep 26, 2011 at 08:04:28AM -0700, gneeri wrote:
> > Hi Alex,
>
> > Thanks so much for the detailed reply. I had begun thinking along
> > these lines, but really appreciate the level of detail regarding the
> > specific types of publish requests, queues and exchanges. This is
> > really helpful and will likely save me many hours of trial and error.
> > I do have just one quick question though regarding the following:
>
> > "In addition, when processing "please log out this user" messages, a
> > machine should *not* acknowledge that it has received the message
> > until after it publishes its response (this way, if the machine dies
> > while processing, the sender will get a basic.return for it)."
>
> > Am I correct in understanding from this that, in the direct exchange
> > context, I can control when the consumer machine sends back its ack,
> > by calling something like basicAck at some point in the consumption/
> > delivery code? If so, I would expect that I could skip the return
> > publishing (i.e., the response) from the consumer to the producer and
> > simply check confirms and confirms + basic.returns for each message
> > sent directly to each machine. Moreover, if, prior to publishing, I
> > associate the machine (or direct queue corresponding to it) to the
> > message to be sent, I should be able to find out which machines
> > present which form of response (i.e., confirm or confirm +
> > basic.return). Does this make sense or am I still reading too much
> > into the consumer ack (such that, for example, the confirmation
> > received by the publisher just represents the handoff of the message
> > to the queue and not its uptake by the consumer)?
>
> > Thanks again, really, I very much appreciate your help.
>
> > Cheers,
> > Jonathan
>
> > On Sep 26, 5:54 am, Alexandru Scvorţov <alexan... at rabbitmq.com> wrote:
> > > Hi Jonathan,
>
> > > > 1. Do consumer acks have more significance in transactions, such that
> > > > I could use those to effect the functionality I'm looking for?
>
> > > Not really, no.  I don't think there's anyway to get what you want
> > > automatically.
>
> > > > 2. If not, is there any way (apart from the admin HTTP API) for
> > > > determining the number of consumers (e.g., bound queues) for a given
> > > > exchange? This way I could test the number of reply messages as you
> > > > suggest; otherwise, with needing to register and account for a dynamic
> > > > array of servers, I would lose a good deal of what I had hoped would
> > > > be the benefit of a fanout broadcast.
>
> > > There's no AMQP way of finding how many queues are bound to an exchange.
> > > As you say, there's the management API, but using that is likely to lead
> > > to races (what happens if you query the number of queues, then one of
> > > the machines dies before it sends its acknowledgement?).
>
> > > You could kind of get what you want with AMQP.  Use a fanout exchange
> > > like you want for broadcast messages, but use a direct exchange for
> > > direct communication between machines.  You could use the fanout
> > > exchange to make sure that all the machines know about each other (in
> > > particular, that each knows the queues the others bound to the direct
> > > exchange, so that they can communicate directly).  And you could use the
> > > direct exchange to publish persistent, confirm, mandatory, immediate messages
> > > directly to other machines.  With these messages, the sender will be
> > > notified if the other machine received the message (only a confirm is
> > > received for the publish), or if the other machine went offline in the past
> > > (a basic.return and a confirm is received).  This way, the sending
> > > machine would know which other machines received its message and, so,
> > > will know to wait for the correct number of responses.  The queue bound
> > > to the direct exchange would have to be durable, auto-delete for this to
> > > work (durable for confirms to work and auto-delete so that once the sole
> > > consumer dies, the queue is deleted and publishing to it will result in
> > > a basic.return).  In addition, when processing "please log out this
> > > user" messages, a machine should *not* acknowledge that it has
> > > received the message until after it publishes its response (this way, if
> > > the machine dies while processing, the sender will get a basic.return for
> > > it).
>
> > > Hope this helps.
>
> > > Cheers,
> > > Alex
>
> > > On Fri, Sep 23, 2011 at 04:44:14PM -0700, gneeri wrote:
> > > > Hi Matthias,
>
> > > > Thanks for clarifying that. From the explication of the rules for
> > > > confirms, it had appeared to me that the relationships to queues and
> > > > consumers had more weight in the confirmation process. That not being
> > > > the case, I have 2 brief follow up questions:
>
> > > > 1. Do consumer acks have more significance in transactions, such that
> > > > I could use those to effect the functionality I'm looking for?
>
> > > > 2. If not, is there any way (apart from the admin HTTP API) for
> > > > determining the number of consumers (e.g., bound queues) for a given
> > > > exchange? This way I could test the number of reply messages as you
> > > > suggest; otherwise, with needing to register and account for a dynamic
> > > > array of servers, I would lose a good deal of what I had hoped would
> > > > be the benefit of a fanout broadcast.
>
> > > > Thanks in advance,
> > > > Jonathan
>
> > > > On Sep 23, 6:55 pm, Matthias Radestock <matth... at rabbitmq.com> wrote:
> > > > > On 23/09/11 21:26, gneeri wrote:
>
> > > > > > My expectation, then, is that producer would publish its message and
> > > > > > then be blocked until the delayed consumer returns an ack
>
> > > > > Confirms mean one thing and one thing only: that the broker has accepted
> > > > > responsibility for the message.
>
> > > > > They concern the interaction between producers and the broker, and  have
> > > > > nothing to do with consumer acks.
>
> > > > > Yes, there are some conditions in which a consumer ack can trigger a
> > > > > confirm in RabbitMQ, but that is an implementation detail and not
> > > > > something you'd want to design an application around.
>
> > > > > > My hope is that I will not have to aggregate a list of servers and
> > > > > > pass along a single message to each server, but perhaps that is what
> > > > > > will be required.
>
> > > > > You can still send a single message via a fanout exchange. But the
> > > > > consumers then in turn need to send a message back to the sender. Your
> > > > > sender will need to know how many consumers there are so it knows when
> > > > > it has gotten all responses..
>
> > > > > Regards,
>
> > > > > Matthias.
> > > > > _______________________________________________
> > > > > rabbitmq-discuss mailing list
> > > > > rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> > > > _______________________________________________
> > > > rabbitmq-discuss mailing list
> > > > rabbitmq-disc... at lists.rabbitmq.com
> > > >https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
> > > _______________________________________________
> > > rabbitmq-discuss mailing list
> > > rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> > _______________________________________________
> > rabbitmq-discuss mailing list
> > rabbitmq-disc... at lists.rabbitmq.com
> >https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss