[rabbitmq-discuss] Form Submission - Add A Question

[Ben Hood]

Sean,

On Thu, Nov 20, 2008 at 12:55 PM, Sean Treadway <sean at soundcloud.com> wrote:
> Thanks Ben, it does indeed help.  For example, I didn't know that Basic.Get
> automatically Acks a message.

http://lettuce.squarespace.com/faq/receiving-messages/does-basicget-automatically-acknowledge-a-message.html

> Also, from someone approaching RabbitMQ without a strong background in
> messaging I'm still trying to map out the differences in terminology -
> particularly with regards to queue depth, and consumers vs connections vs
> clients.  I read 'queue depth' as 'queue size', as 'how many messages have
> been published waiting for acknowledgment'. But 'queue size' probably means
> about 10 different things to you :)

http://lettuce.squarespace.com/faq/queues/what-is-queue-depth.html

> I'll try to describe my confusion with
> an example:
>
> Publish 1
> Publish 2
> Publish 3
> Client A: checks depth with Queue.Declare passive
>
> Client A gets a queue depth of 3.

If this was the only interaction with the queue, then you could
feasibly expect this response.

> But the way RabbitMQ (and possibly AMQP) works is not very obvious:
>
> Publish 1
> Publish 2
> Publish 3
> Client B: sends Basic.Consume
> Client B: receives Basic.Deliver 1
> Client B: sends Bacic.Ack 1
> Client A: checks depth
>
> I would assume that the queue depth should be 2, but what is returned is 0.
>  Without the Ack, I would still assume the queue depth to be 3.

Everything is asynchronous here and it is not clear whether

a) The the order of events over time from the broker's perspective
actually matches the client's perception;
b) The remaining 2 messages that you are supposedly missing are
actually in mid-flight somewhere.

This will depend in part on how your client behaves, e.g. whether it
absorbs socket back pressure or not.

Two questions:

a) Which statistic are you referring to when you are reporting the queue depth?
b) What library are you using?

> I write this here, because I've been trying to formulate a question but
> can't really come up with one.  Maybe something like "how is queue depth
> calculated?"  or "how do you measure the queue depth of all un-ack'd
> messages?"  Or even something as simple as a glossary of common messaging
> terminology and what each vendor uses for the terms.

Please see the linked FAQ.

> Another issue that I took much time to debug was failing to understand how
> messages are delivered to clients, where I am assuming a fair distribution
> to clients.  Example:
>
> Publish 1
> Client A: Basic.Consume
> Publish 2
> Publish 3
> Publsih 4
> Client B: Basic.Consume
> Publish 5
> Publish 6
>
> The delivered order I would assume would be close to this:
>
> Client A: Deliver/Ack 1
> Client B: Deliver/Ack 2
> Client A: Deliver/Ack 3
> Client B: Deliver/Ack 4
> Client A: Deliver/Ack 5
> Client B: Deliver/Ack 6
>
> But is actually:
>
> Client A: Deliver/Ack 1
> Client B: Deliver/Ack 5
> Client A: Deliver/Ack 2
> Client A: Deliver/Ack 3
> Client A: Deliver/Ack 4
> Client A: Deliver/Ack 6

Do client A and B have their own channel or do they share a channel?

> Client B never sees 2,3 or 4.  In the use case of load balancing - we need
> to start more clients when the queue depth grows, but this behavior prevents
> the new clients to catch up on the previous messages.  Do you have any
> suggestions for how to accomplish fair distribution to 'late comer' clients?
>  Is this even possible, or are there best practices that I'm missing?  Or
> are there ways of configuring RabbitMQ to deliver messages like in the 1st
> example?

Yes, it's a command that is called basic.qos whereby you set a
per-channel pre-fetch window and basically credit consumers
proportionally to the number of acks they send back.

It's currently in development ATM (branch 18577 refers).

HTH,

Ben