[rabbitmq-discuss] Form Submission - Add A Question

[Sean Treadway]

On Nov 20, 2008, at 23: 55, Ben Hood wrote:
> http://lettuce.squarespace.com/faq/receiving-messages/does-basicget-automatically-acknowledge-a-message.html
> http://lettuce.squarespace.com/faq/queues/what-is-queue-depth.html

Thanks for formulating these two into the FAQs.

>> Publish 1
>> Publish 2
>> Publish 3
>> Client B: sends Basic.Consume
>> Client B: receives Basic.Deliver 1
>> Client B: sends Bacic.Ack 1
>> Client A: checks depth
>>
>> I would assume that the queue depth should be 2, but what is  
>> returned is 0.
>> Without the Ack, I would still assume the queue depth to be 3.
>
> Everything is asynchronous here and it is not clear whether
>
> a) The the order of events over time from the broker's perspective
> actually matches the client's perception;

 From what I was testing, enough time was passing between each event  
that the order of events was reasonably predictable.

> b) The remaining 2 messages that you are supposedly missing are
> actually in mid-flight somewhere.

What qualifies as "mid-flight" and how/when does that happen?

> This will depend in part on how your client behaves, e.g. whether it
> absorbs socket back pressure or not.

I don't believe I can control any socket options with the library I'm  
using.

> Two questions:
>
> a) Which statistic are you referring to when you are reporting the  
> queue depth?
> b) What library are you using?

I'm using the result returned by Queue.Declare passive=1 from RabbitMQ  
1.4.0 using Aman's Ruby library from http://github.com/tmm1/amqp/commits/d2cbf1e48eacdd3a6708212ced4bc1a204d5ae21

The test case that I'm basing my observations on is zipped up with  
instructions in the README.  The tests require the EventMachine gem,  
and amqp 0.5.9 or source to be installed.

>> Another issue that I took much time to debug was failing to  
>> understand how
>> messages are delivered to clients, where I am assuming a fair  
>> distribution
>> to clients.  Example:
>>
>> Publish 1
>> Client A: Basic.Consume
>> Publish 2
>> Publish 3
>> Publsih 4
>> Client B: Basic.Consume
>> Publish 5
>> Publish 6
>>
>> The delivered order I would assume would be close to this:
>>
>> Client A: Deliver/Ack 1
>> Client B: Deliver/Ack 2
>> Client A: Deliver/Ack 3
>> Client B: Deliver/Ack 4
>> Client A: Deliver/Ack 5
>> Client B: Deliver/Ack 6
>>
>> But is actually:
>>
>> Client A: Deliver/Ack 1
>> Client B: Deliver/Ack 5
>> Client A: Deliver/Ack 2
>> Client A: Deliver/Ack 3
>> Client A: Deliver/Ack 4
>> Client A: Deliver/Ack 6
>
> Do client A and B have their own channel or do they share a channel?

A and B have different channels, they are running on separate processes.

>> Client B never sees 2,3 or 4.  In the use case of load balancing -  
>> we need
>> to start more clients when the queue depth grows, but this behavior  
>> prevents
>> the new clients to catch up on the previous messages.  Do you have  
>> any
>> suggestions for how to accomplish fair distribution to 'late comer'  
>> clients?
>> Is this even possible, or are there best practices that I'm  
>> missing?  Or
>> are there ways of configuring RabbitMQ to deliver messages like in  
>> the 1st
>> example?
>
> Yes, it's a command that is called basic.qos whereby you set a
> per-channel pre-fetch window and basically credit consumers
> proportionally to the number of acks they send back.
>
> It's currently in development ATM (branch 18577 refers).

Excellent to hear.

I'm sure that Basic.Get can be used, but in my rough tests, I couldn't  
get un-ack'd messages redelivered.  I believe I was failing to close  
my channel to get them redelivered.  I'll also take another look at  
Basic.Get with no_ack=true.

Thanks!

-Sean



-------------- next part --------------
A non-text attachment was scrubbed...
Name: queue_depth.tgz
Type: application/octet-stream
Size: 1663 bytes
Desc: not available
Url : http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20081121/9982af4d/attachment.obj 
-------------- next part --------------