[rabbitmq-discuss] basic design question

[Isaac Cambron]

I did get the tip revision to work (the problem was just that I hadn't shut
down the server properly). However, I'm still not seeing the desired
behavior. I have two programs:

The client, which does this:

ch.QueueDeclare("myqueue");
ch.ExchangeDeclare("ex", ExchangeType.Direct);
ch.QueueBind("myqueue", "ex", "test", false, null);
for (int i = 0; i < 11; i++ )
                    ch.BasicPublish("ex", "test", null,
Encoding.UTF8.GetBytes("message " + i));

and the server, which does this:

                QueueingBasicConsumer consumer = new
QueueingBasicConsumer();
                ch.BasicConsume("myqueue", true, null, consumer);
                ch.BasicQos(0, 1, false);

                while (true)
                {
                    try
                    {
                        BasicDeliverEventArgs e = (BasicDeliverEventArgs)
consumer.Queue.Dequeue();
                        Console.WriteLine(Encoding.UTF8.GetString(e.Body));

                        Thread.Sleep(seconds * 1000); //pretend to take a
while

                        ch.BasicAck(e.DeliveryTag, false);
                    }


If I run the client and then two different instances of the server, one with
the seconds variable set to 1 second and the other with 2, I'd expect the
former instance to print twice as many message bodies. Instead, they
alternate, as if they had just aggressively fetched everything in the queue
in a round-robin way.

Do I have something conceptually wrong, or is that an implementation issue?


On Tue, Apr 21, 2009 at 10:19 PM, Isaac Cambron <icambron at gmail.com> wrote:

> Matthias,
>
> Thanks, that's very helpful. A couple of follow-up questions:
>
> 1. What snapshot should I use? I tried out the default revision (not sure
> what that tracks) and it didn't queue like I expected; the first consumer to
> subscribe got all the tasks, even if more subscribers were added before the
> acks. I can't get the tip to run at all (crashes on startup). Presumably I
> need something in between?
>
> 2. It's not clear to me what all of the other available qos parameters
> mean. It appears (and again, wasn't able to try this on the tip revision)
> that only BasicQos(0, n, false) is supported. This sounds like it's what I
> want anyway (with, obv, n=1); is that right?
>
> If what I have was expected to work, I can provide source code, or the
> crash dump if that's unexpected too.
>
> Thanks,
> Isaac
>
> On Tue, Apr 21, 2009 at 7:38 PM, Matthias Radestock <matthias at lshift.net>wrote:
>
>> Isaac,
>>
>> Isaac Cambron wrote:
>>
>>> I have a client that will publish a bunch of tasks to execute - tasks A,
>>> B, C, D, E. I have a pool of machines that, individually, know how to do
>>> some subset of the tasks. For example, let's say that machine 1 can do A, B,
>>> and C, and machine 2 can do C, D, and E. A machine can only do one task at a
>>> time, and I only want one machine to do any particular task. Tasks can wait
>>> indefinitely until there's a free machine capable of executing it.
>>>
>>> I'm not sure what pieces go where. Do I create one queue per task? (If
>>> so, how will I make the machines only respond to one message from any queue
>>> at a time?)
>>>
>>
>> That, in combination with a basic.qos prefetch count of 1, should work.
>>
>> More specifically ...
>>
>> - create a direct exchange to which tasks get published, using the task
>> type as the routing key
>>
>> - create one queue per task type and bind it to the direct exchange with
>> the tasks type as the binding key
>>
>> - let each worker machine open a channel, set the basic.qos prefetch count
>> to 1, and then subscribe to (basic.consume) each of the queues corresponding
>> to task types supported by that worker.
>>
>> - when a worker is done with a task it basic.ack's the message
>>
>>
>> The prefetch count limits the number of tasks a worker will be sent before
>> ack'ing them. Note that the basic.qos functionality required to do that has
>> not made it into an official release yet; you'll have to use  a recent
>> snapshot of the RabbitMQ source. Your example is actually a pretty
>> interesting use case for basic.qos, exploiting some of the more advanced
>> aspects of that feature, so I'd love to hear how it works out for you.
>>
>>
>> Regards,
>>
>> Matthias.
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20090421/60a609fd/attachment.htm