[rabbitmq-discuss] basic design question

[Matthias Radestock]

Isaac,

Isaac Cambron wrote:
> I have a client that will publish a bunch of tasks to execute - tasks A, 
> B, C, D, E. I have a pool of machines that, individually, know how to do 
> some subset of the tasks. For example, let's say that machine 1 can do 
> A, B, and C, and machine 2 can do C, D, and E. A machine can only do one 
> task at a time, and I only want one machine to do any particular task. 
> Tasks can wait indefinitely until there's a free machine capable of 
> executing it.
> 
> I'm not sure what pieces go where. Do I create one queue per task? (If 
> so, how will I make the machines only respond to one message from any 
> queue at a time?)

That, in combination with a basic.qos prefetch count of 1, should work.

More specifically ...

- create a direct exchange to which tasks get published, using the task 
type as the routing key

- create one queue per task type and bind it to the direct exchange with 
the tasks type as the binding key

- let each worker machine open a channel, set the basic.qos prefetch 
count to 1, and then subscribe to (basic.consume) each of the queues 
corresponding to task types supported by that worker.

- when a worker is done with a task it basic.ack's the message


The prefetch count limits the number of tasks a worker will be sent 
before ack'ing them. Note that the basic.qos functionality required to 
do that has not made it into an official release yet; you'll have to use 
  a recent snapshot of the RabbitMQ source. Your example is actually a 
pretty interesting use case for basic.qos, exploiting some of the more 
advanced aspects of that feature, so I'd love to hear how it works out 
for you.


Regards,

Matthias.