[rabbitmq-discuss] basic design question

Matthias Radestock matthias at lshift.net
Wed Apr 22 00:38:45 BST 2009


Isaac Cambron wrote:
> I have a client that will publish a bunch of tasks to execute - tasks A, 
> B, C, D, E. I have a pool of machines that, individually, know how to do 
> some subset of the tasks. For example, let's say that machine 1 can do 
> A, B, and C, and machine 2 can do C, D, and E. A machine can only do one 
> task at a time, and I only want one machine to do any particular task. 
> Tasks can wait indefinitely until there's a free machine capable of 
> executing it.
> I'm not sure what pieces go where. Do I create one queue per task? (If 
> so, how will I make the machines only respond to one message from any 
> queue at a time?)

That, in combination with a basic.qos prefetch count of 1, should work.

More specifically ...

- create a direct exchange to which tasks get published, using the task 
type as the routing key

- create one queue per task type and bind it to the direct exchange with 
the tasks type as the binding key

- let each worker machine open a channel, set the basic.qos prefetch 
count to 1, and then subscribe to (basic.consume) each of the queues 
corresponding to task types supported by that worker.

- when a worker is done with a task it basic.ack's the message

The prefetch count limits the number of tasks a worker will be sent 
before ack'ing them. Note that the basic.qos functionality required to 
do that has not made it into an official release yet; you'll have to use 
  a recent snapshot of the RabbitMQ source. Your example is actually a 
pretty interesting use case for basic.qos, exploiting some of the more 
advanced aspects of that feature, so I'd love to hear how it works out 
for you.



More information about the rabbitmq-discuss mailing list