[rabbitmq-discuss] Round-robin exchanges?

Tue Oct 11 15:52:16 BST 2011

That was the big thing I was missing - channels can be subscribed to more
than one queue!  Thank you so much, that was a nice breakthrough.
Adam

On Tue, Oct 11, 2011 at 2:19 AM, Matthias Radestock
<matthias at rabbitmq.com>wrote:

> Adam,
>
>
> On 11/10/11 02:48, Adam Rabung wrote:
>
>> Hello,
>> I am trying to implement a work queue in RabbitMQ.  I have several
>> machines that will act as worker pools for consuming my queues.  I would
>> like each job to be processed only once.  Some jobs
>> have environmental requirements, ie must be executed on a machine with
>> an SSD.  Not all worker pools will meet these requirements.  A first
>> approach would be to have a queue for every permutation of requirements:
>> "No Requirements", "Requires SSD", "Requires Certificate", etc and have
>> each worker pool subscribe to all queues which it can handle.
>>
>
> That is a sound approach.
>
>
>  A majority of the jobs will have no requirements, so many worker pools
>> will be underutilized.
>>
>
> As you say, the worker pools would subscribe to all the queues with
> combinations of requirements they can handle. That includes the "No
> Requirements" queue. The subscriptions can be active *simultaneously*, so
> work in the "No Requirements" queue would be round-robin routed to all
> workers, jobs in the "Require SSD" queue will be routed to all workers that
> have SSDs, etc.
>
> If each worker uses a single channel with a basic.qos prefetch setting of 1
> and subscribes to all the relevant queues on that channel, it will be fed
> work items one at a time from the subset of all these queues that have
> messages in them. There's some logic at the server that ensures this is
> reasonably fair, though you will most likely find that queues which can be
> handled by many workers drain faster than others.
>
> Regards,
>
> Matthias.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20111011/22e1fc5c/attachment.htm>