[rabbitmq-discuss] Round-robin exchanges?
adamrabung at gmail.com
Tue Oct 11 15:52:16 BST 2011
That was the big thing I was missing - channels can be subscribed to more
than one queue! Thank you so much, that was a nice breakthrough.
On Tue, Oct 11, 2011 at 2:19 AM, Matthias Radestock
<matthias at rabbitmq.com>wrote:
> On 11/10/11 02:48, Adam Rabung wrote:
>> I am trying to implement a work queue in RabbitMQ. I have several
>> machines that will act as worker pools for consuming my queues. I would
>> like each job to be processed only once. Some jobs
>> have environmental requirements, ie must be executed on a machine with
>> an SSD. Not all worker pools will meet these requirements. A first
>> approach would be to have a queue for every permutation of requirements:
>> "No Requirements", "Requires SSD", "Requires Certificate", etc and have
>> each worker pool subscribe to all queues which it can handle.
> That is a sound approach.
> A majority of the jobs will have no requirements, so many worker pools
>> will be underutilized.
> As you say, the worker pools would subscribe to all the queues with
> combinations of requirements they can handle. That includes the "No
> Requirements" queue. The subscriptions can be active *simultaneously*, so
> work in the "No Requirements" queue would be round-robin routed to all
> workers, jobs in the "Require SSD" queue will be routed to all workers that
> have SSDs, etc.
> If each worker uses a single channel with a basic.qos prefetch setting of 1
> and subscribes to all the relevant queues on that channel, it will be fed
> work items one at a time from the subset of all these queues that have
> messages in them. There's some logic at the server that ensures this is
> reasonably fair, though you will most likely find that queues which can be
> handled by many workers drain faster than others.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the rabbitmq-discuss