Hello,<div>I am trying to implement a work queue in RabbitMQ. I have several machines that will act as worker pools for consuming my queues. I would like each job to be processed only once. Some jobs have environmental requirements, ie must be executed on a machine with an SSD. Not all worker pools will meet these requirements. A first approach would be to have a queue for every permutation of requirements: "No Requirements", "Requires SSD", "Requires Certificate", etc and have each worker pool subscribe to all queues which it can handle. </div>
<div><br></div><div>A majority of the jobs will have no requirements, so many worker pools will be underutilized. To fight this, I am hoping to distribute jobs in a round-robin fashion to all queues that can handle them. For example, a no-requirements job can be handled by any worker pool, and therefore should be distributed equally to all of them. That is, jobs w/ no requirements will be evenly distributed to "No Requirements", "Requires SSD", "Requires Certificate", etc.</div>
<div><br></div><div>I tried this with topic exchanges, but of course they will distribute the job to ALL matching queues, rather than a single arbitrary queue.</div><div><br></div><div>Some terrible alternative approaches I have so far:</div>
<div>1. Have producers submit to a single queue, and have a home-grown "load balancer" consumer on the other end which distributes jobs to appropriate queues.</div><div>2. When a worker pool spins up, rather than have each worker consume a dedicate queue, have at least some of them hop between queues in an attempt to keep them busy.</div>
<div>3. Hacks with timestamps/modulo arithmetic in the routing key to trick the exchange into being round robin.</div><div><br></div><div>Being new to RabbitMQ, my intuition tells me #1 will be a big performance problem, #2 will be a performance and maintenance problem, and #3 just won't end up working. I suspect I'm missing something big here :)</div>
<div><br></div><div>Has anyone successfully load-balanced their workers in a situation like this?</div><div><br></div><div>Any advice much appreciated.</div><div>Adam</div>