[rabbitmq-discuss] Randomized Round-Robin?

[Martin Koch]

You are correct that work will tend to be evenly distributed over time but if all jobs take close to the same time, then there will be a tendency to load the hosts sequentially using only 1/N of N hosts, particularly after launching hosts. Also, if I add a new host with workers, then all workers on this host will be selected sequentially before using workers on the existing hosts. So therefore I believe that it would be useful to have a mechanism to randomly select the next worker. 

/Martin


On 28/03/2013, at 16.44, Emile Joubert <emile at rabbitmq.com> wrote:

> 
> Hi Martin,
> 
> On 27/03/13 13:26, Martin Koch wrote:
>> I have a setup with many workers on several hosts; each worker consumes
>> tasks from a work queue. 
> 
>> What I would like to achieve is that a random worker is chosen instead
>> of the deterministic round-robin.
> It is not clear what this achieves - why does it matter if consumers are
> initially selected in the same order that they joined?
> 
> If the number workers per host exceeds the capacity of the host to
> perform work in parallel, and work arrives very slowly, and workers are
> selected in sequence then you may see suboptimal throughput. The
> solution then is to reduce the number of workers per host. The number of
> consumers per host should not exceed the capacity of the host to perform
> tasks in parallel. This is often approximated by the number of cores.
> 
> 
> 
> 
> 
> -Emile
> 
> 
> 
> 
>