[rabbitmq-discuss] Proposal: Change consumers Round-Robin behaviour

Thu May 2 12:53:41 BST 2013

On 02/05/13 12:27, Jesper Louis Andersen wrote:
> Hi,

Hi.

<snip>

> When this happens, we pick a random consumer in the queue and move him
> to the front. Over time, this "shuffles" the queue into a random order.
> It is also not going to cost anything on the critical path since we only
> do it when we have an empty queue and excess workers.

Bear in mind that the "empty queue and excess workers" case *is* on the 
critical path for straight-through messaging (i.e. where the queue is on 
average empty and we are streaming messages through it as fast as possible).

> And we are going
> to do very little work unless the queue has a behaviour where it empties
> often in which case you get full random distribution on the consumers
> with this scheme.

Again, this is a common case for some users.

Also: quite a lot of people *want* round-robin behaviour here.

> The background for the proposal is that Round-robin distribution of
> messages often tend to bad behaviour over time. By adding a bit of
> randomness to the process, we automatically alleviate a number of
> determinism-problems and get better distribution of messages over
> consumers. One could also imagine different distribution schemes, but
> those will be more expensive in practice compared to this proposal,
> which should only have a cost when the queue is not under heavy load.
>
> * Did I miss anything?
> * Is this a good or bad idea? And why?
> * Do we break any rules w.r.t. AMQP by implementing this?
> * Is priority on the queue going to be harder to implement? (I don't
> think so, but...)

So I am not (yet) convinced. This would not violate the spec or make 
priorities harder to implement but I'm not convinced that the majority 
of users would see it as a step forward.

Does anybody who currently depends on round-robin want to speak up?

Cheers, Simon

-- 
Simon MacMullen
RabbitMQ, VMware