[rabbitmq-discuss] [rabbitmq-consistent-hash-exchange] Question related to workload distribution

[Emile Joubert]

Hi Josh,

On 25/01/12 23:47, Josh Stone wrote:
> Regarding workers seeing a message with a routing key they're not
> expecting, I'm not sure how a worker would know if a message is
> acceptable for them without knowing what the hash ring looks like. If
> our workers did mirror the hash algorithm, when a re-mapping occurred
> they could check locally to see what work was effected. This brings us
> closer to something like Dynamo where each node maintains a view of
> the hash ring. In our case, each worker's view of the hash ring would
> match that on the consistent hash exchange. This is likely the only
> way they could handle key-remappings.
> 
> Has anyone gone down this road?

I doubt it, because the consistent hash exchange uses a random number
generator to ensure a fair distribution among all the bindings. What you
are proposing will require workers to have access to the same random
number stream and therefore infeasible.

If you share more details about the problem you are trying to solve then
we may be able to make alternative suggestions. Is there any reason why
your problem cannot be solved by having multiple workers subscribe to
the same queue and sharing the load in that way?



-Emile