[rabbitmq-discuss] [rabbitmq-consistent-hash-exchange] Question related to workload distribution

[Jon Brisbin]

----- Original Message -----
> Hi Josh,
> 
> On 25/01/12 23:47, Josh Stone wrote:
> > Regarding workers seeing a message with a routing key they're not
> > expecting, I'm not sure how a worker would know if a message is
> > acceptable for them without knowing what the hash ring looks like.
> > If
> > our workers did mirror the hash algorithm, when a re-mapping
> > occurred
> > they could check locally to see what work was effected. This brings
> > us
> > closer to something like Dynamo where each node maintains a view of
> > the hash ring. In our case, each worker's view of the hash ring
> > would
> > match that on the consistent hash exchange. This is likely the only
> > way they could handle key-remappings.
> > 
> > Has anyone gone down this road?
> 
> I doubt it, because the consistent hash exchange uses a random number
> generator to ensure a fair distribution among all the bindings. What
> you
> are proposing will require workers to have access to the same random
> number stream and therefore infeasible.


Actually I think one could use statebox for this:

https://github.com/mochi/statebox


Thanks! 

Jon Brisbin 
http://about.me/jbrisbin 
http//jbrisbin.com 


> 
> If you share more details about the problem you are trying to solve
> then
> we may be able to make alternative suggestions. Is there any reason
> why
> your problem cannot be solved by having multiple workers subscribe to
> the same queue and sharing the load in that way?
> 
> 
> 
> -Emile
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>