[rabbitmq-discuss] [rabbitmq-consistent-hash-exchange] Question related to workload distribution

[Josh Stone]

Thanks for the response Emile,

Key remapping is ok and expected. I'm just curious how users deal with
that when using the consistent hash exchange to distribute work. Do
you have any insight into what those strategies might look like?

Cheers,
Josh

On Jan 25, 10:31 am, Emile Joubert <em... at rabbitmq.com> wrote:
> Hi Josh,
>
> On 24/01/12 23:39, Josh Stone wrote:
>
> > We're looking at using rabbitmq with the consistent hash exchange to
> > distribute workload to a set of workers with each worker subscribing
> > to one or more topics. It looks like this will work well, until a
> > worker along with its topics either comes or leaves, which will
> > disrupt the hashing.
>
> Keep in mind that when binding a queue to a consistent hash exchange you
> don't bind with a topic. You bind with a number that indicates what
> share of messages the queue will receive. Those messages can have any
> routing key. If you want workers to handle only particular topics then
> you should use a topic exchange.
>
> > Particularly, if a new worker enters and creates some topics on the
> > exchange, messages that previously would have been delivered to
> > another topic will be directed to the new topic. And if an existing
> > worker was operating on some data from a previous message and a new
> > message to operate on that data is sent to a new topic, we could have
> > a race condition.
>
> For the consistent hash exchange some keys will be unavoidably be
> remapped when the bindings change. The advantage is that not all keys
> need to be remapped. If your workers can only deal with one or a limited
> number of topics then you should not be using a consistent hash exchange.
>
> -Emile
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss