[rabbitmq-discuss] [rabbitmq-consistent-hash-exchange] Question related to workload distribution

[Josh Stone]

Hi Emile -

A followup question. When a new queue is created and a key-remapping
occurs, is there any way I could learn which keys were re-mapped? This
would allow me to migrate and data/work corresponding to those keys.
Perhaps a message could be published to a common queue indicating
which keys were re-mapped.

Cheers,
Josh

On Jan 25, 10:31 am, Emile Joubert <em... at rabbitmq.com> wrote:
> Hi Josh,
>
> On 24/01/12 23:39, Josh Stone wrote:
>
> > We're looking at using rabbitmq with the consistent hash exchange to
> > distribute workload to a set of workers with each worker subscribing
> > to one or more topics. It looks like this will work well, until a
> > worker along with its topics either comes or leaves, which will
> > disrupt the hashing.
>
> Keep in mind that when binding a queue to a consistent hash exchange you
> don't bind with a topic. You bind with a number that indicates what
> share of messages the queue will receive. Those messages can have any
> routing key. If you want workers to handle only particular topics then
> you should use a topic exchange.
>
> > Particularly, if a new worker enters and creates some topics on the
> > exchange, messages that previously would have been delivered to
> > another topic will be directed to the new topic. And if an existing
> > worker was operating on some data from a previous message and a new
> > message to operate on that data is sent to a new topic, we could have
> > a race condition.
>
> For the consistent hash exchange some keys will be unavoidably be
> remapped when the bindings change. The advantage is that not all keys
> need to be remapped. If your workers can only deal with one or a limited
> number of topics then you should not be using a consistent hash exchange.
>
> -Emile
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss