[rabbitmq-discuss] RabbitMQ Consistent Hashing Exchange

[Tim Watson]

We've got an open bug to provide a 'message groups' feature like this and are hoping to get it ready for a feature release ASAP.

Just out of interest, for your use-case, how would you expect failover to work. Do you want all future messages delivered to another random consumer? What about un-acked messages that were already delivered - we plan on redelivering these - and how do you plan on explicitly terminating a group? Would a special message (or header) given by the producer suit that purpose? What about consumers that don't disconnect but indicate they'd like to stop processing that group of messages? Do you have consumers that want to recieve more than one group in parallel? And what about producers that go away - do you close the group when they disconnect or keep it open indefinitely, or use a timeout?

None of those things are very clear to me from the activemq docs, so hearing from a real world user would be nice! :)

Cheers,
Tim

On 3 Jul 2013, at 11:57, John Turner <john.turner at thoughtforge.net> wrote:

> Hi All,
> 
> I'm very familiar with JMS and have used the GroupID extension to provide processing affinity to a single consumer.  For further information on how this works see this ActiveMQ documentation: http://activemq.apache.org/message-groups.html
> 
> This solves the following requirement:
> 
> * Messages are distributed to a queue with a GroupID header.
> * The broker guarantees that messages with a specific GroupID (say '1234') will only be processed by a single consumer.
> * If the consumer fails (i.e. the connection is broken or closed), the broker will allow another consumer to process messages with that GroupID
> 
> Using RabbitMQ and the Consistent Hashing Plugin does not not appear to solve the problem because if we are using static (durable) queues, the consumer failover mechanism needs to be implemented outside the server.  Does anyone have any insight into this problem they could share?
> 
> An alternative would be to extend the queue to ensure apply the same consistent hashing mechanism to the actual queue rather than the broker.  Does anyone know if it si possible to extend the queue implementation? 
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss