[rabbitmq-discuss] RabbitMQ Consistent Hashing Exchange

[Tim Watson]

John,

This is very helpful, thanks.

On 4 Jul 2013, at 11:48, John Turner <john.turner at thoughtforge.net> wrote:

>  I'm very happy to help you with requirements here so if a call would help let me know and I'll set it up.
> 

Possibly, though I'll wait until I've got specific questions before taking up too
much of your time. I've also got to be careful that I don't design the feature with only 1 use case in mind. :)

> Do you have a timeline for this feature?

Not really. We're still I'm the analysis/design phase ATM, though I hope to start cutting code this week. Ideally i'd like to get it into the next feature release but I can't commit to that and there's no official timeline on that release either. If you want a complete guess then I'd expect to see another feature release in the next couple of months, but that really is "finger in the air" and I'm not responsible for decision making about releases anyway.

> Do you want all future messages delivered to another random consumer? (Yes)

Good, that's what we had in mind too.

>  
> What about un-acked messages that were already delivered - we plan on redelivering these (Agree) - and how do you plan on explicitly terminating a group? (This is not a requirement.  Groups should only be terminated by a connection closing either explicitly or due to failure)
>  

We will probably add this for the initial release anyway, since most jms message group implementations do so - not that we're tied to what jms does, but it's a useful guideline.


> Would a special message (or header) given by the producer suit that purpose? (If people want this functionality I would suggest this is the best way.  

That seems to be the consensus among other group implementations too.

> You might also consider giving the consumer the ability to terminate a message group but we don't have a specific use case for this.  It might be something for future releases.)
>  

I tend to agree. The consumer can already reject messages (or batches thereof) anyway, and can disconnect if its intention is to disassociate itself from the group.

I suspect that making the semantics of republication configurable (or policy driven) may improve this area, perhaps by supporting some kind of "unit of work" facility.

> Do you have consumers that want to recieve more than one group in parallel? (Absolutely)

Good to know. What ordering guarantees would you expect in that case?


> And what about producers that go away - do you close the group when they disconnect or keep it open indefinitely, or use a timeout? (I would keep it open indefinitely because multiple producers may be contributing messages to a single group.

Good point.

>  You have some challenges around clean up here though.  It could be reasonable to say that when all messages in a group have been processed that the group closes after a timeout.  When subsequent messages arrive the group is recreated.)

If the consumer wasn't notified about the group closure, that could make it difficult to reason about message ordering, especially if we wanted to support some kind of msg-group-seq-num or associate message seq num within a group.

Cheers,
Tim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130708/cce5a62d/attachment.htm>