[rabbitmq-discuss] Confirm consistent hash exchange behavior

[SteveO]

Subsequent bindings would have no affect on routing keys that have already
been seen. When a message with a new routing key arrives, new bindings (or
even existing ones) could be used for the new routing key.

Once I looked beyond the specific requirements of our application, and
thinking about distribution, I completely realize the behavior I seek breaks
the nature of the CHX. I understand why it's not doing what I want. So I
guess what I am looking at is some crazy feature or a completely different
exchange altogether.

Here are the high level requirements:
1. Need to scale
2. Need to process messages with the same routing key in order

Here is what is highly desired:
1. Elastic scaling of queues
2. No back-channel consumer/producer communication

A single queue with multiple consumers gives me req1, but not req2. A single
queue with a single consumer gives me req2, but not req1. By way of a CHX,
multiple queues and multiple consumers gets me both req1 and req2. However
with the CHX, if I accomplish scaling up by adding another queue and
consumer, I can run into situations where I can't guarantee req2. That's
because I can get messages with the same routing key in more than one queue.
For an alternative method of scale, I can accomplish req1 and req2 by adding
another CHX. However, with our application that would require some
back-channel communication between producers and consumers which is highly
undesirable.

A type of "pick and stick" method could work well for us. Our application is
processing messages for events that span a finite duration of time. These
events could last a few minutes, a few hours or a few days. Messages
pertaining to a specific event are signified by a GUID in the routing key.
Once the event is done, messages are consumed and processed, the routing key
for that event is never seen again. I just can't trust the timeliness of my
consumers, and I have to guarantee message processing order. That's why I
have to ensure that messages with the same routing key only exist in one
queue at a time.

I suspect I could have discussed this situation much better if had simply
stated that rather than questioning the CHX : )

Is this a crazy feature request for a switch to turn this on in the CHX?
Am I looking at a custom exchange?
Something else?

Thanks again.

s



--
View this message in context: http://rabbitmq.1065348.n5.nabble.com/Confirm-consistent-hash-exchange-behavior-tp25458p25471.html
Sent from the RabbitMQ mailing list archive at Nabble.com.