<div dir="ltr">Hi,<div><br></div><div style>I think this solution can address your problem:</div><div style><br></div><div style>- Create queues 1, 2, .... N</div><div style>- Bind queues to the CHX</div><div style>- publish the queue names to a queue called "lock" for example.</div>
<div style><br></div><div style>Start consumer A, and make it consume _just one message_ from queue "lock", in ack mode. That message will contain the queue name from where that consumer needs to get messages. </div>
<div style><br></div><div style>Then you can start consumer B and do the same, so you get the name of the next queue where to get the messages.</div><div style><br></div><div style>Because you are using the CH exchange, messages witch certain routing key will always end up in the same queue. Therefore you can also process messages in order.</div>
<div style><br></div><div style>If you need to stop one of those consumers, they need to basic_reject(requeue=true) the message they first got from the "lock" queue so another consumer can use that queue. If one of those consumers crashes then the message with the queue name will get requeued. NOTE: you never ack any message from the "lock" queue.</div>
<div style><br></div><div style>Does this help?</div><div style><br></div><div style>Regards,</div><div style><br></div><div style>Alvaro</div><div style><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">
On Fri, Mar 15, 2013 at 4:46 PM, SteveO <span dir="ltr"><<a href="mailto:steveolivier22@gmail.com" target="_blank">steveolivier22@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Subsequent bindings would have no affect on routing keys that have already<br>
been seen. When a message with a new routing key arrives, new bindings (or<br>
even existing ones) could be used for the new routing key.<br>
<br>
Once I looked beyond the specific requirements of our application, and<br>
thinking about distribution, I completely realize the behavior I seek breaks<br>
the nature of the CHX. I understand why it's not doing what I want. So I<br>
guess what I am looking at is some crazy feature or a completely different<br>
exchange altogether.<br>
<br>
Here are the high level requirements:<br>
1. Need to scale<br>
2. Need to process messages with the same routing key in order<br>
<br>
Here is what is highly desired:<br>
1. Elastic scaling of queues<br>
2. No back-channel consumer/producer communication<br>
<br>
A single queue with multiple consumers gives me req1, but not req2. A single<br>
queue with a single consumer gives me req2, but not req1. By way of a CHX,<br>
multiple queues and multiple consumers gets me both req1 and req2. However<br>
with the CHX, if I accomplish scaling up by adding another queue and<br>
consumer, I can run into situations where I can't guarantee req2. That's<br>
because I can get messages with the same routing key in more than one queue.<br>
For an alternative method of scale, I can accomplish req1 and req2 by adding<br>
another CHX. However, with our application that would require some<br>
back-channel communication between producers and consumers which is highly<br>
undesirable.<br>
<br>
A type of "pick and stick" method could work well for us. Our application is<br>
processing messages for events that span a finite duration of time. These<br>
events could last a few minutes, a few hours or a few days. Messages<br>
pertaining to a specific event are signified by a GUID in the routing key.<br>
Once the event is done, messages are consumed and processed, the routing key<br>
for that event is never seen again. I just can't trust the timeliness of my<br>
consumers, and I have to guarantee message processing order. That's why I<br>
have to ensure that messages with the same routing key only exist in one<br>
queue at a time.<br>
<br>
I suspect I could have discussed this situation much better if had simply<br>
stated that rather than questioning the CHX : )<br>
<br>
Is this a crazy feature request for a switch to turn this on in the CHX?<br>
Am I looking at a custom exchange?<br>
Something else?<br>
<br>
Thanks again.<br>
<br>
s<br>
<br>
<br>
<br>
--<br>
View this message in context: <a href="http://rabbitmq.1065348.n5.nabble.com/Confirm-consistent-hash-exchange-behavior-tp25458p25471.html" target="_blank">http://rabbitmq.1065348.n5.nabble.com/Confirm-consistent-hash-exchange-behavior-tp25458p25471.html</a><br>
<div class="HOEnZb"><div class="h5">Sent from the RabbitMQ mailing list archive at Nabble.com.<br>
_______________________________________________<br>
rabbitmq-discuss mailing list<br>
<a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a><br>
<a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a><br>
</div></div></blockquote></div><br></div>