<div dir="ltr"><div class="gmail_extra"><br><div class="gmail_quote">2013/7/24 Flavio Pompermaier <span dir="ltr"><<a href="mailto:pompermaier@okkam.it" target="_blank">pompermaier@okkam.it</a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>1) One of the N consumers receive the end-message</div><div>2) This consumer opens a new temporary feedback-queue to receive feedbacks from other consumers and post to a fanout control queue a "wait for end of processing" to all other consumers (containing also the id of the feedback-queue)</div>
<div>3) Sender ignores such message, other consumers sends a "I finished my stuff" message once finished their work to the feedback-queue (whose id is indicated in the "wait for end of processing" message)</div>
<div>4) Once that the Consumer who initiated the "stop phase" receive all the N-1 feedbacks (or after a certain timeout) it can delete the temporary feedback queue and start the map reduce job</div></blockquote>
<div><br></div><div>This sounds about right.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div></div>
<div>What is missing in this scenario is how consumers can be informed of the number of active consumers (i.e. N)..any help about this? Do you think this steps could be reasonable?</div></blockquote></div><br>This is a good question. Add to this that consumers can crash or become unavailable because of</div>
<div class="gmail_extra">network failures. Coordination over RabbitMQ alone will probably be more difficult than using</div><div class="gmail_extra">Redis for this purpose (as a "presence service", more or less), e.g. using its set data structures.</div>
<div class="gmail_extra"><br></div><div class="gmail_extra">Any consumer would add itself to a set on start and any consumer can read the set and see</div><div class="gmail_extra">who is online. I'm not very up-to-date with Redis but ideally you'd use its features to detect</div>
<div class="gmail_extra">consumers leaving.</div><div class="gmail_extra">-- <br>MK<br><br><a href="http://github.com/michaelklishin" target="_blank">http://github.com/michaelklishin</a><br><a href="http://twitter.com/michaelklishin" target="_blank">http://twitter.com/michaelklishin</a><br>
</div></div>