<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><div><div>On Oct 1, 2010, at 8:51 AM, Alexis Richardson wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><p>Why can't you use a checksum instead? Each time you create a set of n subtasks from some task T, attach a fraction m/n to each subtask where m is the fraction attached to T. Start with m equals 1. The sum of the fractions will always be 1. No need for shared counters...<br></p></blockquote><div><br></div>Wouldn't I have to know how many subtasks I'll create total when the first subtask goes out? I don't know that in this case. </div><div><br></div><div>Could you give me an example of what you're thinking?</div><div><br></div><div>jb</div><div><br><blockquote type="cite"><p>
</p><div><br class="webkit-block-placeholder"></div><blockquote type="cite">On Oct 1, 2010 3:35 PM, "Jon Brisbin" <<a href="mailto:jon.brisbin@npcinternational.com">jon.brisbin@npcinternational.com</a>> wrote:<br><br><div style="word-wrap:break-word">I'm also wondering if anyone uses counts to determine when a job is finished or not. By that I mean, increment a counter for every outgoing message and decrement the counter when a response is received. In the case of a map/reduce job, I'd need to do something like:<div>
<br></div><div>SQL -> Map phase = +1 (per row)</div><div>Map phase -> Reduce phase = -1 (that we got the original msg) +1 * (num of emit's)</div><div>Reduce phase -> Response|ReReduce = -1 (for emit's) +1 (for response/rereduce)</div>
<div>[ReReduce -> Response] = -1 +1 (for sending response)</div><div>Response = -1</div><div><br></div><div>Essentially, each step would decrement a counter for the incoming message and increment the counter for the outgoing message. A reduce phase might decrement the counter 1000 times and increment it once. But since the map phase incremented it 1000 times prior, the count after map/reduce would be "1". The response listener would then decrement the counter when it processed the response, see that it's now zero, and know to continue.</div>
<div><br></div><div>If my goal is to beat processing times on the AS/400 when doing large financial calculations (daily acct'g reports take several hours to generate), I can't really depend on timeouts to make sure I've gathered all my results. I want the job to return as soon as results are ready. I'd like to go to management and show them a 2 hr -> 15 min improvement by using parallel processing.</div>
<div><br></div><div>I'm just wondering if using ZooKeeper or similar to do distributed, synchronized counters will have enough atomicity to not miss a count incr/decr. If I miss even one, I'm screwed because it'll never get back to zero (or get there prematurely).</div>
<div><br></div><div>I need a sentence with a question mark or this will definitely go unanswered: are message counters like this a good way to monitor asynchronous, distributed processing state?</div><div><br></div><div>Thanks! :)<p>
<font color="#500050">
Jon Brisbin
Portal Webmaster
NPC International, Inc.
</font></p><p><font color="#500050">On Oct 1, 2010, at 8:11 AM, Jon Brisbin wrote:
> I had not really looked at the spring integration ...</font></p></div></div><br>_______________________________________________<br>
rabbitmq-discuss mailing list<br>
<a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a><br>
<a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a><br>
<br></blockquote><div><br class="webkit-block-placeholder"></div>
_______________________________________________<br>rabbitmq-discuss mailing list<br><a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a><br>https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss<br></blockquote></div><br><div>
<span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Palatino; font-size: 14px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Times; font-size: 14px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><div><font class="Apple-style-span" face="Palatino" size="4"><span class="Apple-style-span" style="font-size: 14px; "><br class="Apple-interchange-newline">Thanks!</span></font></div><div><font class="Apple-style-span" face="Palatino" size="4"><span class="Apple-style-span" style="font-size: 14px; "><br></span></font></div><div><font class="Apple-style-span" face="Palatino" size="4"><span class="Apple-style-span" style="font-size: 14px; ">J. Brisbin</span></font></div><div><font class="Apple-style-span" face="Palatino" size="4"><span class="Apple-style-span" style="font-size: 14px; "><a href="http://jbrisbin.com/">http://jbrisbin.com/</a></span></font></div><div><font class="Apple-style-span" face="Palatino" size="4"><span class="Apple-style-span" style="font-size: 14px; "><br></span></font></div><div><font class="Apple-style-span" face="Palatino" size="4"><span class="Apple-style-span" style="font-size: 14px; "><br></span></font></div></div><font class="Apple-style-span" face="Palatino" size="4"><span class="Apple-style-span" style="font-size: 14px; "><br></span></font></div></span><br class="Apple-interchange-newline"></span><br class="Apple-interchange-newline"></span>
</div>
<br></div></body></html>