<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><div>On Oct 28, 2008, at 4:31 AM, Ben Hood wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>In general, you can help by either contributing code yourself or by<br>financing the reprioritization of the roadmap.<br><br>Disk overflow or queue paging is on the mid term roadmap as something<br>we are going to do, but we still need to gather requirements.<br><br>Here are a few examples:<br><br>- When do you decide to page things to disk?</div></blockquote><div><br></div><div>I'd say memory high watermark reached or, if reached before, a number</div><div>of messages in the queue.</div><br><blockquote type="cite"><div>- Is it done on memory consumption or queue depth?</div></blockquote><div><br></div><div>Both. Depends on which one is reached first, I can see use cases for</div><div>both of these triggers.</div><br><blockquote type="cite"><div>- Is this configurable per queue or across the broker?</div></blockquote><div><br></div><div>Across the broker, other AMQP implementation have a 'maximum number</div><div>of messages in a queue' as a per broker option.</div><br><blockquote type="cite"><div>- What are sensible defaults so that people who haven't even though<br> about paging don't get affected by overly-agressive defaults?</div></blockquote><div><br></div><div>If a user was not affected before it won't be affected even after because</div><div>he won't need to overflow to disk anyway. In case the limit hit is the number</div><div>of messages then simply set it to infinite when no different is specified. Then</div><div>it will only use memory watermark and this would give them more robustness</div><div>going from simply crashing the erlang VM because memory is finished to</div><div>slowing down because you are now reading from disk.</div><br><blockquote type="cite"><div>- When and how do you swap back in - is this automatic or manual?</div></blockquote><div><br></div><div>When the buffer in the disk is empty then rabbitmq can stop using it, I expect</div><div>it to deliver messages in order so once the buffer kicks in I'd queue stuff on</div><div>disk immediately.</div><br><blockquote type="cite"><div>- How do you decide when the low water mark has been reached after<br> having commenced the page-in? Do you resume flow control at this point?</div></blockquote><div><br></div><div>When the disk buffer is empty. And you simply resume working without it.</div><br><blockquote type="cite"><div>- If you do page, are you interested in last image caching?</div></blockquote><div><br></div><div>I'm not sure what this means but looks like an optimization, it's probably</div><div>interesting to have but caching can come when the system can resist</div><div>prolonged consumer downtimes.</div><div><br></div><div>Anyway being an optimization in the queue process it should be fairly</div><div>isolated in it. I can totally see keeping the current page always in memory</div><div>(if this is the optimization we are talking about), the big problem is keeping</div><div>everything in memory.</div><br><blockquote type="cite"><div>- Do you want to apply application level heuristics to selectively purge<br> overflowed queues?</div></blockquote><div><br></div><div>ActiveMQ offers basically 3 different ways to deal with the problem:</div><div>kill the queue, drop all new messages, drop old messages.</div><div><br></div><div>This can be taken care of in the publish command of rabbitmq.</div><br><blockquote type="cite"><div>- What role do TTLs play in this scenario?</div></blockquote><div><br></div><div>It's not the role of the message broker to kill single messages, it's an application</div><div>level decision. I can see a configuration option in the queue though that together<br></div><div>with the 3 options above can provide a 'kill all the messages older than X seconds'.</div><div><br></div><div>This can be taken care inside the queue itself when getting the top of the queue.</div><br><blockquote type="cite"><div>- Furthermore, we do already have a fast message persister - it's just<br> that it's geared to write as quickly as possible, not read.</div></blockquote><div><br></div><div>And this is a great to me.</div><br><blockquote type="cite"><div>- Ask yourself, if we implement paging, is are we potentially reinventing<br> a wheel that the OS has already invented?</div></blockquote><div><br></div><div>Well, sure. But erlang fails before when it cannot malloc memory.</div><br><blockquote type="cite"><div>- And as always, what do you do when your SAN fills up?<br></div></blockquote><div><br></div><div>If a 500-600GB disk fills up it means I had at least ~80 times more time to</div><div>fix the problem somehow, given an average of 8GB of memory in a webserver.</div><div>So if with 8GB of memory I had 10 hours to fix it, with a disk I have 30 days to<br></div><div>fix it, and I can even add new disks with a good disk array or filesystem to buy</div><div>me more time.</div><br><blockquote type="cite"><div>One suggestion to kick things off is to begin a more structured<br>analysis of the whole problem on the wiki and start a dedicated<br>discussion thread around this. For example, one could start a document<br>highlighting the motivation and requirements and let interested<br>parties comment on this.<br></div></blockquote><div><br></div><div>Good, I'm all for this. And I'm starting with this reply.</div><div><br></div><blockquote type="cite"><div>Having said that, whilst it may be straight forward to do a hello<br>world overflow, the devil is in the detail of all of the moving parts<br>and different scenarios that you have to account for whilst maintaing<br>the clarity of the code base. And testing it of course.<br></div></blockquote></div><div><br></div><div>Yep, this is just right.</div><br><div apple-content-edited="true"> <span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><div><span class="Apple-style-span" style="border-collapse: collapse; font-family: arial; font-size: 13px; white-space: pre-wrap; -webkit-border-horizontal-spacing: 2px; -webkit-border-vertical-spacing: 2px; ">-- </span></div><div><span class="Apple-style-span" style="border-collapse: collapse; font-family: arial; font-size: 13px; white-space: pre-wrap; -webkit-border-horizontal-spacing: 2px; -webkit-border-vertical-spacing: 2px; ">Valentino Volonghi aka Dialtone</span></div><div><span class="Apple-style-span" style="border-collapse: collapse; font-family: arial; font-size: 13px; white-space: pre-wrap; -webkit-border-horizontal-spacing: 2px; -webkit-border-vertical-spacing: 2px; ">Now running MacOS X 10.5</span></div><div><span class="Apple-style-span" style="border-collapse: collapse; font-family: arial; font-size: 13px; white-space: pre-wrap; -webkit-border-horizontal-spacing: 2px; -webkit-border-vertical-spacing: 2px; ">Home Page: <a href="http://www.twisted.it">http://www.twisted.it</a> </span></div><div><span class="Apple-style-span" style="border-collapse: collapse; font-family: arial; font-size: 13px; white-space: pre-wrap; -webkit-border-horizontal-spacing: 2px; -webkit-border-vertical-spacing: 2px; "><a href="http://www.adroll.com">http://www.adroll.com</a></span></div></div></div></span></div></span></div></span> </div><br></body></html>