<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Hi Matthew,<div><br></div><div><div><div>On Oct 13, 2011, at 12:32 AM, Matthew Sackman wrote:</div><br><blockquote type="cite"><div>Huh, that's interesting. I'm curious though - how do the properties of<br>your implementation differ from our default? - I'm not trying to<br>discourage you at all, just curious. Even if it's just for gaining<br>understanding and experience of implementing Rabbit's internal APIs,<br>it's still valuable.</div></blockquote><div><br></div><div>So my first goal was to learn more about the Rabbit's internal APIs. Besides that I haven't conducted any benchmarks to see what could be the advantage of using Bitcask over ETS.</div><div><br></div><div>According to the Basho devs they wanted to achieve this out of Bitcask:</div><div><br></div><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10.9px/normal Helvetica; ">- low latency per item read or written</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10.9px/normal Helvetica; ">- high throughput, especially when writing an incoming stream of random items</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10.9px/normal Helvetica; ">- ability to handle datasets much larger than RAM w/o degradation</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10.9px/normal Helvetica; ">- crash friendliness, both in terms of fast recovery and not losing data</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10.9px/normal Helvetica; ">- ease of backup and restore</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10.9px/normal Helvetica; ">- a relatively simple, understandable (and thus supportable) code structure and data format</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10.9px/normal Helvetica; ">- predictable behavior under heavy access load or large volume </div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10.9px/normal Helvetica; ">- a license that allowed for easy default use in Riak</div></div><div><br></div><div>From those items I think that the ability to handle datasets much larger than RAM w/o degradation could be an advantage over ETS. Still� If I understand correctly the datastore implementation I think when Rabbit start to get pressure on RAM usage it does some flushes to disk. Perhaps there could be a benefit there� not sure.</div><div><br></div><div>What I liked from this experiment is that there's no much code needed to have a different index implementation. Basho also created a LevelDB Erlang/C++ NIF, I think that can also be made a index quite easy. I've created a wrapper for it for the umbrella but didn't had time to also implement the index behaviour.</div><br><blockquote type="cite"><div>Simplest way is to set the msg_store_index_module param in your<br>rabbitmq.config file - check the rabbit.app to see how it's set, but<br>there's nothing unusual there. Also check rabbitmq-toke which<br>(re-)implements the same interface and probably changes the env slightly<br>differently... ;)<br></div></blockquote><div><br></div></div>Yes� in my plugin I'm using the rabbit_boot_step thing to set my plugin as the default index.</div><div><br></div><div>Cheers,</div><div><br></div><div>Alvaro</div></body></html>