[rabbitmq-discuss] About the new persistor
matthew at lshift.net
Wed Mar 10 10:37:59 GMT 2010
On Wed, Mar 10, 2010 at 06:22:52PM +0800, Alvaro Videla wrote:
> After checking the code of the new persister, I saw that It can be used as a K+V store, with memory cache and persistence to disk. Please correct me if I'm wrong.
You're not quite right there. The new persister is much more than just
the msg_store module. The c. 7000 new lines are in:
Plus changes to rabbit_amqqueue_process.erl amongst others.
It really depends quite what you define as the "new persister". You're
not wrong though, in that the msg_store can be used as a K+V store (with
reference counting). However, it is very carefully designed to work
best with the access patterns that queues provide - it may not do so
well on randomised patterns.
> So, it would be nice if the module rabbit_msg_store can be used standalone.
> In which other modules it depends?
xref is probably the best way to find out. But as a minimum:
And of course if you really want zero-RAM-cost-per-entry you should be
using the rabbitmq-toke plugin which pushes the index into Tokyo
Cabinet, which you'll also need.
> What do you guys think about it?
Not sure really. It hasn't really been designed with that in mind, so I
have no idea how well it'll work.
> I believe it could help easing the benchmarking of the persister too.
I don't really agree there - the queue_index also plays a crucial role
and writes to disk too. We've done and are doing extensive system-wide
testing, and as you've no doubt seen in rabbit_tests, there are some
large tests to stress the msg_store on its own.
More information about the rabbitmq-discuss