[rabbitmq-discuss] Custom storage backends for queues (MySQL)

Mon Jan 17 22:45:25 GMT 2011

Christian

Congrats on Pulse btw!

On Mon, Jan 17, 2011 at 10:16 PM, Christian Legnitto
<clegnitto at mozilla.com> wrote:
> Would this just store undelivered messages (I think that's what's in the mnesia db currently) or would it be a store of all messages that have entered Rabbit (more like a historical message cache)? I'm interested in the latter, preferably with Riak.

The latter.

In fact the former are stored in our own persistent message heap
created by the team:
http://www.rabbitmq.com/blog/2010/08/27/growing-up/

Mnesia is used for things like bindings and other metadata.

> Now that I think about it, probably wouldn't be too hard to write a rabbit plugin that ships all messages from all exchanges over to Riak using the Riak erlang lib. Does something like that already exist?

No but as you point out it would be doable.  But a closer binding, at
the storage API layer, could be more useful - no?

alexis

> Thanks,
> Christian
>
> On Jan 17, 2011, at 12:31 PM, Alexis Richardson <alexis at rabbitmq.com> wrote:
>
>> Jon
>>
>> If you have the appetite for a Riak backend, using the storage API,
>> that would be interesting to see...
>>
>> alexis
>>
>>
>>
>> On Mon, Jan 17, 2011 at 8:22 PM, Jon Brisbin <jon at jbrisbin.com> wrote:
>>>
>>> On Jan 15, 2011, at 11:14 AM, Jerry Kuch wrote:
>>>
>>> Exist support for custom storage backends (like MySQL) in RabbitMQ 2.2 or
>>> not?
>>>
>>> Storage backends based on MySQL and Mnesia are currently under development.
>>>  There has also been some discussion of various NoSQL stores, although no
>>> firm plans have been made.  We should have something more to say in the near
>>> future.
>>>
>>> I think it would be cool to have messages stored in K/V stores like Riak,
>>> Couch, or Mongo and be able to subscribe to queues based on queries (like
>>> SQL views) which is something I've had on my mind recently. I dug into the
>>> core RabbitMQ code the other day to see how feasible this would be and it
>>> looks reasonably straightforward. I've been using Riak a lot lately, so I'd
>>> likely start with that and the protobuff client.
>>> What about metadata? I think that's all stored in mnesia now, right? How
>>> hard would it be to keep that data in the same storage backend?
>>>
>>> Thanks!
>>> Jon Brisbin
>>>         Web: http://jbrisbin.com/
>>>     Twitter: @j_brisbin
>>>       Skype: jon.brisbin
>>>
>>> _______________________________________________
>>> rabbitmq-discuss mailing list
>>> rabbitmq-discuss at lists.rabbitmq.com
>>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>>
>>>
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.rabbitmq.com
>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>