[rabbitmq-discuss] Custom storage backends for queues (MySQL)

Mon Jan 17 22:49:12 GMT 2011

On Jan 17, 2011, at 4:16 PM, Christian Legnitto wrote:

> Would this just store undelivered messages (I think that's what's in the mnesia db currently) or would it be a store of all messages that have entered Rabbit (more like a historical message cache)? I'm interested in the latter, preferably with Riak.

I'd think it would be trivial to do the latter by maintaining some metadata about the message (like setting a "delivered" flag).

> Now that I think about it, probably wouldn't be too hard to write a rabbit plugin that ships all messages from all exchanges over to Riak using the Riak erlang lib. Does something like that already exist?

I've tested the latest version of my webhooks plugin against Riak, so that might work for you if you're not using exclusive queues:

https://github.com/jbrisbin/rabbitmq-webhooks

It uses the HTTP API and forwards messages from queues to Riak (or any other HTTP server, for that matter).

jb

> 
> Thanks,
> Christian
> 
> On Jan 17, 2011, at 12:31 PM, Alexis Richardson <alexis at rabbitmq.com> wrote:
> 
>> Jon
>> 
>> If you have the appetite for a Riak backend, using the storage API,
>> that would be interesting to see...
>> 
>> alexis
>> 
>> 
>> 
>> On Mon, Jan 17, 2011 at 8:22 PM, Jon Brisbin <jon at jbrisbin.com> wrote:
>>> 
>>> On Jan 15, 2011, at 11:14 AM, Jerry Kuch wrote:
>>> 
>>> Exist support for custom storage backends (like MySQL) in RabbitMQ 2.2 or
>>> not?
>>> 
>>> Storage backends based on MySQL and Mnesia are currently under development.
>>> There has also been some discussion of various NoSQL stores, although no
>>> firm plans have been made.  We should have something more to say in the near
>>> future.
>>> 
>>> I think it would be cool to have messages stored in K/V stores like Riak,
>>> Couch, or Mongo and be able to subscribe to queues based on queries (like
>>> SQL views) which is something I've had on my mind recently. I dug into the
>>> core RabbitMQ code the other day to see how feasible this would be and it
>>> looks reasonably straightforward. I've been using Riak a lot lately, so I'd
>>> likely start with that and the protobuff client.
>>> What about metadata? I think that's all stored in mnesia now, right? How
>>> hard would it be to keep that data in the same storage backend?
>>> 
>>> Thanks!
>>> Jon Brisbin
>>>        Web: http://jbrisbin.com/
>>>    Twitter: @j_brisbin
>>>      Skype: jon.brisbin
>>> 
>>> _______________________________________________
>>> rabbitmq-discuss mailing list
>>> rabbitmq-discuss at lists.rabbitmq.com
>>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>> 
>>> 
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.rabbitmq.com
>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

Jon Brisbin
Portal Webmaster
NPC International, Inc.

Twitter: @j_brisbin
Skype: jon.brisbin