[rabbitmq-discuss] RabbitKernel

[Jon Brisbin]

To expand some (and for the benefit of the list), I guess what I'm thinking is related to a problem I see on the horizon as I build more utilities that have at their foundation AMQP messaging. Each component needs to talk to the other components running on different VMs within the cloud on different exchanges/queues and each has their own system of subscribing consumers and producing messages. This means: if I have three instances of tcServer running on a VM and each webapp is using the cloud session manager and and the server is using the JMX invoker, I've got four different ConnectionFactories, four sets of subscribers, four sets of producers (times the number of workers I have configured; I'll likely have 40 or more of these per VM). This doesn't seem terribly scalable to me, so I'm hunting for a way to use messaging on a VM for communicating to various components (that are likely unrelated) in a more reasonable way.

OSGi has always appealed to me because of module versioning, live reloading, etc... and the ROC approach that NetKernel takes to abstracting OSGi seems to offer (in my initial estimation at least) the ability to de-duplicate the various ways I'm managing connections, channels, subscribers, and producers (which is to say: each time I write a module, I try and improve it; the side-effect being each module does these things a little differently). 

I'm starting with a proof-of-concept NetKernel module that offers endpoints for publishing messages. That seems the easiest to start with. But after that, I think creating a system of consumers and resource "callbacks" would be fairly lightweight and, more than anything, better than what I've got now because it wouldn't be copied all over the place in different ways.

There seems to be a "best practice" way of managing the actual sending and receiving of messages irregardless of the application into which this functionality is injected. Once I can encapsulate that into a module, I have to figure out how IPC will happen! :/ I'm leaning toward Scala/Akka remote actors but there are more ways to go about this, I'm sure. I could also use local Redis servers that would use a variation on OSGi's "whiteboard" pattern where code inside the RabbitMQ NetKernel module would write data to a local Redis server where it could be picked up by code running in other JVMs on the same machine. I just haven't thought enough about it yet to decide which is best (my luck will be that it's actually a combination of various things).

Jon Brisbin
Portal Webmaster
NPC International, Inc.



On Jul 29, 2010, at 4:34 PM, David Wragg wrote:

> Hi Jon,
> 
> Jon Brisbin <jon.brisbin at npcinternational.com> writes:
>> Has anyone worked with RabbitMQ inside NetKernel?
>> 
>> http://www.1060research.com/netkernel/
>> 
>> I know enough about OSGI to make myself dangerous. I've worked some
>> with Apache's ESB, so when I came across NetKernel, I thought I'd at
>> least explore it a little to see if it would make our cloud
>> infrastructure easier to manage. I'd love to have some AMQP
>> integration in NetKernel to play with, so I wondered if anyone has
>> already worked on something or if there are plans to do so?
> 
> I don't think we've heard of RabbitMQ and NetKernel being used together.
> 
> I wonder how you would combine the resource oriented computing paradigm
> of NetKernel with a messaging approach?  Or were you thinking of using
> AMQP as a transport for Netkernel?
> 
> Dvid
> 
> -- 
> David Wragg
> Staff Engineer, RabbitMQ
> SpringSource, a division of VMware