jon at jbrisbin.com
Tue Sep 7 20:33:56 BST 2010
Thanks for the excellent thoughts here! :) Comments below...
On Sep 7, 2010, at 2:02 PM, darrencruse wrote:
> a. Are you familiar with the new NKP protocol? Sounds like you should at
> least take a look if not - it's very much about cloud computing using
> NetKernel. They've built a high speed protocol using Apache Etch for
> allowing multiple NetKernel's to work together.
> And they're latest downloads include an example doing a pub/sub model using
> NetKernel with the NKP Protocol.
This sounds very good, indeed! I'll be investigating this more right away.
> b. But what you said about looking at the JMS source as a reference for
> your RabbitMQ Transport was what I was going to suggest you beat me to it.
Haven't actually written any code on this yet. Too many other things demanding my attention.
I'm reworking my cloud job scheduler using Spring AMQP right now, so it might actually be a perfect time to revisit this in earnest and see if I can make something work. If you have a need for something similar, I love to collaborate. :)
> c. I didn't know enough to know is there definitely not a JMS Provider for
> RabbitMQ (probably a dumb question from a brief google it sounds like maybe
> there's not). If there was I wondering if in fact you could just use
> NetKernel's JMS Transport with RabbitMQ.
IMHO it's just as well there isn't a JMS provider for RabbitMQ. ;)
I like AMQP because it's truly cross-platform. I can write producers in Python and consumers in Java. Or Ruby. Or Erlang. :)
> d. From your arch drawing I was surprised you were talking about Scala
> Actors - why wouldn't you just run NetKernel on both JVMs? i.e. The
> single-within-a JVM scala actor model seems redundant with NetKernel's
> dymamic request resolution between endpoints within a single NetKernel, and
> distributed actors likewise seem redundant with using your RabbitMQ
> Transport and possibly using asynchronous requests. It seems to me the real
> beauty whether using NetKernel or actors would be the elegance of choosing
> one approach and using that consistently throughout the architecture - it
> wasn't clear to me why you'd want to mix both. But I'm probably just not
> following what you were after - are you integrating existing systems this
> way that already use scala actors or something?
I guess that's a little confusing because I don't fully know what I want! :)
The thing that prompted my looking into NetKernel was the frustration with having multiple copies of all the connections, channels, consumers, etc... in a Tomcat session manager I wrote for private clouds. Since I run multiple instances of Tomcat/tcServer per virtual machine (VMware virtual machine in this case), it seemed silly to me to be running, on a single "box", several instances of my session manager (one for each webapp) times the number of tcServer instances I'm running (usually 3) when all I need on that Linux box is a single manager for messages and a single cache that can service multiple tcServer instances without all the resource usage involved in duplicating everything. The better use I can make of those small VMs, the more traffic I can funnel through them. I need a way to communicate from a client in a session manager running in a Tomcat VM to the AMQP manager that's doing the message sending and session object mangling. Scala actors seemed to make sense here because they're friendly to the kind of cloud-scale distributed computing I'm trying to do but are still Java compatible. I wasn't sure I could get the same kind of ease of component communication by embedding NetKernel inside each Tomcat (again, I was concerned about unnecessary duplication of resources, whether it's NetKernel or RabbitMQ client stuff) considering a session manager is instantiated per webapp.
> e. Likewise I noticed you highlighted the use of cron on the one box, but
> NetKernel has nice cron like capabilites based on quartz. So again it seems
> like there'd be more synergy by using NetKernel for those needs too(?)
I'd prefer quartz, actually. I'd gladly stay within the context of NetKernel if I could.
In general I use the term "cron" to refer to a scheduled, recurring job of some kind rather than the UNIX "cron" specifically. :)
> Just some thoughts fwiw (again I'm not a RabbitMQ expert and I didn't look
> closely at your thread so I may be missing some things in what you're
I really appreciate your thoughts! :) You've inspired me to get back in there and check out the NKP stuff and see if that will be a good fit. I've kind of hit a brick wall on another project I'm working on, so I think I'll get back into the job scheduler code today.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the rabbitmq-discuss