[rabbitmq-discuss] RabbitMQ-based vcloud session clustering

Wed Apr 7 22:55:21 BST 2010

Jon

Very interesting.  More below.

On Wed, Apr 7, 2010 at 8:07 PM, Jon Brisbin
<jon.brisbin at npcinternational.com> wrote:
>
> On Apr 7, 2010, at 1:21 PM, Alexis Richardson wrote:
>
>> I imagine most folks would treat VMs as machines.
>
> I used to call my servers "boxes" or "machines". Now I refer to my VMs as "nodes", which I feel more accurately describes them. They are task-oriented instances of Ubuntu Linux (technically-speaking) but functionally, they are of a particular type (e.g. "tcServer instance", "RabbitMQ broker", etc...).

Understood.

>> But perhaps the VMware product suite could be helpful too....
>
> We're using the free stuff, so we don't get much help there. :) My original Nirvana was to have an environment that shrunk and grew based on user demand. When more resources were needed, the system could start additional VMs. That's still sort of my goal, but I've softened on the dynamism, which is still not well-supported in most third-party applications we use. Apache, for instance, is a beast that is anything but cloud-friendly. When my application servers start and stop, I have to update a central "master" node and a cron job keeps the ProxyMember directives up-to-date and restarts Apache if it sees membership change. RabbitMQ needs similar hand-holding because it doesn't seem well-equipped to exist within a dynamic and complex environment where it is only a citizen. I'm tackling this in my vcloud tomcat session manager by having each node keep a list of all active sessions and the node they were on last.

This is fascinating.  Are you saying, for example, that you would like
RabbitMQ to deal with group membership and discovery in a more
'elastic' way?  What about things like this:
http://cacm.acm.org/blogs/blog-cacm/83396-errors-in-database-systems-eventual-consistency-and-the-cap-theorem/fulltext

> As more developers adopt cloud topologies, these issues will become more and more critical. I still want to be able to reclaim CPU resources that are being locked up by running more VMs than I really need at any given moment simply because other services can't handle dynamic membership well. I just don't know when I'll be able to achieve that. I just got off a conference call with some industry analysts and we were talking about this very topic.

It's hot, see for example:

http://gigaom.com/2009/10/25/the-future-is-big-data-in-the-cloud/
http://kirkwylie.blogspot.com/2010/03/qcon-london-2010.html

alexis

>
> Jon Brisbin
> Portal Webmaster
> NPC International, Inc.
>
>
>
>