[rabbitmq-discuss] Caching proxy for cloud HA

[Jon Brisbin]

On May 25, 2010, at 4:51 PM, David Wragg wrote:

> What you describe doesn't seem feasible for AMQP, at least in
> general.  Matthias mentioned a few reasons, but I expect it's possible
> to come up with many more objections.

I wasn't really looking at AMQP specifically. In my mind, a TCP proxy like HAProxy is just that. It hooks two bytes streams together. On one end is a client that needs to talk to a broker. On the other end are n number of brokers who can talk to the client. The HA in this case is very application and protocol agnostic. It would seem to the broker that a client was directly connected.

> It might be possible to get further by making application-specific
> assumptions.  For example, if you know that certain messages are part of
> idempotent or cacheable request-response exchanges.  But once you start
> making such assumptions, it's questionable what the result has to do
> with AMQP.  Why not just use an application-specific protocol instead,
> or HTTP, which explicitly covers issues such as cacheability and
> idempotency?

That just doesn't sound like fun at all! :)

I'm not interested in re-inventing new protocols to transmit my data. I'm interested in starting n number of clones of a virtual machine on which I have RabbitMQ installed and providing one (or all) of those brokers to a client that needs to send or consume messages and shielding the client from having to know who is active and who isn't. It's a basic cloud scenario where membership in a functional group is dynamic and changeable over time. I don't see that this has anything at all to do with AMQP as a protocol. I'm thinking strictly in terms of TCP byte streams to whatever broker is active. Long-term (and pie-in-the-sky) it would be nice to have all these brokers work together as a unit, but I know that's a long way off. Basic clustering works for me at the size and amount of traffic I give it, so using HAProxy to front these servers seems to approximate very closely what I'm wanting here. What I was thinking was basically HAProxy (which I hadn't heard of until recently) but RabbitMQ-specific.

It might even be possible to write an HAProxy handler so you could do acl's for AMQP like you do for TCP and HTTP! :) With dynamic reconfiguration, that would be killer...

Jon Brisbin
Portal Webmaster
NPC International, Inc.