[rabbitmq-discuss] Are queues replicated across a cluster?

[Bill Moseley]

On Fri, Jan 14, 2011 at 9:08 AM, Matthew Sackman <matthew at rabbitmq.com>wrote:

>
> Entirely up to you - resources are visible throughout the cluster and so
> any sane load balancing should be just fine. If they all connect to the
> same node then you'll likely find that all resources exist solely on
> that node and thus the other nodes are not being used at all. Thus
> really, you want to make sure that the clients which create resources
> (queues in particular) are connected across all the nodes.
>
> The other thing to consider is the inter-node hops required to route
> messages. Thus you may wish to make sure that if client 1 is
> predominantly sending messages to or consuming messages from queue 1,
> that client 1 is connected to the same node on which queue 1 resides.
>

Hum.  There's always trade-offs.

The top concern is that a producer (and consumer) can connect to a node.
 What looks like a good choice is HAProxy in front of a set of RabbitMQ
nodes.  And maybe a pair of HAProxy machines with Heartbeat to avoid the
single point of failure.  Configuration is easier because workers just talk
to one HAProxy.

But, then there's no control over which node a producer or consumer connects
to.  So, then there's going to be be some (probably half) cross-node
messages.

Perhaps if the point of the second node is for HA then should just configure
HAProxy to only use the second node as a hot-backup and make both nodes disk
nodes.  The nature of a backup is that it must be able to handle all
traffic, anyway.

If we design with the idea messages might get dropped, and always declare
exchanges, queues, and bindings on every connection then maybe there's no
need for a cluster or using the Pacemaker HA example and just run
independent RabbitMQ instances with a method of fail-over.


Thanks very much for the help, Matthew.


Would I be wrong to assume that these configuration issues have been solved
by many people already?


-- 
Bill Moseley
moseley at hank.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20110114/36bea295/attachment.htm>