[rabbitmq-discuss] RabbitMQ Clustering with Federation

[Simon MacMullen]

No, at the moment that's only for federation. This may change in future.

Cheers, Simon

On 01/08/2013 4:53PM, Robert Parker wrote:
> Ah, so is there a way to alleviate the need for both the publisher (in
> our case, mass transit) or the consumer (a .net reporting app) to use
> load balancer VIP for client connections to any cluster using a compound
> connection string via the .NET API?  That is, is it possible to use
> something similar to
>   '{"uri":["amqp://remote1-__node1","amqp://remote1-node2"]}' on the
> client side for either publishing or consuming such that we don't need
> an external load balancer at all to achieve high availability for BOTH
> client connections and federation connections?
>
>
> Thanks,
> Robert
>
>
> On Thu, Aug 1, 2013 at 10:34 AM, Simon MacMullen <simon at rabbitmq.com
> <mailto:simon at rabbitmq.com>> wrote:
>
>     On 31/07/2013 5:35PM, Robert Parker wrote:
>
>         I'm having some trouble getting the right architecture configured to
>         support rabbitmq federation in a clustered high availability
>         environment.  Our basic setup is four clusters of two nodes each
>         which
>         are separated by a WAN.
>
>         "home" is the "home" cluster and "remote1", "remote2" and
>         "remote3" are
>         remote clusters.  There are a lot of queues being created and
>         published
>         to via the API identically on all clusters but I only want to
>         federate
>         ONE of the queues, which is called "EventListener" back to the
>         "home"
>         cluster from all the "remote" clusters.  I want messages
>         published to
>         this queue on each remote cluster to be federated to the home
>         cluster to
>         be consumed by a separate event listener service that only lives
>         at the
>         "home" site.
>
>
>     Just to be clear: you are aware that (currently) it's only
>     *exchanges* which can be federated, not queues?
>
>         Each cluster is configured with:
>
>
>     <snip>
>
>         We have configured three hardware load balancer VIPs that load
>         balance
>         connections on port 5672 between both nodes in each cluster, and
>         have
>         configured federation upstreams to use these and also configured our
>         application to use these VIPs for queue/exchange creation and for
>         publishing messages.  Here's our federation config script thats
>         run on
>         node1 of the home cluster only:
>
>
>     <snip>
>
>         However I am noticing that the "rabbitmqctl eval
>         'rabbit_federation_status:__status()." command only shows federation
>         config on node2 of the home cluster despite having been
>         configured on node1:
>
>
>     Federation works by establishing link(s) from the downstream to the
>     upstream(s). You can define the upstreams on any node in the
>     downstream cluster (the configuration is cluster-wide) and the
>     cluster will arbitrarily start the link processes on any node (and
>     will migrate them around the cluster if nodes go down).
>
>     *However*, rabbit_federation_status:__status() will only report
>     links that are running on the local node. This is probably not well
>     documented. Hmm. Let's fix that.
>
>     (The HTTP API exported by rabbitmq_federation_management will report
>     links on all nodes.)
>
>     <snip>
>
>         And I'm noticing that messages published to the queues via the load
>         balancer VIPs to the remote clusters are not showing up in the
>         "EventListener" queue of the home cluster.
>
>
>     Note that you have federated an exchange called "EventListener", not
>     a queue. So if you bind a queue to the exchange "EventListener" on
>     home, and publish to the exchange "EventListener" on remote1/2/3
>     then you should see messages turning up in the queue on home. Is
>     that what you're doing?
>
>         Is this not the right way to set this up?
>
>
>     It sounds like you're basically right, apart from the exchange vs
>     queue thing.
>
>         Is it possible to configure
>         this without using an external load balancer and still achieve high
>         availability?
>
>
>     Since 3.1.0, yes. You can specify multiple URIs in a single
>     upstream, and federation will connect to a randomly chosen one (and
>     keep retrying different ones if that one goes down). Use something like:
>
>     rabbitmqctl set_parameter federation-upstream remote1-upstream
>     '{"uri":["amqp://remote1-__node1","amqp://remote1-node2"]__,"expires":3600000}'
>
>         If I federate to a single node of a cluster directly,
>         what happens if that node goes down?
>
>
>     If you only specify one URI, it will keep retrying until that node
>     comes back up.
>
>         Do I have to configure redundant
>         federation between every node of home cluster to every node of each
>         remote cluster?
>
>
>     No - since the config is shared across the cluster. So you can set
>     up the upstreams on one node of the home cluster, pointing to all
>     nodes of the remote clusters.
>
>         If I don't use a load balancer VIP to publish to from my
>         application, if the single node of the cluster I've configured goes
>         down, it wont help me that the other node is still available.
>
>
>     Not sure what you mean here. Hopefully I've explained enough now.
>
>     Cheers, Simon
>
>     --
>     Simon MacMullen
>     RabbitMQ, Pivotal
>
>

-- 
Simon MacMullen
RabbitMQ, Pivotal