[rabbitmq-discuss] RabbitMQ Clustering with Federation
Simon MacMullen
simon at rabbitmq.com
Thu Aug 1 17:23:43 BST 2013
That looks correct to me. But note that support for multiple URIs was
only added in 3.1.0 - which version are you running?
Cheers, Simon
On 01/08/2013 5:15PM, Robert Parker wrote:
> Thanks.
>
> Can you confirm the exact syntax of that compound federation upstream
> config? I'm getting:
>
> rabbitmqctl set_parameter federation-upstream remote1-upstream
> '{"uri":["amqp://remote1-node1","amqp://remote1-node2"],"expires":3600000}'
>
> Setting runtime parameter "remote1-upstream" for component
> "federation-upstream" to
> "{\"uri\":[\"amqp://remote1-node1\",\"amqp://remote1-node2\"],\"expires\":3600000}"
> ...
> Error: Validation failed
>
> uri should be binary, actually was [<<"amqp://remote1-node1">>,
> <<"amqp://remote1-node2">>]
>
>
>
> On Thu, Aug 1, 2013 at 10:55 AM, Simon MacMullen <simon at rabbitmq.com
> <mailto:simon at rabbitmq.com>> wrote:
>
> No, at the moment that's only for federation. This may change in future.
>
> Cheers, Simon
>
>
> On 01/08/2013 4:53PM, Robert Parker wrote:
>
> Ah, so is there a way to alleviate the need for both the
> publisher (in
> our case, mass transit) or the consumer (a .net reporting app)
> to use
> load balancer VIP for client connections to any cluster using a
> compound
> connection string via the .NET API? That is, is it possible to use
> something similar to
>
> '{"uri":["amqp://remote1-____node1","amqp://remote1-node2"]__}'
> on the
>
> client side for either publishing or consuming such that we
> don't need
> an external load balancer at all to achieve high availability
> for BOTH
> client connections and federation connections?
>
>
> Thanks,
> Robert
>
>
> On Thu, Aug 1, 2013 at 10:34 AM, Simon MacMullen
> <simon at rabbitmq.com <mailto:simon at rabbitmq.com>
> <mailto:simon at rabbitmq.com <mailto:simon at rabbitmq.com>>> wrote:
>
> On 31/07/2013 5:35PM, Robert Parker wrote:
>
> I'm having some trouble getting the right architecture
> configured to
> support rabbitmq federation in a clustered high
> availability
> environment. Our basic setup is four clusters of two
> nodes each
> which
> are separated by a WAN.
>
> "home" is the "home" cluster and "remote1", "remote2" and
> "remote3" are
> remote clusters. There are a lot of queues being
> created and
> published
> to via the API identically on all clusters but I only
> want to
> federate
> ONE of the queues, which is called "EventListener" back
> to the
> "home"
> cluster from all the "remote" clusters. I want messages
> published to
> this queue on each remote cluster to be federated to
> the home
> cluster to
> be consumed by a separate event listener service that
> only lives
> at the
> "home" site.
>
>
> Just to be clear: you are aware that (currently) it's only
> *exchanges* which can be federated, not queues?
>
> Each cluster is configured with:
>
>
> <snip>
>
> We have configured three hardware load balancer VIPs
> that load
> balance
> connections on port 5672 between both nodes in each
> cluster, and
> have
> configured federation upstreams to use these and also
> configured our
> application to use these VIPs for queue/exchange
> creation and for
> publishing messages. Here's our federation config
> script thats
> run on
> node1 of the home cluster only:
>
>
> <snip>
>
> However I am noticing that the "rabbitmqctl eval
> 'rabbit_federation_status:____status()." command only
> shows federation
>
> config on node2 of the home cluster despite having been
> configured on node1:
>
>
> Federation works by establishing link(s) from the
> downstream to the
> upstream(s). You can define the upstreams on any node in the
> downstream cluster (the configuration is cluster-wide) and the
> cluster will arbitrarily start the link processes on any
> node (and
> will migrate them around the cluster if nodes go down).
>
> *However*, rabbit_federation_status:____status() will only
> report
>
> links that are running on the local node. This is probably
> not well
> documented. Hmm. Let's fix that.
>
> (The HTTP API exported by rabbitmq_federation_management
> will report
> links on all nodes.)
>
> <snip>
>
> And I'm noticing that messages published to the queues
> via the load
> balancer VIPs to the remote clusters are not showing up
> in the
> "EventListener" queue of the home cluster.
>
>
> Note that you have federated an exchange called
> "EventListener", not
> a queue. So if you bind a queue to the exchange
> "EventListener" on
> home, and publish to the exchange "EventListener" on
> remote1/2/3
> then you should see messages turning up in the queue on
> home. Is
> that what you're doing?
>
> Is this not the right way to set this up?
>
>
> It sounds like you're basically right, apart from the
> exchange vs
> queue thing.
>
> Is it possible to configure
> this without using an external load balancer and still
> achieve high
> availability?
>
>
> Since 3.1.0, yes. You can specify multiple URIs in a single
> upstream, and federation will connect to a randomly chosen
> one (and
> keep retrying different ones if that one goes down). Use
> something like:
>
> rabbitmqctl set_parameter federation-upstream remote1-upstream
>
> '{"uri":["amqp://remote1-____node1","amqp://remote1-node2"]____,"expires":3600000}'
>
>
> If I federate to a single node of a cluster directly,
> what happens if that node goes down?
>
>
> If you only specify one URI, it will keep retrying until
> that node
> comes back up.
>
> Do I have to configure redundant
> federation between every node of home cluster to every
> node of each
> remote cluster?
>
>
> No - since the config is shared across the cluster. So you
> can set
> up the upstreams on one node of the home cluster, pointing
> to all
> nodes of the remote clusters.
>
> If I don't use a load balancer VIP to publish to from my
> application, if the single node of the cluster I've
> configured goes
> down, it wont help me that the other node is still
> available.
>
>
> Not sure what you mean here. Hopefully I've explained
> enough now.
>
> Cheers, Simon
>
> --
> Simon MacMullen
> RabbitMQ, Pivotal
>
>
>
> --
> Simon MacMullen
> RabbitMQ, Pivotal
>
>
--
Simon MacMullen
RabbitMQ, Pivotal
More information about the rabbitmq-discuss
mailing list