[rabbitmq-discuss] Multifunctional Rabbit Architectures

Sun Feb 23 20:45:09 GMT 2014

See below. -ml

On Sun, Feb 23, 2014 at 8:47 AM, Eric Berg <eric.berg at salesforce.com> wrote:

> Thanks Michael and Jerry for your input, I really appreciate it.
>
> Michael,
> Roughly how many nodes are in your proxy cluster?
>

We have 3 nodes in each proxy cluster, one in each AWS zone ina region.

> What sort of message rate/size does it handle?
>

Roughly 1M msgs per day growing to 100M. Often bursty. Median size 2KB,
some up to 200KB.

> What sort of HA policy do you have if a proxy cluster node goes down?
>

All queues we define are HA; cluster_partition_handling is pause_minority.

> How does the cluster proxy? Federation or shovel?
>

The proxy is passive. Shovels run on the core or product vhosts (see below).

> Do all downstream clusters serve the same purpose or are they separated
> by function?
>

We run some downstream 'product' clusters on behalf of internal clients,
giving them most of the control, as reliable waystations (replicated and
parallelized services) into the nyt⨍aбrik regional core clusters. Other
internal clients connect directly from the proxies to the core clusters.

How did you go about determining this topology for your requirements? Was
> it just a trial and error type experiment or are there well established
> patterns that I am unaware of?
>

We have experimented over the past 2 years and have been guided by
principles more than by patterns: simple, global, headless, replicated,
idempotent, auto-balancing, everything active.

>
>
> Jerry,
> When you say "It's very widely done" do you mean:
> A. A large cluster to contain everything or
> B. A 'proxy' type upstream cluster that serves as a global entry point
>
> Thanks so much guys!
>
>
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20140223/4239bc80/attachment.html>