<div dir="ltr">We found the AWS Tokyo region to be the best transit point to/from China, although that was a while ago.<div><br></div><div>Tokyo has 3 zones which now is the primary reason we base clusters there.</div><div>
<br></div><div>Singapore should be good to.</div><div><br></div><div>ml</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Nov 13, 2013 at 8:33 AM, Alvaro Videla <span dir="ltr"><<a href="mailto:videlalvaro@gmail.com" target="_blank">videlalvaro@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi,<div><br></div><div>I've been trying to setup something like this, more like an experiment/research to see how far federation could be taken across AWS availability zones, but I have to admit that I've been sidetracked by various projects and conferences.<br>
<div class="gmail_extra"><br></div><div class="gmail_extra">When I was living in China I remember that a company there used to have a similar problem as you, when sending data to the US.</div><div class="gmail_extra"><br>
</div><div class="gmail_extra">Considering that you have upstreams at various locations in the world, could it be the case that you have better TCP bandwidth from China to another non US location, say HK, Japan, Singapore, or Korea?</div>
<div class="gmail_extra"><br></div><div class="gmail_extra">If that's the case you could perhaps try to form a different federation graph that would allow you to achieve some max flow routing. Say publishing from China to a country that has better bandwidth than from China to the US, and then from that country to the US, or to a third country and so on. I know this might sound nice theoretically but I haven't tried it. I throw this out nonetheless, in case it might lead you to a solution.</div>
<div class="gmail_extra"><br></div><div class="gmail_extra">Regards,</div><div class="gmail_extra"><br></div><div class="gmail_extra">Alvaro<br><br><div class="gmail_quote"><div><div class="h5">On Thu, Nov 7, 2013 at 6:47 PM, Dustin <span dir="ltr"><<a href="mailto:dustink.ml@gmail.com" target="_blank">dustink.ml@gmail.com</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5"><div dir="ltr">Hello All!<div><br></div><div>I wanted to shoot this out to see if anyone has had any experience with using RabbitMQ for a mass global data ingestion pipeline. A small disclaimer, I'm a total RMQ noob :)</div>
<div><br></div><div>We currently have a fan-in design, where we have a single downstream 2 node HA cluster in the same data center as our data warehouse. We have around 22 upstreams (also 2 node HA clusters) located in datacenters all over the world. The configuration is extremely simple. We have a single direct exchange, which everything publishes to. Each application uses a specified routing key for that application. We end up with queue per application (currently around 10). We are running 3.0.0 on the downstream cluster (been waiting for a maintenance window to upgrade) and 3.1.5 on the upstreams.</div>
<div><br></div><div>This design has held up well, and we are averaging around 20k/sec messages a day.</div><div><br></div><div>We have ran into 2 problems which won't allow us to scale any further. The first is the max bandwidth for a single TCP connection across the globe (specifically between the US and China). The second is we have maxed out the CPU for the federation clients on the downstream (SSL is enabled, I'm not sure how much CPU overhead that adds).</div>
<div><br></div><div>For the CPU issue, I figured the newly added federated queues would be a perfect solution to the problem. I can setup additional Rabbits on the downstream side, setup the federation links, and have everything load balance nicely. The only thing it doesn't address is the max bandwidth for a single TCP connection. Because of our initial design, we would run into max bandwidth problems for each queue.</div>
<div><br></div><div>Our current objective is to be able to send 20k/sec messages from each datacenter for a single application. In China, the most we can do is around 2.5k/sec (ends up being around 1.6MB/sec, this is on a good day). Because this message load will be from a single application, with the current design, it will be tied to a single routing key. So for China, I would need around 8 TCP connections to do this.</div>
<div><br></div><div>For this use case, message order doesn't matter. Does anyone have any ideas on how I can setup multiple federation links that will be load balanced? Here are some ideas I have, but they all feel hacky.</div>
<div><br></div><div>1) On the upstreams, use a consistent hash exchange, with exchange to exchange bindings to 8 direct exchanges, which would be federated.</div><div>2) Run multiple instances of RMQ on the downstream machines, and use federated queues. Total number of instances across all machines should be greater than 8.</div>
<div><br></div><div>My apologies in advance if I'm missing something obvious. Please let me know if I'm trying to fit a round peg in a square hole. :)</div><div><br></div><div>Thanks!</div><span><font color="#888888"><div>
<br></div><div>-Dustin</div>
<div><br></div></font></span></div>
<br></div></div>_______________________________________________<br>
rabbitmq-discuss mailing list<br>
<a href="mailto:rabbitmq-discuss@lists.rabbitmq.com" target="_blank">rabbitmq-discuss@lists.rabbitmq.com</a><br>
<a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a><br>
<br></blockquote></div><br></div></div></div>
<br>_______________________________________________<br>
rabbitmq-discuss mailing list<br>
<a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a><br>
<a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a><br>
<br></blockquote></div><br></div>