[rabbitmq-discuss] Fwd: Performance bottleneck on inter-node connection

Thu Apr 17 17:54:24 BST 2014

Watch also NUMA stuff on these bigger multi-core systems.  I've seen some
very interesting effects due to NUMA and how it affects erlang.  Look at
these posts for some more details:
http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2013-January/024790.html

Typically a system with as many cores as you've got requires some more
"tweaking" to get good performance out of it.  One of the best articles I'd
found was this:
http://www.agoragames.com/blog/2011/06/24/of-penguins-rabbits-and-buses/

I don't THINK this would have an impact on your issues with inter-node
communications but I have to toss it out there.  Also, another fun thing we
saw was long DNS lookups causing some odd issues as well.
Jason

On Thu, Apr 17, 2014 at 8:56 AM, Matthias Radestock
<matthias at rabbitmq.com>wrote:

> On 15/04/14 14:20, joseph rouphael wrote:
>
>> On Mon, Apr 14, 2014 at 12:00 PM, Matthias Radestock
>> <matthias at rabbitmq.com <mailto:matthias at rabbitmq.com>> wrote:
>>     What happens when you run the above test with both nodes on the same
>>     machine?
>>
>> The result is almost the same: Fluctuating between 10 and 23KHZ
>>
>
> Thanks for running all these tests. I've reproduced your observations.
>
> What we are seeing here is, at least partially, an issue with the Erlang
> scheduler.
>
> I noticed that when running the test with PerfTest, single producer,
> single consumer, two nodes, the consuming node was using 100% CPU, i.e. the
> equivalent of a single core. That is usually an indication of an Erlang
> scheduler issue since there is an appreciable degree of parallelism that
> should be exploitable.
>
> For this particular test I managed to double performance by supplying the
> "+swt very_low" or "+sfwi 1" flags to Erlang via RABBITMQ_SERVER_ERL_ARGS.
>
> So these flags, plus the other +s... flags of the Erlang VM, are worth
> playing with.
>
>
> Matthias.
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>

-- 
Jason McIntosh
https://github.com/jasonmcintosh/
573-424-7612
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20140417/b2897768/attachment.html>