[rabbitmq-discuss] questions about RabbitMQ linear scalability

[Zhihui Jiao]

I don't think RabbitMQ will have linear scalability in any case(ha, non-ha):
In ha mode, message are broadcast to multiple nodes, so the more node you have, the worse performance you get 
if your setting is to ha all nodes int the cluster;
In non-ha mode, message data only reside in one queue on only one node, but queue meta are spread to all nodes
in the cluster, so if adding more nodes, you may got some performance improvement, but not linear improvement.

Zhihui Jiao

在 2013-8-22，上午11:15，Junius Wang <wangjunbo924 at gmail.com> 写道：

> I’m sorry to pick this email again because we really want to see any comments from all of you.
> The key point is that we try to prove the linear scaling of RabbitMQ high availability queues but no lucky. What we see are that:
> 1.       The throughput of two node cluster is 50%-60% worse than a single node broker.
> 2.       Adding more node did have improvement on throughput but we only got 25% improvement(throughput of 3 node cluster is 25% better than 2 node cluster. 4 node cluster is 25% better than 3 node cluster too). What we expected is a 45-degree line, that means when 2 nodes are used the throughput is double. With 3 nodes, then triple.
>  
> Can anybody give any advice? What’s the best practice in using RabbitMQ mirror queues?
>  
>  
> From: Junius Wang [mailto:wangjunbo924 at gmail.com] 
> Sent: Thursday, August 08, 2013 6:50 PM
> To: rabbitmq-discuss at lists.rabbitmq.com
> Subject: questions about RabbitMQ linear scalability
>  
> Hi guys,
> Our team are performing some performance tests and we try to prove that we can get linear scalability of our services, it means when we add more RabbitMQ nodes, the throughputs should increase as well.  We use RabbitMQ as our backend message system. However, when we add more RabbitMQ servers to cluster, the throughput decreases which is not expected and I would like to ask for your help. Following are the details, any comments are appreciated.
>  
> We setup the our testing environment on AWS. All instances, including app servers, RabbitMQ servers and Jmeter servers(host test tool Jmeter 2.7), are of  type m1.xlarge(8ECUs, 4 Cores and 15Gib memory). For RabbitMQ servers, “EBS optimized” are enabled, and the PIOPs are set to 2000 to improve the disc read/write speed. There are two ELBs. One for App servers and the another for RabbitMQ servers.  Here are some more environment configurations:
> Instances Type
> m1.xlarge
> connect pool size for App servers
> 40000
> JVM args for App servers
> -Xmx12G -Xms6G -XX:MaxPermSize=2G
> App servers count
> 2
> Jmeter servers
> 10
> total concurrent users
> 4000
> PIOPs for RabbitMQ Servers
> 2000
> Consumer Prefetch Count
> 1000
> message size
> 1KB
> hipe_compile
> FALSE
>  
> The designed topologic of broker:
> <image001.jpg>
> 1)      We create only one global topic exchange “events” to route all incoming messages. It’s durable.
> 2)      There will be an event log queue which receive all messages with binding key “#”. It’s durable as well.
> 3)      There will be several queues for our external services and each service queue may have several bindings to the global exchange which we called eventypes.
> Actually In our test scenarios,  we create only one service queue and only one binding on this queue. Message publishers publish messages using the only binding routing key.  So in fact, every message resides on two queues: the event log queue and a test service queue. And all messages are persistent.
>  
> We start with 1 single RabbitMQ node. And then run different configurations with 2-RabbitMQ node cluster. For example ,1) change the node type 2) enable/disable mirrored queues(also called high available queue, showed as “ha” in the following tables). The test results shown as following:
> configurations
> Avg response time(ms)
> Throughput(requests/sec)
> 1 rabbitmq node
> 1125
> 2702.904
> 2disc,ha
> 2957
> 1192.645
> 1disc+1ram,ha
> 3654
> 998.126
> 1disc+1ram,non-ha
> 1110
> 2799.826
>  
> 1015
> 2720.422
> 2disc,non-ha
> 1289
> 2570.826
>  
> 1080
> 2845.821
>  
> 1118
> 2802.278
> We can see that, while two rabbitmq servers are used, we see no throughput improvement. This doesn’t make sense to me. To my understanding,  Here comes my questions:
> 1.       What does RabbitMQ cluster scalability means exactly? Does it refer to the capacity of handing messages? The queue capacity? Or the message number?
> 2.       The  mirrored queues seems to affect performance in a large scale. What’s the possible cause of this?
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130822/7bd45e06/attachment.htm>