[rabbitmq-discuss] Scaling rabbitmq on ec2
kournjaev at gmail.com
Mon Jan 6 08:22:05 GMT 2014
Anyone ? There must be at least someone in that group that could answer my
On Tuesday, December 31, 2013 11:20:58 AM UTC+2, Roman Kournjaev wrote:
> I hope someone can help me out with some advice , cause I am struggling
> for quite a while now.
> I have some ad server application running on AWS ec2. The server are
> tomcats that can scale , with the load right now I have 10 servers running.
> The overall load on the servers is something 1K rps . It produces twice as
> much messages that have to be persisted eventually.
> Right now the servers send these messages to rabbitmq instances ( that are
> not connected to a cluster ) , each tomcat has an open connection to every
> rabbitmq broker and just picks one randomly to send a message.
> On the other side I have a consumer that consumes all the messages from
> all the brokers.
> The brokers are running on a medium instance ( thats 1 cpu and 3.5 GB ram
> ) and the queues are not persistent or mirrored. The issue is that the
> brokers get to 100 cpu at speeds reaching 700 messages/sec.
> I guess connecting the brokers to a cluster will only decrease the
> performance also i will have to configure a tcp load balancer to connect to
> the cluster from tomcats. Also creating a rabbitmq cluster is not trivial
> at all , especially if the brokers are on an auto scaling group.
> So for the question :
> 1. I read some benchmarks out there and could not understand whether
> the 700m/sec is slow or not. I can use a bigger instance with more CPU ,
> but will the load grow linear ?
> 2. Do you spot anything wrong in the architecture ? Can you think of
> an overall better approach in the message chain ?
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the rabbitmq-discuss