[rabbitmq-discuss] direct queue throughput

[Chuck Remes]

On Aug 28, 2009, at 5:16 PM, Charity Majors wrote:

> Hi.  I'm working on a distributed load testing tool for mysql, where
> we capture several hours worth of peak query load and replay the
> queries against different db versions and configurations.  Rabbitmq is
> the pipeline between the central aggregator process and the workers.
>
> I have (essentially) two queues set up, one for farming query
> sequences out to workers and one for sending mysql return codes back
> to the central.  Both queues are non-durable and use a direct
> exchange.
>
> So far, so good.  But it's starting to look like this configuration
> isn't fast enough to let me test SSDs.  Rabbitmq appears to top out at
> a throughput rate of around 7k msgs/sec for the two queues.  Does that
> number sound right?  The rate is the same whether I'm running on an 8
> core box with 32 gigs of memory or a 4 core with 4 gigs.

I don't have an answer for you but I do have a few things to point out.

1. RabbitMQ is written in Erlang. I am pretty sure the latest stable  
releases are still a single-threaded process (though internally the  
application uses hundreds or thousands of "green" threads). Therefore  
you won't get much throughput increase by running on a larger multi- 
core box.

2. You don't really describe what kind of data stream the queues  
carry. A fanout exchange has higher throughput than a direct exchange  
since it does fewer routing checks. That may or may not be a more  
appropriate choice for you.

3. Are you certain that your worker processes are keeping up? It's  
possible you are only seeing 7k msgs/sec because your worker processes  
aren't consuming them fast enough (ie. back pressure on the queue).

4. It sounds odd that you have higher than 7k queries/sec in your  
environment. Is your mysql system really capable of returning every  
query within 140 microseconds? I'm probably just misunderstanding  
something here so feel free to clarify what you mean by 7k queries/sec.

I hope some of this is useful to you.

cr