[rabbitmq-discuss] Performance degrades with increasing queue depth
matthias at lshift.net
Tue Sep 8 06:00:49 BST 2009
Chris Duncan wrote:
> 1. In your test_push.rb you have the line -
> /msg_server.exchange(EXCHANGE).publish("This is my msg")/ // The
> exchange method declares an exchange, so you are doing this every
> time you publish a message
I've run Aisha's original test_push.rb code through the tracer
(http://www.rabbitmq.com/examples.html#tracer), and can see only *one*
exchange.declare, right at the beginning. So that's not the problem.
Aisha Fenton wrote:
> Code below uses tmm1-amqp instead of Bunny. I get 2000mps using this
> code, draining from the same queue.
As Chris mentioned in his response, this uses a subscription, which is a
much more efficient way of retrieving messages than basic.get. Also, on
the publishing side, test_push.rb comes nowhere close to maxing out
rabbit - I guess ruby is simply too slow for that ;)
Back to your original query though ...
> To isolate the problem, I've reduce what I'm doing down to the
> attached code files. One pushes messages, the other pops them.
I ran the tests and got the following results:
- 20k messages: 500Hz, with beam.smp using 30% of the CPU
- 200k messages: 300Hz, with beam.smp using 45% of the CPU - that kind
of slowdown is not totally unexpected and probably due to the increased
cost of gc and the nature of the queue data structure we are using,
- 200k messages: 35Hz, with beam.smp using 95% of the CPU - yes, that's
right, the same test as the previous one but only 1/9th of the throughput!
So there is definitely something odd going on here.
Digging a little deeper, I think the observed behaviour may have to do
with Erlang process hibernation. I consistently get the slower result
when I start consuming after the queue process has gone into
hibernation, which happens after about one second of idleness. By
contrast, if I start consuming straight away, without the process
hibernating first, I consistently get the higher rate.
I have filed a bug to investigate this further.
More information about the rabbitmq-discuss