[rabbitmq-discuss] Performance degrades with increasing queue depth

Aisha Fenton aisha.fenton at gmail.com
Tue Sep 8 02:37:31 BST 2009


Go for it Chris

On 8/09/2009, at 11:36 AM, Chris Duncan wrote:

> Hi Aisha,
>
> Hope you don't mind me butting in here.
>
> 2009/9/7 Aisha Fenton <aisha.fenton at gmail.com>
> Hi Chunk.
> I don't know Suhail. And his problem may be different from mine.
>
> To isolate the problem, I've reduce what I'm doing down to the  
> attached code files. One pushes messages, the other pops them.
>
> I see the problem even when only test_push.rb is running (as long as  
> the queue is sufficiently pre-populated). Nothing else is using  
> rabbitmq when I'm doing the test.
>
> I'm getting the same problem on several different machines here --  
> my local MacOSX box and a server class Debian box.
>
>
>> exchange - durable or not?
> not durable
>
>> exchange - type? (Looks like "fanout" from the original post)
>
> fanout
>
>> number of bindings per exchange
>
> 1
>
>> queue  - durable?
> not durable
>
>> publishing settings - require ack? persistent? immediate?
>
> no ack, non persistent, and non immediate.
>
>> message size? 1K according to the original post
>
> < 1K
>
>
>
> test_push.rb
> ---
>
> require "rubygems"
> require "bunny"
>
> EXCHANGE = "raw_telemetry"
> QUEUE = "process_telemetry"
>
> count ||= 0
> prev_time ||= Time.now
> msg_server = Bunny.new(:spec => '08')
> msg_server.start
>
> msg_server.exchange(EXCHANGE, :type  => :fanout)
> msg_server.queue(QUEUE).bind(EXCHANGE)
>
> loop do
>
>   msg_server.exchange(EXCHANGE).publish("This is my msg")
>
>   count += 1
>   if count > 100
>     t = Time.now
>     puts("msgs pushes per sec: #{count / (t - prev_time)}")
>     prev_time = t
>     count = 0
>   end
>
> end
>
> test_pop.rb
> ---
>
> require "rubygems"
> require "bunny"
>
> @msg_server = Bunny.new(:spec => '08')
> @msg_server.start
>
> def start
>   count ||= 0
>   prev_time ||= Time.now
>   queue = @msg_server.queue("process_telemetry")
>
>   loop do
>     result = queue.pop
>     next if result == :queue_empty
>
>     count += 1
>     if count > 100
>       t = Time.now
>       puts("msgs pops per sec: #{count / (t - prev_time)}")
>       prev_time = t
>       count = 0
>     end
>   end
>
> end
>
> begin
>   start()
> ensure
>   puts "stopping"
>   @msg_server.stop
> end
>
>
> I have some observations -
>
> 1. In your test_push.rb you have the line -
>
> msg_server.exchange(EXCHANGE).publish("This is my msg")
>
> The exchange method declares an exchange, so you are doing this  
> every time you publish a message which will slow things down.  
> Declare the exchange using a variable outside the loop like this -
>
> my_exchange = msg_server.exchange(EXCHANGE, :type  => :fanout)
>
> Then use the variable inside the loop -
>
> my_exchange.publish("This is my msg")
>
> I think Paolo has highlighted the same thing.
>
> 2. In your test_pop.rb you don't need to specify :spec => '08'.  
> Bunny.new defaults to the AMQP 0-8 spec version.
>
> Also, do you mean to have -
>
> next if result == :queue_empty
>
> How are you breaking out of the loop? Might you want to use 'break'  
> instead of 'next'?
>
> 3. A Bunny instance only ever uses 2 channels unless you explicitly  
> create more. Since you don't seem to be creating more channels I  
> don't think that's where your issue lies.
>
> -- 
> Regards
>
> Chris

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20090908/f9c4ea0e/attachment.htm 


More information about the rabbitmq-discuss mailing list