[rabbitmq-discuss] Performance degrades with increasing queue depth
Aisha Fenton
aisha.fenton at gmail.com
Tue Sep 8 02:37:31 BST 2009
Go for it Chris
On 8/09/2009, at 11:36 AM, Chris Duncan wrote:
> Hi Aisha,
>
> Hope you don't mind me butting in here.
>
> 2009/9/7 Aisha Fenton <aisha.fenton at gmail.com>
> Hi Chunk.
> I don't know Suhail. And his problem may be different from mine.
>
> To isolate the problem, I've reduce what I'm doing down to the
> attached code files. One pushes messages, the other pops them.
>
> I see the problem even when only test_push.rb is running (as long as
> the queue is sufficiently pre-populated). Nothing else is using
> rabbitmq when I'm doing the test.
>
> I'm getting the same problem on several different machines here --
> my local MacOSX box and a server class Debian box.
>
>
>> exchange - durable or not?
> not durable
>
>> exchange - type? (Looks like "fanout" from the original post)
>
> fanout
>
>> number of bindings per exchange
>
> 1
>
>> queue - durable?
> not durable
>
>> publishing settings - require ack? persistent? immediate?
>
> no ack, non persistent, and non immediate.
>
>> message size? 1K according to the original post
>
> < 1K
>
>
>
> test_push.rb
> ---
>
> require "rubygems"
> require "bunny"
>
> EXCHANGE = "raw_telemetry"
> QUEUE = "process_telemetry"
>
> count ||= 0
> prev_time ||= Time.now
> msg_server = Bunny.new(:spec => '08')
> msg_server.start
>
> msg_server.exchange(EXCHANGE, :type => :fanout)
> msg_server.queue(QUEUE).bind(EXCHANGE)
>
> loop do
>
> msg_server.exchange(EXCHANGE).publish("This is my msg")
>
> count += 1
> if count > 100
> t = Time.now
> puts("msgs pushes per sec: #{count / (t - prev_time)}")
> prev_time = t
> count = 0
> end
>
> end
>
> test_pop.rb
> ---
>
> require "rubygems"
> require "bunny"
>
> @msg_server = Bunny.new(:spec => '08')
> @msg_server.start
>
> def start
> count ||= 0
> prev_time ||= Time.now
> queue = @msg_server.queue("process_telemetry")
>
> loop do
> result = queue.pop
> next if result == :queue_empty
>
> count += 1
> if count > 100
> t = Time.now
> puts("msgs pops per sec: #{count / (t - prev_time)}")
> prev_time = t
> count = 0
> end
> end
>
> end
>
> begin
> start()
> ensure
> puts "stopping"
> @msg_server.stop
> end
>
>
> I have some observations -
>
> 1. In your test_push.rb you have the line -
>
> msg_server.exchange(EXCHANGE).publish("This is my msg")
>
> The exchange method declares an exchange, so you are doing this
> every time you publish a message which will slow things down.
> Declare the exchange using a variable outside the loop like this -
>
> my_exchange = msg_server.exchange(EXCHANGE, :type => :fanout)
>
> Then use the variable inside the loop -
>
> my_exchange.publish("This is my msg")
>
> I think Paolo has highlighted the same thing.
>
> 2. In your test_pop.rb you don't need to specify :spec => '08'.
> Bunny.new defaults to the AMQP 0-8 spec version.
>
> Also, do you mean to have -
>
> next if result == :queue_empty
>
> How are you breaking out of the loop? Might you want to use 'break'
> instead of 'next'?
>
> 3. A Bunny instance only ever uses 2 channels unless you explicitly
> create more. Since you don't seem to be creating more channels I
> don't think that's where your issue lies.
>
> --
> Regards
>
> Chris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20090908/f9c4ea0e/attachment.htm
More information about the rabbitmq-discuss
mailing list