[rabbitmq-discuss] Consumer slows down (a lot) after producer exceeds a certain publish rate, why?

[Zhibo Wei]

Hi Alvaro,

Thanks for pointing out this.

But when I checked the connection status from management plugin, only
publisher connections were in 'Flow/Blocked' states, all consumers'
connections were still 'running' but in a very low rate.

And, as far as I know, 15000 msgs/sec is still far from the ceiling (Based
on
http://www.rabbitmq.com/blog/2012/04/25/rabbitmq-performance-measurements-part-2/
).

Did I miss anything?

Thanks,
Zhibo


On Fri, Sep 27, 2013 at 2:52 PM, Alvaro Videla <videlalvaro at gmail.com>wrote:

> Hi,
>
> Take a look at the flow control mechanism:
> http://www.rabbitmq.com/memory.html
>
> "A per-connection mechanism that prevents messages being published faster
> than they can be routed to queues." Perhaps your publishers are hitting
> that.
> Regards,
>
> Alvaro
>
>
> On Fri, Sep 27, 2013 at 2:44 AM, Zhibo Wei <zweicmu at gmail.com> wrote:
>
>> 3.15 rabbitmq-server
>> 3.15 java-client (consumer)
>>  3.04 erlang (producer)
>> ec2 m1.large
>> 1 durable queue. no exchange.
>> Producer has 5 connections, each connection holds 20 channels.
>> Consumer has only 1 connection, who holds 20 channels, autoack = true.
>>
>> If the publish rate is equal or below 15000msgs/sec, then the consumer
>> can hold it up (Queue never grows). However, if the publish rate exceeds
>> 15,000, say 18,000 msgs/sec, then the delivery rate will drop to
>> 400~2000/sec, then queue starts paging, blocks producer, and then
>> everything crashes.
>>
>> The CPU and memory usages are just fine, but I'm not sure what else could
>> cause such problem. Bandwidth? Socket buffer size?
>>
>> Anyone saw this kind of issue before? Any clues? Any other things I
>> should check?
>>
>> Thanks,
>> Zhibo
>>
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.rabbitmq.com
>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>
>>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130927/727f1044/attachment.htm>