[rabbitmq-discuss] new queue causes extremely poor disk IO
tim at rabbitmq.com
Fri May 17 20:45:38 BST 2013
There have been innumerable bug fixes and performance improvements to rabbit since 2.7.1 - is upgrading a possibility?
On 17 May 2013, at 16:38, Kevin Goess wrote:
> Hi, we were hoping to get some pointers in looking into this problem. We're running rabbitmq 2.7.1-1 in a xen VM on debian 6 squeeze.
> We generally process about 20 messages/sec, with occasional transient spikes to 500.
> We recently added a new queue and saw the disk utilization jump to 60-80% and the iowait to 40-60%. Read sectors/sec is around 0, and write sectors/sec is about 1k, which is unchanged from before adding this new queue.
> The input and output on this new queue is only 1-10 per second. The messages in it are only 60-70 bytes. We currently have a backlog of about 100,000 messages in the queue, we maxed out at 250,000.
> The individual consumers of this new queue don't fetch that many messages--the work they're associated with takes 1-30 seconds to perform. We're using STOMP as a message transport.
> The problem is that every consumer we add to the queue seems to increase disk activity by about 10%, and by the time we have 8 consumers, the rabbitmq server is overloaded enough that other queues start falling behind in their deliveries.
> If we shut off all the consumers on the new queue, disk activity immediately falls to easily sustainable levels (near zero iowait).
> Is there anything we can look at to get further information on the problem? Any pointers you have would be helpful. Thanks!
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
More information about the rabbitmq-discuss