[rabbitmq-discuss] RabbitMQ writes non stop to disk

[Raphael Simon]

This is the output of the same command line on the broker with high i/o:

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz
avgqu-sz   await  svctm  %util
dm-0              0.00     0.00    0.00   83.62     0.00  6217.08    74.35
  0.06    0.78   0.50   4.20

So you're right I was looking at blocks written per second. Is there a way
to easily retrieve the average ingress rate across all queues for a broker?
Maybe what's happening is a rogue client is spamming that specific server.
I'd like to see the average ingress rate across all our brokers to validate
that theory.

Thanks for the help troubleshooting!

--
Raphael.

On Wed, Oct 26, 2011 at 6:29 PM, Matthias Radestock
<matthias at rabbitmq.com>wrote:

> Raphael,
>
>
> On 27/10/11 01:47, Matthias Radestock wrote:
>
>> Can you find out what files the 6k/s writes are going to? It can't be
>> just the persistent message store since, as you observed, that's only
>> writing 16M about every couple of minutes. And while I would expect to
>> see some writes to the queue index journal files, these shouldn't add up
>> to anywhere near 6k/s with the traffic levels you are reporting.
>>
>
> Btw, are you definitely looking at the correct column in the iostat output?
>
> As a test I ran 'MulticastMain -f persistent -q 1 -r 17 -s 50000' (i.e.
> 17msg/s, persistent, qos=1, 50k message size), and the same with "-r 8" -
> which roughly corresponds to the rates I saw reported on the two queues in
> the system with non-negligible traffic. I adjusted rabbit's memory
> thresholds s.t. the target_ram_count of the queues dropped to zero but w/o
> raising the memory alarm. 'iostat -d -x 10' shows:
>
> Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz
> avgqu-sz   await  svctm  %util
> sda               0.00   219.70    0.00   57.70     0.00  2219.20 38.46
> 0.61   10.57  10.55  60.90
>
> So that's just 57.70 writes/s.
>
>
> Matthias.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20111026/27933598/attachment.htm>