[rabbitmq-discuss] Almost all statistics drop to 0 every hour

[Irmo Manie]

On Tue, Nov 13, 2012 at 12:24 PM, Simon MacMullen <simon at rabbitmq.com>wrote:

> On 13/11/12 10:35, Irmo Manie wrote:
>
>> I'm still seeing this behavior and it seems to get worse.
>> Every hour there is a drop in delivery rate to 0 and now also the
>> publish rate drops to 0.
>> Sometimes even for a period of 5 minutes
>>
>> I'm really sure that messaging continues and the statistics are a lie.
>> We use rabbitmq as our main communication hub and certainly a drop in
>> real-time market data going from ~500 msg/s to 0 for about 5 minutes
>> must be noticed somewhere if this is true.
>>
>> I'm wondering if it might be the writing part of the statistics to the
>> database. Could that timeout?
>>
>
> No.
>
> But if the database is busy with something else then it could take a while
> to process the updates it's received. I find it hard to believe that it
> could be busy for 5 minutes though.
>
> There is nothing scheduled once an hour in RabbitMQ though - so whatever
> is happening once an hour must be outside the broker. Figuring out what
> that is would help a lot.
>
>
Clear :-)


>
>  I don't really want to restart to see if that fixes the problem.
>> It's been running for 177 days now and apart from the weird statistics
>> we don't have any problems.
>>
>> Is there anything I could do as additional checks to figure out where
>> the problem might be?
>>
>
> Just to be clear: is this just the rates in /api/overview or the ones in
> /api/queues as well?
>
>
Both show 0 as the rates


> I'd also be interested to see the output of:
>
> rabbitmqctl eval 'process_info(global:whereis_**name(rabbit_mgmt_db),
> [message_queue_len,status, current_stacktrace]).'
>
> when the statistics drop to 0.
>
>
I'll try to check this and let you know if I find anything weird

Thanks so far for the information and help.

- Irmo


>
> Cheers, Simon
>
> --
> Simon MacMullen
> RabbitMQ, VMware
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20121113/99e5168e/attachment.htm>