[rabbitmq-discuss] Management Plugin stops gathering stats ?

[Rene Parra]

Helps if I provide the link for the video:

http://screencast.com/t/BtWXVsvijqa5

TimePoints:
* @ 0m00s:  - Memory Pressure WORKING, Throttling ACTIVE
* @ 0m37s:  - BANG! Statistics stop collecting
* @ 0m56s:  - Showing how Memory Level now exceeded.


On 8/25/11 12:12 PM, "Rene Parra" <rparra at homeaway.com> wrote:

> Matthias, et.al, 
> 
> It took me a while to answer this, sorry for the delay.
> 
> We have since upgraded to 2.5.1 and erlang R14B03.
> 
> The publish rates seem to only work if I use the CachingConnectionFactory in
> Spring vs the SingleConnectionFactory. Makes sense, because when I use
> SingleConnectionFactory, my publisher is creating and closing a connection on
> every single publish.... therefore rabbit only can create statistics for a
> connection that no longer exists (wouldn't make sense unless you are doing
> historic stats, which I'd be surprised if you do).
> 
> FWIW,
> 
> The stats bug, and the memory bug seem to be better in 2.5.1/R14B03. We are
> progressing with our soak / load tests at a very light load (100 TPS).
> 
> I was able to exhibit the stats bug, and hence memory bug however in
> 2.5.1/R14B03 under high load.
> 
> The video is here:
> 
> TimePoints:
> * @ 0m00s:  - Memory Pressure WORKING, Throttling ACTIVE
> * @ 0m37s:  - BANG! Statistics stop collecting
> * @ 0m56s:  - Showing how Memory Level now exceeded.
> 
> Good news is all the messages were indeed delivered.
> Bad news even after the test completed, I had to restart RabbitMQ to recover
> the memory back down to a normal state and get the stats/memory level/publish
> throttling working again.
> 
> We are going to continue with our tests @ 100TPS (since that is all we need
> for our initial application) but would appreciate any other insight that we
> can provide for you on this issue.
> 
> Thank you for your help!
> --rparra
> 
> Note: The above screencast was done after we upgraded to 2.5.1 but BEFORE we
> upgraded to erlang R14B03.  We have not run the heavy load against R14B03 yet.
> 
> On 8/23/11 3:35 PM, "Matthias Radestock" <matthias at rabbitmq.com> wrote:
> 
>> Rene,
>> 
>> On 23/08/11 20:59, Rene Parra wrote:
>>>>> Here¹s a ScreenCast of the management UI working and then ³FAILS²:
>>>>> 
>>>>> http://goo.gl/jrvAP
>>>>> 
>>>>> (please fast forward to these time points in the screencast)
>>>>> 
>>>>> @2m 16s:  The system runs normally and then I demo the queues stats
>>>>> collection working perfectly.
>> 
>> Er, no, something is wrong here. We see Delivers and Acks, but no queued
>> messages and no publishes. So where are the delivered messages coming
>> from? ...
>> 
>>>>> @2m 56s:  Connection View shows normal operating behavior.
>> 
>> This shows a 25kb/s inbound rate on one of the connections. Presumably
>> these are the publishes, but for some reason they don't show up in the
>> stats.
>> 
>> I definitely recommend upgrading to 2.5.1. If the problem still occurs
>> there it will be easier for us to track down.
>> 
>> 
>> Regards,
>> 
>> Matthias.