[rabbitmq-discuss] Per-Connection Flow Control - RMQ 2.8.1

Wed Apr 25 15:41:57 BST 2012

Hi Matthias,

I haven't run these same tests on 2.7.1 yet, since its our production 
machines (which are currently stable... enough).

The spiking and the locking is not unusual? ;)
Maybe the large gaps between queue memory and actual memory is not 
unusual also? ;)
Maybe the flat queue sizes for several seconds while hundreds of 
thousands of records are added and purged? ;)

All joking aside.... =)

When RabbitMQ is cleaning its index/journals and data files, the queues 
are all locked. Cleansing a queue could take several seconds/minutes 
depending on the volume. The current flow control {200,50} hides a lot 
of this, since you will not reach a large enough volume to cause a long 
enough 'purge' to notice.

On my test machine, all the non-persistent messages were dropped to disk 
(causing a lock) around 4GB every time. On smaller systems, its usually 
around 1.5 to 2GB.  This happens regardless of any settings on rabbitmq 
or mnesia (unless I missed something).

A work around I have tried which works (up until the 'purge'), was to 
mount the mnesia directory on a RAM disk and change the message to be 
persistent (instead of non).  This fixed the 4GB lock, since everything 
is on disk as its received (with a slight delay).

Thanks

On 4/25/12 7:12 AM, Matthias Radestock wrote:
> On 23/04/12 14:54, DawgTool wrote:
>> Here are some stats on a few runs, modifying the flow control limits.
>> Attached is some graphs I created to show the memory consumption.
>
> I cannot see anything particularly unusual in those graphs.
>
> The original problem you reported was that you saw lower rates in 
> 2.8.1 than previous releases. Simon previously asked
> <quote>
> Do you have a message rate for your use case that leads to *stable* 
> memory use in 2.7.1 but which cannot be maintained by 2.8.1?
> </quote>
>
>
> Regards,
>
> Matthias.