[rabbitmq-discuss] Per-Connection Flow Control - RMQ 2.8.1

[Matthias Radestock]

On 25/04/12 15:41, DawgTool wrote:
> I haven't run these same tests on 2.7.1 yet, since its our production
> machines (which are currently stable... enough).

I see. So, just to be clear, we are now no longer looking at the 
original problem you reported, namely that you thought 2.7.1 could 
handle a considerably higher rate than 2.8.1?

> The spiking and the locking is not unusual? ;)
> Maybe the large gaps between queue memory and actual memory is not
> unusual also? ;)
> Maybe the flat queue sizes for several seconds while hundreds of
> thousands of records are added and purged? ;)
> All joking aside.... =)

Running, as you were, with considerably higher flow control credits than 
standard make all of the above worse. One of the reasons we introduced 
flow control (and picked the standard settings we have) is to reduce the 
magnitude of these.

> When RabbitMQ is cleaning its index/journals and data files, the queues
> are all locked.

Paging to disk happens largely asynchronously. However, ordering and 
other causality constraints may require certain operations to wait until 
others have completed. There is not much we can do about that.

> The current flow control {200,50} hides a lot of this

Yep. Quite intentionally.

> On my test machine, all the non-persistent messages were dropped to disk
> (causing a lock) around 4GB every time. On smaller systems, its usually
> around 1.5 to 2GB. This happens regardless of any settings on rabbitmq
> or mnesia (unless I missed something).

With the default credit settings I can get publishers to pause for a few 
seconds every now and then, but not longer. Are you seeing something 
different?

Regards,

Matthias.