[rabbitmq-discuss] TCP Backpressure / Flow Control in C# producer

[Ulli Berthold - Exactag GmbH]

Matthias,

we're running rabbit 2.8.4 / 2.8.5 on 64bit linux servers with 64bit erlang and plenty of memory, and there was no "high watermark" warning in the logs either.
Just to make sure i'm not erring to much - if the rabbit web ui shows a connection as "flow" instead of "running" or "blocked" that does mean it is in flow control (backpressure), correct?

Regards

Ulli

-----Ursprüngliche Nachricht-----
Von: Matthias Radestock [mailto:matthias at rabbitmq.com] 
Gesendet: Mittwoch, 8. August 2012 10:52
An: Ulli Berthold - Exactag GmbH
Cc: Discussions about RabbitMQ
Betreff: Re: AW: AW: AW: [rabbitmq-discuss] TCP Backpressure / Flow Control in C# producer

Ulli,

On 08/08/12 09:35, Ulli Berthold - Exactag GmbH wrote:
> Rabbit does handle it, yes, but it slows down the inbound messages at 
> some point (like several million messages in queue, with lots of disk 
> space still available).

With several million messages in the queue you may be hitting the memory limit - the rabbit logs should tell you. Messages have some in-memory footprint *even when paged to disk*.

> We would love a way to disable (or better, suppress) the whole 
> backpressure system until the server is really at its memory and disk 
> limit.

Significant durations of backpressure should only occur when the broker is getting close to the configured limits. You may want to adjust the limits though, as per the docs.

Btw, you are running rabbit 2.8.x, right? Flow control in that is much improved over earlier version.

Also, if you are on Windows, make sure you run the 64-bit version of Erlang R15B01, otherwise memory will be capped at 2GB.

> an alternative to just know when backpressure is applied would help us 
> too because we could temporarily send the next messages through an 
> alternate route.

What I proposed is precisely that - a way to detect backpressure.

> An in-program queue would definitely increase complexity almost beyond 
> the point of usefulness of the queueing system - why use an existing 
> one if I got to build one anyways?

We are talking fifty lines of code here, most of which are boilerplate.

Matthias.