[rabbitmq-discuss] TCP Backpressure / Flow Control in C# producer
matthias at rabbitmq.com
Wed Aug 8 13:29:45 BST 2012
On 08/08/12 12:55, Ulli Berthold - Exactag GmbH wrote:
> well, it does happen here... There are times when our queues grow to
> a couple of million messages (which actually was not a problem in
> 2.7.1), in 2.8.4 it went into flow control mode when the CPU load got
> higher though not limiting out.
When the management plug-in says that a connection is in 'flow' mode,
that does *not* mean that it is no longer accepting messages. It only
indicates that in the last ~10 seconds flow control got triggered at
Also, when you say the CPU load wasn't limiting out, bear in mind that
on a multi-core machine rabbit will not necessarily be able to utilise
all cores - depends very much on the workload.
Are you actually experiencing publishing delays at the client?
> Our problem is that we constantly produce lots of messages which have
> to be forwarded within milliseconds or we will run into frontend
> delays which should not occur.
Yes, I understand that.
> I'm looking for a way to keep the queueing server accepting messages
> at the highest possible rate no matter what, for as long as it's not
> "full" (memory, disk, cpu).
One way to increase the buffering capacity is to increase the tcp buffer
sizes in the kernel.
> And when we finally reach a point where the server is almost full,
> I'd curious if the flow control events of the C# driver will ever be
> fired? As I said, that would enable us to use a file-based fallback
> that's mostly intended for queuing server downtimes.
> channel = connection.CreateModel(); channel.ChannelFlow(true);
> channel.FlowControl += new
RabbitMQ does not send channel.flow commands, so no, these events won't
We were sending channel.flow from rabbit a few years ago but had to
abandon that because a) it proved to be way too slow to react to
critical server conditions, and b) many clients weren't handling
I still think my client-side mini queue idea would work well for you. It
provides a high degree of control over when to trigger a fallback
publishing path and has the added advantage that it will also cope with
temporary network disruptions - i.e. conditions that stall (but not
kill) tcp connections - which no server-generated signal could do.
More information about the rabbitmq-discuss