[rabbitmq-discuss] Connection is always in "flow" State.

[Simon MacMullen]

On 10/07/2014 3:31PM, Jason McIntosh wrote:
> Just a quick confirmation on this.  The documentation states that flow
> control is:
>
>   * A per-connection mechanism that prevents messages being published
>     faster than they can be routed to queues.
>   * A global mechanism that prevents any messages from being published
>     when the memory usage
>     <http://www.rabbitmq.com/memory-use.html> exceeds a configured
>     threshold or free disk space drops below a configured threshold.

That documentation desperately needs a rewrite :-( It doesn't make very 
clear at all that the "flow" state applies to the first of those two 
cases only; the second will show as "blocking" or "blocked".

> In my testing, I've actually seen this as well - where messages can
> backlog (so far seems to the point of available resources), while a
> consumer is chugging along at half the publish speed.

Was this with 3.3.x?

3.3.x ensures that if consumers and producers are both going as fast as 
they can for an individual queue, then the consumers will consume ~10% 
faster than the producers produce.

For earlier releases there is no such guarantee; it's possible for the 
queue to just naturally accept more messages than it delivers. See the 
blog post on the subject:

http://www.rabbitmq.com/blog/2014/04/10/consumer-bias-in-rabbitmq-3-3/

> SO is flow
> control also triggered by consumer processing?

Not really; Michael was misinformed in the post you replied to. Flow 
control is explained a bit more at 
http://www.rabbitmq.com/blog/2014/04/14/finding-bottlenecks-with-rabbitmq-3-3/ 
but in short flow control just means that something in the chain 
(connection -> channel -> queue -> message store) is working as hard as 
it can, and that therefore a publisher is not publishing as fast as it 
would otherwise like to.

> At what point does
> consumer handling trigger flow control if so?  Just looking for a little
> bit of clarification on this - I'm guessing there's some internal logic
> someplace I'm missing on this,

It doesn't, really. Well, it's possible that you could push the queue 
over to being CPU-bound by adding another consumer and thus having it do 
slightly more work, but you'd have to be teetering on the edge anyway.

Cheers, Simon

> Thanks!
> Jason
>
>
> On Tue, Jul 8, 2014 at 3:33 AM, Michael Klishin <mklishin at gopivotal.com
> <mailto:mklishin at gopivotal.com>> wrote:
>
>
>
>     On 8 July 2014 at 07:49:26, Lost (w_pg at qq.com <mailto:w_pg at qq.com>)
>     wrote:
>      > > I have a client to publish message to rabbitmq(1 connection
>      > and 1 channel, in same machine with rabbitmq), no consumer.
>      > Message rates is 2000/s ~ 3000/s
>      > Traffic is 300Kb/s ~ 750kB/s
>      > The channel state is "running", but the connection state is always
>      > "flow".
>      > I found erl.exe cpu is only 5% ~ 10%, but why connection is in "flow"
>      > state?
>
>     Flow control does not depend on CPU usage but rather whether consumers
>     can keep up with producers. Try adding a consumer.
>     --
>     MK
>
>     Staff Software Engineer, Pivotal/RabbitMQ
>     _______________________________________________
>     rabbitmq-discuss mailing list has moved to
>     https://groups.google.com/forum/#!forum/rabbitmq-users,
>     please subscribe to the new list!
>
>     rabbitmq-discuss at lists.rabbitmq.com
>     <mailto:rabbitmq-discuss at lists.rabbitmq.com>
>     https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
>
>
> --
> Jason McIntosh
> https://github.com/jasonmcintosh/
> 573-424-7612
>
>
> _______________________________________________
> rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
> please subscribe to the new list!
>
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>

-- 
Simon MacMullen
RabbitMQ, Pivotal