[rabbitmq-discuss] Connection is always in "flow" State.

Thu Jul 10 16:23:31 BST 2014

Interesting on the 3.3 change.  SO I gotta ask about this situation on that
(though it sounds off hand that'll fix the behavior I've seen on 3.2.4).
 If a consumer is consumer at 80 msgs/second, but incoming rates are able
to publish at 5k a second and the server CPU's/disks/etc. aren't being
touched, will rabbit then put back pressure on the publishing rate forcing
it down to 72 msgs/second (~10% less than consumes)?  I'd assume then
that'd propagate up to the publisher if there's a chain of rabbit servers
all running 3.3 - e.g. the code publish to rabbit which shovels to the
remote rabbit where the code consumer runs.  In this case the code
publisher would get back pressure that'd reduce it's publish rate to
~72msgs/second?

Jason

On Thu, Jul 10, 2014 at 10:01 AM, Simon MacMullen <simon at rabbitmq.com>
wrote:

> On 10/07/2014 3:31PM, Jason McIntosh wrote:
>
>> Just a quick confirmation on this.  The documentation states that flow
>> control is:
>>
>>   * A per-connection mechanism that prevents messages being published
>>
>>     faster than they can be routed to queues.
>>   * A global mechanism that prevents any messages from being published
>>     when the memory usage
>>     <http://www.rabbitmq.com/memory-use.html> exceeds a configured
>>
>>     threshold or free disk space drops below a configured threshold.
>>
>
> That documentation desperately needs a rewrite :-( It doesn't make very
> clear at all that the "flow" state applies to the first of those two cases
> only; the second will show as "blocking" or "blocked".
>
>
>  In my testing, I've actually seen this as well - where messages can
>> backlog (so far seems to the point of available resources), while a
>> consumer is chugging along at half the publish speed.
>>
>
> Was this with 3.3.x?
>
> 3.3.x ensures that if consumers and producers are both going as fast as
> they can for an individual queue, then the consumers will consume ~10%
> faster than the producers produce.
>
> For earlier releases there is no such guarantee; it's possible for the
> queue to just naturally accept more messages than it delivers. See the blog
> post on the subject:
>
> http://www.rabbitmq.com/blog/2014/04/10/consumer-bias-in-rabbitmq-3-3/
>
>
>  SO is flow
>> control also triggered by consumer processing?
>>
>
> Not really; Michael was misinformed in the post you replied to. Flow
> control is explained a bit more at http://www.rabbitmq.com/blog/
> 2014/04/14/finding-bottlenecks-with-rabbitmq-3-3/ but in short flow
> control just means that something in the chain (connection -> channel ->
> queue -> message store) is working as hard as it can, and that therefore a
> publisher is not publishing as fast as it would otherwise like to.
>
>
>  At what point does
>> consumer handling trigger flow control if so?  Just looking for a little
>> bit of clarification on this - I'm guessing there's some internal logic
>> someplace I'm missing on this,
>>
>
> It doesn't, really. Well, it's possible that you could push the queue over
> to being CPU-bound by adding another consumer and thus having it do
> slightly more work, but you'd have to be teetering on the edge anyway.
>
> Cheers, Simon
>
>  Thanks!
>> Jason
>>
>>
>> On Tue, Jul 8, 2014 at 3:33 AM, Michael Klishin <mklishin at gopivotal.com
>> <mailto:mklishin at gopivotal.com>> wrote:
>>
>>
>>
>>     On 8 July 2014 at 07:49:26, Lost (w_pg at qq.com <mailto:w_pg at qq.com>)
>>
>>     wrote:
>>      > > I have a client to publish message to rabbitmq(1 connection
>>      > and 1 channel, in same machine with rabbitmq), no consumer.
>>      > Message rates is 2000/s ~ 3000/s
>>      > Traffic is 300Kb/s ~ 750kB/s
>>      > The channel state is "running", but the connection state is always
>>      > "flow".
>>      > I found erl.exe cpu is only 5% ~ 10%, but why connection is in
>> "flow"
>>      > state?
>>
>>     Flow control does not depend on CPU usage but rather whether consumers
>>     can keep up with producers. Try adding a consumer.
>>     --
>>     MK
>>
>>     Staff Software Engineer, Pivotal/RabbitMQ
>>     _______________________________________________
>>     rabbitmq-discuss mailing list has moved to
>>     https://groups.google.com/forum/#!forum/rabbitmq-users,
>>     please subscribe to the new list!
>>
>>     rabbitmq-discuss at lists.rabbitmq.com
>>     <mailto:rabbitmq-discuss at lists.rabbitmq.com>
>>
>>     https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>
>>
>>
>>
>> --
>> Jason McIntosh
>> https://github.com/jasonmcintosh/
>> 573-424-7612
>>
>>
>> _______________________________________________
>> rabbitmq-discuss mailing list has moved to https://groups.google.com/
>> forum/#!forum/rabbitmq-users,
>> please subscribe to the new list!
>>
>> rabbitmq-discuss at lists.rabbitmq.com
>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>
>>
> --
> Simon MacMullen
> RabbitMQ, Pivotal
>

-- 
Jason McIntosh
https://github.com/jasonmcintosh/
573-424-7612
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20140710/0c952bd4/attachment.html>