<div dir="ltr">Interesting on the 3.3 change. SO I gotta ask about this situation on that (though it sounds off hand that'll fix the behavior I've seen on 3.2.4). If a consumer is consumer at 80 msgs/second, but incoming rates are able to publish at 5k a second and the server CPU's/disks/etc. aren't being touched, will rabbit then put back pressure on the publishing rate forcing it down to 72 msgs/second (~10% less than consumes)? I'd assume then that'd propagate up to the publisher if there's a chain of rabbit servers all running 3.3 - e.g. the code publish to rabbit which shovels to the remote rabbit where the code consumer runs. In this case the code publisher would get back pressure that'd reduce it's publish rate to ~72msgs/second?<div>
<br></div><div>Jason</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Jul 10, 2014 at 10:01 AM, Simon MacMullen <span dir="ltr"><<a href="mailto:simon@rabbitmq.com" target="_blank">simon@rabbitmq.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="">On 10/07/2014 3:31PM, Jason McIntosh wrote:<br>
</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="">
Just a quick confirmation on this. The documentation states that flow<br>
control is:<br>
<br></div>
* A per-connection mechanism that prevents messages being published<div class=""><br>
faster than they can be routed to queues.<br></div>
* A global mechanism that prevents any messages from being published<br>
when the memory usage<br>
<<a href="http://www.rabbitmq.com/memory-use.html" target="_blank">http://www.rabbitmq.com/<u></u>memory-use.html</a>> exceeds a configured<div class=""><br>
threshold or free disk space drops below a configured threshold.<br>
</div></blockquote>
<br>
That documentation desperately needs a rewrite :-( It doesn't make very clear at all that the "flow" state applies to the first of those two cases only; the second will show as "blocking" or "blocked".<div class="">
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
In my testing, I've actually seen this as well - where messages can<br>
backlog (so far seems to the point of available resources), while a<br>
consumer is chugging along at half the publish speed.<br>
</blockquote>
<br></div>
Was this with 3.3.x?<br>
<br>
3.3.x ensures that if consumers and producers are both going as fast as they can for an individual queue, then the consumers will consume ~10% faster than the producers produce.<br>
<br>
For earlier releases there is no such guarantee; it's possible for the queue to just naturally accept more messages than it delivers. See the blog post on the subject:<br>
<br>
<a href="http://www.rabbitmq.com/blog/2014/04/10/consumer-bias-in-rabbitmq-3-3/" target="_blank">http://www.rabbitmq.com/blog/<u></u>2014/04/10/consumer-bias-in-<u></u>rabbitmq-3-3/</a><div class=""><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
SO is flow<br>
control also triggered by consumer processing?<br>
</blockquote>
<br></div>
Not really; Michael was misinformed in the post you replied to. Flow control is explained a bit more at <a href="http://www.rabbitmq.com/blog/2014/04/14/finding-bottlenecks-with-rabbitmq-3-3/" target="_blank">http://www.rabbitmq.com/blog/<u></u>2014/04/14/finding-<u></u>bottlenecks-with-rabbitmq-3-3/</a> but in short flow control just means that something in the chain (connection -> channel -> queue -> message store) is working as hard as it can, and that therefore a publisher is not publishing as fast as it would otherwise like to.<div class="">
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
At what point does<br>
consumer handling trigger flow control if so? Just looking for a little<br>
bit of clarification on this - I'm guessing there's some internal logic<br>
someplace I'm missing on this,<br>
</blockquote>
<br></div>
It doesn't, really. Well, it's possible that you could push the queue over to being CPU-bound by adding another consumer and thus having it do slightly more work, but you'd have to be teetering on the edge anyway.<br>
<br>
Cheers, Simon<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="">
Thanks!<br>
Jason<br>
<br>
<br>
On Tue, Jul 8, 2014 at 3:33 AM, Michael Klishin <<a href="mailto:mklishin@gopivotal.com" target="_blank">mklishin@gopivotal.com</a><br></div>
<mailto:<a href="mailto:mklishin@gopivotal.com" target="_blank">mklishin@gopivotal.com</a><u></u>>> wrote:<br>
<br>
<br>
<br>
On 8 July 2014 at 07:49:26, Lost (<a href="mailto:w_pg@qq.com" target="_blank">w_pg@qq.com</a> <mailto:<a href="mailto:w_pg@qq.com" target="_blank">w_pg@qq.com</a>>)<div class=""><br>
wrote:<br>
> > I have a client to publish message to rabbitmq(1 connection<br>
> and 1 channel, in same machine with rabbitmq), no consumer.<br>
> Message rates is 2000/s ~ 3000/s<br>
> Traffic is 300Kb/s ~ 750kB/s<br>
> The channel state is "running", but the connection state is always<br>
> "flow".<br>
> I found erl.exe cpu is only 5% ~ 10%, but why connection is in "flow"<br>
> state?<br>
<br>
Flow control does not depend on CPU usage but rather whether consumers<br>
can keep up with producers. Try adding a consumer.<br>
--<br>
MK<br>
<br>
Staff Software Engineer, Pivotal/RabbitMQ<br>
______________________________<u></u>_________________<br>
rabbitmq-discuss mailing list has moved to<br>
<a href="https://groups.google.com/forum/#!forum/rabbitmq-users" target="_blank">https://groups.google.com/<u></u>forum/#!forum/rabbitmq-users</a>,<br>
please subscribe to the new list!<br>
<br>
<a href="mailto:rabbitmq-discuss@lists.rabbitmq.com" target="_blank">rabbitmq-discuss@lists.<u></u>rabbitmq.com</a><br></div>
<mailto:<a href="mailto:rabbitmq-discuss@lists.rabbitmq.com" target="_blank">rabbitmq-discuss@<u></u>lists.rabbitmq.com</a>><div class=""><br>
<a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/<u></u>cgi-bin/mailman/listinfo/<u></u>rabbitmq-discuss</a><br>
<br>
<br>
<br>
<br>
--<br>
Jason McIntosh<br>
<a href="https://github.com/jasonmcintosh/" target="_blank">https://github.com/<u></u>jasonmcintosh/</a><br>
<a href="tel:573-424-7612" value="+15734247612" target="_blank">573-424-7612</a><br>
<br>
<br>
______________________________<u></u>_________________<br>
rabbitmq-discuss mailing list has moved to <a href="https://groups.google.com/forum/#!forum/rabbitmq-users" target="_blank">https://groups.google.com/<u></u>forum/#!forum/rabbitmq-users</a>,<br>
please subscribe to the new list!<br>
<br>
<a href="mailto:rabbitmq-discuss@lists.rabbitmq.com" target="_blank">rabbitmq-discuss@lists.<u></u>rabbitmq.com</a><br>
<a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/<u></u>cgi-bin/mailman/listinfo/<u></u>rabbitmq-discuss</a><br>
<br>
</div></blockquote><span class="HOEnZb"><font color="#888888">
<br>
-- <br>
Simon MacMullen<br>
RabbitMQ, Pivotal<br>
</font></span></blockquote></div><br><br clear="all"><div><br></div>-- <br><div dir="ltr">Jason McIntosh<br><a href="https://github.com/jasonmcintosh/" target="_blank">https://github.com/jasonmcintosh/</a><br>573-424-7612</div>
</div>