[rabbitmq-discuss] Connection is always in "flow" State.

Thu Jul 10 16:45:08 BST 2014

Yep glad for the clarification and that's the behavior I'd actually want
thinking about this.  :)  It's like a file load - all IO, no CPU.  SO we
use consumers to do the CPU processing - which is inherently slower in a
ETL concept.  We're basically using RabbitMQ for distributed ETL work (go
distributed computing).  Since file loads are intermittent, you wouldn't
want to slow down file loads.

Thanks for the clarification and that makes it MUCH more clear!
Jason

On Thu, Jul 10, 2014 at 10:41 AM, Simon MacMullen <simon at rabbitmq.com>
wrote:

> On 10/07/2014 4:23PM, Jason McIntosh wrote:
>
>> Interesting on the 3.3 change.  SO I gotta ask about this situation on
>> that (though it sounds off hand that'll fix the behavior I've seen on
>> 3.2.4).  If a consumer is consumer at 80 msgs/second, but incoming rates
>> are able to publish at 5k a second and the server CPU's/disks/etc.
>> aren't being touched, will rabbit then put back pressure on the
>> publishing rate forcing it down to 72 msgs/second (~10% less than
>> consumes)?
>>
>
> No, no, no!
>
> (And this is why writing documentation is hard! At least for me.)
>
> I somewhat oversimplified. What really happens is that the queue will
> choose to do work that relates to consumers rather than publishers as long
> as:
>
> 1) The queue is running flat out; it never has any idle time.
> 2) The queue actually has work to do that pertains to consuming.
> 3) Messages are being consumed at less than 110% of the rate they're being
> published.
>
> In the case you describe above, 2) will be false 99% of the time, at which
> point the queue will accept publishes as fast as it can.
>
> In other words, when I said "if consumers and producers are both going as
> fast as they can" I meant if the limiting factor is the queue itself, not
> whatever work they're doing.
>
> I hope that is clearer this time round.
>
>
>  I'd assume then that'd propagate up to the publisher if
>> there's a chain of rabbit servers all running 3.3 - e.g. the code
>> publish to rabbit which shovels to the remote rabbit where the code
>> consumer runs.  In this case the code publisher would get back pressure
>> that'd reduce it's publish rate to ~72msgs/second?
>>
>
> Which would be a disaster. It's meant to be a queuing system, after all :-)
>
> Cheers, Simon
>
>
> --
> Simon MacMullen
> RabbitMQ, Pivotal
>

-- 
Jason McIntosh
https://github.com/jasonmcintosh/
573-424-7612
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20140710/9d4026bb/attachment.html>