[rabbitmq-discuss] RabbitMQ Producer Flow Control

[Simon MacMullen]

Hi Aaron - there's currently no way to turn RabbitMQ's flow control off 
since it's doing something different from ActiveMQ's flow control 
(assuming I read their documentation correctly).

RabbitMQ's flow control is there to make sure that messages are accepted 
by the broker at no more than the rate that the queues can enqueue them. 
In other words, if flow control were turned off, messages would back up 
very quickly in the channel and connection processes, and memory use 
shoots for the sky very quickly.

As I understand it ActiveMQ uses flow control to limit message ingress 
rates to prevent queues from becoming too big - in other words it kicks 
in earlier than RabbitMQ's control. The consequence of turning it off is 
just that queues get bigger - in other words, ActiveMQ with flow control 
off is equivalent to RabbitMQ with flow control on. I think.

The real question then is why your queue is only able to accept 4k 
messages / sec. If they are persistent messages in a durable queue I 
would expect that - that's the cost of getting the messages onto disc. 
Ditto if the messages are non-persistent but the queues are large enough 
to need to page to disc. Oh, and mirrored queues were quite slow in 2.x 
too, they're much faster now. Could any of these things account for the 
performance you're seeing?

Cheers, Simon

On 04/02/13 18:48, Aaron Blew wrote:
> Hey Simon,
> Thanks for getting back to me.  After upgrading to 2.8.7 I'm still
> unable to flow messages in to the head node faster than 4,000/sec while
> the consumer is working.  I appreciate that flow control is trying to
> prevent the server from being over-run, but in some situations it would
> be very beneficial for an administrator to override this behavior.  Are
> there any plans to add this kind of behavior?  I'm currently seriously
> considering a switch to ActiveMQ/stomp since it gives me the knobs to
> tweak this behavior both in real time and via configuration.
>
> Thanks!
> -Aaron
>
>
> On Mon, Feb 4, 2013 at 3:30 AM, Simon MacMullen <simon at rabbitmq.com
> <mailto:simon at rabbitmq.com>> wrote:
>
>     Hi. There's a bug in versions of RabbitMQ prior to 2.8.4 where under
>     some circumstances queues can over-prioritise getting rid of message
>     to such an extent that they don't accept any more until empty.
>
>     I recommend an upgrade to 2.8.7 at least, or preferably 3.0.2.
>
>     And in this case flow control is just the messenger, so you don't
>     want to shoot it :-)
>
>     Cheers, Simon
>
>
>     On 01/02/13 17:31, Aaron Blew wrote:
>
>         Hi all,
>         I'm currently using RabbitMQ 2.8.1 as the backbone of my Logstash
>         architecture.  Currently I've got 5 RabbitMQ servers running shovels
>         that push messages onto the "mothership", where data is consumed
>         by a
>         Logstash agent that feeds into various endpoints.
>
>         The issue is that under high load the Logstash consumer on the
>         mothership isn't fast enough at to keep up with the data coming in
>         (generally above 4,000 messages/sec).  Normally this wouldn't be a
>         problem since we've got capacity to buffer things on the
>         mothership, but
>         RabbitMQ starts rate limiting the shovels that are sending data in,
>         which compounds the issue since the nodes shoveling data don't
>         have as
>         much IO capacity as the mothership does.  If I stop the slow
>         consumer on
>         the mothership, the shovels play data in as fast as they can
>         acknowledge it.
>
>         Ideally I'd be able to turn off producer flow control on the
>         mothership
>         side so the shovels wouldn't be slowed down except in cases
>         where I'm
>         approaching high watermarks or something along those lines.
>
>         Can anyone offer any suggestions on how I might be able to
>         adjust the
>         behavior?
>
>         Thanks,
>         -Aaron
>
>
>         _________________________________________________
>         rabbitmq-discuss mailing list
>         rabbitmq-discuss at lists.__rabbitmq.com
>         <mailto:rabbitmq-discuss at lists.rabbitmq.com>
>         https://lists.rabbitmq.com/__cgi-bin/mailman/listinfo/__rabbitmq-discuss
>         <https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss>
>
>
>
>     --
>     Simon MacMullen
>     RabbitMQ, VMware
>
>


-- 
Simon MacMullen
RabbitMQ, VMware