[rabbitmq-discuss] Usefulness of Channel.Flow with RabbitMQ 2.x.x

[Matthew Sackman]

On Tue, Nov 09, 2010 at 08:43:27AM -0800, A. MacLeod wrote:
> Am I correct in concluding that a RabbitMQ 2.x.x broker will never
> generate a channel.flow message, ever?

Yes, you are.

>  I grepped the Erlang source
> and saw that the broker responds to incoming channel.flow messages
> appropriately.  There is a comment in the memory management code that
> mentions that a memory alarm should cause channel.flow, but I did not
> see anything in actual code to back that up.

Superb - the comment is out of date. I have fixed that.

> A post from Marek M. in
> http://groups.google.com/group/rabbitmq-discuss/browse_thread/thread/2d26d2d740c216be/a3ff743a389bb360
> indicates that channel.flow behavior has changed between 1.8.1 and
> 2.0.0.  From what I understand, the broker now effects flow control by
> just not reading bytes from the socket instead of issuing channel.flow
> messages.  This is consistent with a test I ran, using the 2.0.0 Java
> client library, where I have a producer thread continuously shove 8kb
> messages into a direct exchange, and have a slow-poke consumer using
> basic.consume to receive acknowledgement-required messages very
> slowly.  Once the memory fills up, the producer visibly slows down,
> but FlowListener never has handleFlow called.

Indeed that is correct.

> I have also tried some experiments where I use something like
> consumerChannel.flow(false) to stop flow on the consumer's channel.
> However, when I try to reopen the flow by doing
> consumerChannel.flow(true), my consumer instance (of a class derived
> from DefaultConsumer) never receives any more messages.  Do I need to
> re-issue consumerChannel.basicConsume(...) after I re-enable flow from
> the consumer side?

No, you shouldn't do. The Java client has a test for channel.flow in
com.rabbitmq.client.test.functional.QosTests. If you can't make it work
please let us know - it should all "just work".

FWIW, the idea of the broker sending channel.flow to the client is a
good one - it's sensible for the broker to be able to ignore just
publishes amongst other methods on each given channel.

Unfortunately, channel.flow is synchronous, so the broker has to wait
for the client to respond (and send back the flow_ok) before it can
start getting upperty should the producer continue to send messages. The
problem here then is what timeout do you apply before you just start
ignoring the producer anyway? If the client is only doing publishes
anyway, you have absolutely no idea what the round trip time is.
Furthermore, as we discovered when we tried to implement it, there's the
possibility that the client _did_ respond and issue the flow_ok, but the
broker was so overloaded it didn't notice it until after the timeout.
Thus the client gets disconnected even when it did all the right things.

Consequently, relying on TCP backpressure is pretty much the only
sensible thing that can be done.

Matthew