[rabbitmq-discuss] flow control in rabbitmq

[Simon MacMullen]

On 03/01/2014 13:56, sandeep kumar wrote:
> Hi ,
>
> I m using rabbitmqserver v 3.2.0
> Scenario 1:
> I continously sent messages to rabbitMQ Server using basic publish()
> without "publisher confirms".
> when the memory limit is breached by the server, the connections are
> getting blocked.
> Even after the connections are blocked, for some interval of time, i see
> publish() being successful .
> Those successfully published messages which are sent after blocked
> connection are neither in queue nor in rdq files
>
> Is the message loss expected?

Note that the messages aren't "lost" exactly; they are in network 
buffers and will get into the broker as soon as the memory alarm clears 
(barring failure).

> Scenario 2:
>
> I continously sent messages to server using basic publish() with
> "publisher confirms" for each message.
> when the memory limit is breached by the server the connections are
> getting blocked.
> Even after the connections are blocked, for some interval of time, i see
> publish() being successful .
> Those successfully published messages which are sent after blocked
> connection are neither in queue nor in rdq files
> There is message loss and the rate of message publishing is dropping
> drastically by 80 to 90 percent if we use " publisher confirms"

I'm not sure how you are seeing "message loss" even by the definition of 
"messages waiting in network buffers" when using publisher confirms.

If you are seeing a performance drop of 90% when using confirms, I 
suspect you are publishing one message, then waiting for a confirm, then 
publishing another. That's an antipattern because network latency 
becomes a limiting factor. You should either publish a batch of 
messages, then wait for them to be confirmed (often easier) or (ideally) 
publish messages as a continuous stream and handle confirms as they come in.

You might also be interested in 
http://www.rabbitmq.com/connection-blocked.html

Cheers, Simon