[rabbitmq-discuss] Producer confirms as a heuristic for rate limiting

Thu Feb 28 08:49:28 GMT 2013

Matthias, 

On Wednesday, 27 February 2013 at 18:52, Matthias Radestock wrote:

> Using confirms to exert back pressure on a producing application is
> fine. But I wouldn't expect that to result in particularly stable 
> latencies. For a few reasons:
> 
> 1) one way that the internal confirm logic gets triggered is by "absence 
> of other stuff happening", so when throttling producers, confirms may 
> well happen a lot quicker.
> 

If I understand you correctly, then this is precisely the feedback loop that I'm trying to achieve - when the sender is throttled, this gives the internals of Rabbit more head room to do whatever it is a Rabbit does. In Rabbit becoming more free, the rate of confirms increases, and hence the more credit the sender will receive. When the sender ramps up again, the reverse should happen, and so the feedback cycle continues.

> 
> 2) more generally, much of the internal confirm logic will try to batch 
> things if it can, which reduces the overall cost but makes the timing of 
> confirms more variable.

That's understandable - fine grained ingress credits would come at a premium in any case.

In my scenario I'm setting a timestamp into the message header and the receiver is decoding this in order to compare it with its current timestamp. With the measurement being end to end, would the asynchronous nature of the confirms not be obviated by the fact that you are measuring the speed of the uplink plus the routing overhead plus the speed of a separate delivery (he says had waving)?

> 3) confirms do come at a noticeable penalty on throughput and latency. 
> So a benchmark employing confirms would not be a very good proxy for a 
> real system running w/o confirms.

What constitutes the performance penalty in this case? Do confirms increase the latency because they add to the measurable round trip as seen by the sender, or does the fact that they are turned on add extra processing effort which is CPU time that has to be spread over each message and hence on average adding latency to each message? If the answer is the latter, does the same apply to throughput - since confirms have a non-zero accounting cost, having them turned on diminishes the ability to process as many items in the same amount of time? 

> 
> 
> Having said all that, go and try it :)

I'm very tempted, but the reason why I'm asking first is to avoid going down any rabbit holes that a more experienced person could potentially advise me to avoid :-)

Cheers,

Ben 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130228/c8177618/attachment.htm>