[rabbitmq-discuss] Publisher confirms performance

[Matthias Radestock]

Allen,

On 10/09/12 22:06, Allen Riddle wrote:
> I am curious what people are seeing in their production deployments
> of RabbitMQ using publisher confirms in a cluster of nodes with
> durable queues.

Do you see the same results when running your tests against a single
node instead of a cluster? I suspect so, but would be good to get
confirmation of that.

Also, I am assuming you are running a recent (2.8.x) version of rabbit, 
right?

> With local benchmarks against a cluster with 2 nodes, I'm
> synchronously publishing messages and confirming each, and it takes
> roughly 46 ms on average per message.

You are asking RabbitMQ to confirm that the message has been written to
disk. That requires an fsync. Which is very, very expensive.

You can amortise that cost by: a) dealing with confirms asynchronously,
rather than waiting for a confirmation after every publish, and b)
publishing on multiple connections/channels, as long as that is
compatible with the message ordering guarantees your app requires.

> When I take away the publisher confirms piece, it falls down to 8 ms,

Be careful what you are measuring here. If you don't wait for confirms 
then publishing is an entirely asynchronous operation, so if you are 
simply timing the duration of the 'publish' API call then you may just 
be timing how long it takes for the client API to stuff the message into 
a local buffer.

Anyway, taking away confirms means that the client won't have to wait 
for a round trip to the server, and the fsync delay there.

> and when I remove durable queues, there is a net zero difference.

Not sure what you mean here. Zero difference compared to what? Making 
the queues non-durable eliminates the need to wait for an fsync. There's 
still the roundtrip latency though.

Regards,

Matthias.