[rabbitmq-discuss] Enabling publisher confirms damages performance by a factor of 10x

[ModronMaze]

On 23.08.2013 11:17, Matthias Radestock wrote:
> The performance you are seeing is what I'd expect when publishers wait for
> confirmation after every single message. But you say above that publishers
> mark messages as confirmed "asynchronously", so presumably you are not
> doing that. 
You are right. I am maintaining a map of unconfirmed messages and removing
them in confirmation callback. Currently there is no checking or waiting at
all.

> Here's a test using MulticastMain (which ships with the Java client),
> which I believe reproduces your test case exactly, except that the
> consumer is not doing any work and the messages are 1k in size (you didn't
> indicate how large the messages are, so I just guessed). 
1k is ok. I run these tests and got results similar to yours:

./runjava.sh com.rabbitmq.examples.MulticastMain -f persistent -c 1000 -q
512 -C 200000 -s 1024 -i 10
time: 10,041s, sent: 2647 msg/s, confirmed: 2547 msg/s, nacked: 0 msg/s,
received: 2121 msg/s, min/avg/max latency: 6692/1110959/1869794 microseconds
time: 20,055s, sent: 2832 msg/s, confirmed: 2832 msg/s, nacked: 0 msg/s,
received: 2321 msg/s, min/avg/max latency: 1852817/2620249/3774515
microseconds
time: 30,056s, sent: 2646 msg/s, confirmed: 2646 msg/s, nacked: 0 msg/s,
received: 2149 msg/s, min/avg/max latency: 3577167/4903928/5873513
microseconds

./runjava.sh com.rabbitmq.examples.MulticastMain -f persistent -c 1 -q 512
-C 200000 -s 1024 -i 10
time: 10,034s, sent: 16 msg/s, confirmed: 16 msg/s, nacked: 0 msg/s,
received: 16 msg/s, min/avg/max latency: 590/877/9948 microseconds
time: 20,046s, sent: 16 msg/s, confirmed: 16 msg/s, nacked: 0 msg/s,
received: 16 msg/s, min/avg/max latency: 519/701/1562 microseconds
time: 30,086s, sent: 18 msg/s, confirmed: 18 msg/s, nacked: 0 msg/s,
received: 18 msg/s, min/avg/max latency: 506/784/8283 microseconds

I verified my implementation once again and there is one difference: I am
using flag mandatory. So I run the test with this flag:

./runjava.sh com.rabbitmq.examples.MulticastMain -f persistent mandatory -c
2000 -q 512 -C 200000 -s 1024 -i 10
time: 10,068s, sent: 212 msg/s, returned: 0 msg/s, confirmed: 83 msg/s,
nacked: 0 msg/s, received: 84 msg/s, min/avg/max latency:
11511/5031823/8173326 microseconds
time: 20,089s, sent: 96 msg/s, returned: 0 msg/s, confirmed: 91 msg/s,
nacked: 0 msg/s, received: 91 msg/s, min/avg/max latency:
8249958/10334398/13137973 microseconds
time: 30,284s, sent: 90 msg/s, returned: 0 msg/s, confirmed: 86 msg/s,
nacked: 0 msg/s, received: 86 msg/s, min/avg/max latency:
12775852/15313769/17857810 microseconds

./runjava.sh com.rabbitmq.examples.MulticastMain -f persistent mandatory -q
512 -C 200000 -s 1024 -i 10
time: 10,000s, sent: 6979 msg/s, returned: 0 msg/s, received: 6766 msg/s,
min/avg/max latency: 8588/347330/1112170 microseconds
time: 20,000s, sent: 7955 msg/s, returned: 0 msg/s, received: 7965 msg/s,
min/avg/max latency: 159450/238629/369709 microseconds


It appears that the problem happens when I use both publisher confirms and
mandatory flag. Dropping mandatory flag solves it (I tested it and sending
is much faster in my application).
Still, I am surprised to see such drop in performance. What is the reason?

Regards,
ModronMaze



--
View this message in context: http://rabbitmq.1065348.n5.nabble.com/Enabling-publisher-confirms-damages-performance-by-a-factor-of-10x-tp28914p29119.html
Sent from the RabbitMQ mailing list archive at Nabble.com.