[rabbitmq-discuss] Puka client and publisher confirms performance

[Marek Majkowski]

On Wed, Mar 14, 2012 at 12:31, Christos Stavrakakis
<stavr.chris at gmail.com> wrote:
> I am using Puka client [1] in order to send messages with guaranteed
> delivery using publisher confirms.
>
> Trying to send 200.000 messages I see that waiting for the confirms, takes a
> lot of time.
>
> To be more specific:
>
>     promises = []
>     t0 = time.time()
>     for i in range(0,200000):
>         promise = client.basic_publish(exchange='test_exchange',
> routing_key='test',
>                                                               body="Hello
> world!")
>         promises.append(promise)
>
>     t1 = time.time()
>     client.wait(promises)

Right. This waits for _any_ promise to finish. So that doesn't guarantee
that all the messages will be delivered to the server.

If you want to wait for all the promises consider using wait_for_all.

>     t2 = time.time()
>     promise = client.close()
>     client.wait(promise)
>
> In this example t1-t0 is arround 9s while t2-t1 is more than 100 seconds!!
> Increasing the number of messages results in even worse performance.
>
> Can anyone explain this behavior ? Is there a more effective way to use
> publisher confirms ?

Right. Sending 100k messages to an empty exchange (no bindings):
For  client.wait(promises[-1]):
buffering:  4.14369106293
waiting:  3.26472091675
closing:  0.0569450855255

For client.wait_for_all(promises):
buffering:  4.61279606819
waiting:  4.67923402786
closing:  0.0479018688202

For client.wait(promises) (ie: waiting for first message to be delivered):
buffering:  4.18469595909
waiting:  0.0100059509277
closing:  3.30765604973

But, when we start sending to a queue that exists:
For  client.wait(promises[-1]):
buffering:  4.18939805031
waiting:  91.0133070946
closing:  0.150832891464

For some reason this gets really slow. Additionally, it looks like Puka is using
a lot of CPU. Not sure why.

Marek