[rabbitmq-discuss] Puka client and publisher confirms performance

[Marek Majkowski]

On Wed, Mar 14, 2012 at 13:14, Marek Majkowski <majek04 at gmail.com> wrote:
> On Wed, Mar 14, 2012 at 12:31, Christos Stavrakakis
> <stavr.chris at gmail.com> wrote:
>> I am using Puka client [1] in order to send messages with guaranteed
>> delivery using publisher confirms.
>>
>> Trying to send 200.000 messages I see that waiting for the confirms, takes a
>> lot of time.
>>
>> To be more specific:
>>
>>     promises = []
>>     t0 = time.time()
>>     for i in range(0,200000):
>>         promise = client.basic_publish(exchange='test_exchange',
>> routing_key='test',
>>                                                               body="Hello
>> world!")
>>         promises.append(promise)
>>
>>     t1 = time.time()
>>     client.wait(promises)
>>     t2 = time.time()
>>     promise = client.close()
>>     client.wait(promise)
>>
>> In this example t1-t0 is arround 9s while t2-t1 is more than 100 seconds!!
>> Increasing the number of messages results in even worse performance.
>>
>> Can anyone explain this behavior ? Is there a more effective way to use
>> publisher confirms ?

Right, to the point:
 - first part of code adds messages to the write buffer - no network activity
 - second part of the code - wait for replies from rabbit
   (there may be some cpu cost for managing a big number of promises,
    within Puka library, this is TBD)

Marek