[rabbitmq-discuss] Question on throughput with RabbitMQ-3.1.1

Tue Jun 25 16:55:39 BST 2013

Hi Simon, 

When u say I am using confirm delivery, isn't that the way one confirm message has reached to rabbitmq server ?

This is what I understood as publisher confirm usage. 
Am i wrong in my understanding ?

Also if i want to make sure there is zero message loss both when i publish to server (using above method) as well as by explicitly making sure consumer ack method ?

If the way i do is not correct then what is the correct way to ensure zero message loss ? 

I apologize if i am asking too many questions here and if troubling u but i am little bit behind to understand  what is that i m doing wrong ?
also how it is correctly done?
And once i have this understanding then its easy to make changes in my scripts. 

Thanks !

P

Sent from my iPhone

On 25 jun 2013, at 15:16, Simon MacMullen <simon at rabbitmq.com> wrote:

> Just from quickly scanning your script I notice that you are invoking:
> 
>        self.channel.confirm_delivery()
> 
> on every publish.
> 
> As far as I can see from the pika source code this will send confirm.select. That's a synchronous method, so you're incurring a network round trip per publish. That's almost certainly where all your performance is going. You're also not passing in a callback, so it's not really accomplishing anything client-side.
> 
> Cheers, Simon
> 
> On 25/06/13 14:07, Priyanki Vashi wrote:
>> Hi ,
>> 
>> By mistake I had attached a wrong producer in my previous  zip file so
>> attaching the correct one now.
>> Sorry for that but I have so many different files, which I have tried.
>> So resending the zip files again with correct producer and consumer (I
>> believe)
>> 
>> //P
>> 
>> 
>> 
>> 
>> On Tue, Jun 25, 2013 at 2:59 PM, Emile Joubert <emile at rabbitmq.com
>> <mailto:emile at rabbitmq.com>> wrote:
>> 
>> 
>>    Hi,
>> 
>>    On 25/06/13 13:38, Priyanki Vashi wrote:
>> 
>>     > I have very high end server with 20 CPUs and 120 GB of RAM so I think
>>     > resources wise it's not the bottleneck.
>> 
>>    And what about the network speed? I would still try an independent
>>    bandwidth test from network to disk to give you a comparison reference.
>> 
>>    The numbers you quoted are about 2 orders of magnitude lower than I
>>    would expect for that hardware. Bear in mind that the queue process is
>>    typically the most CPU-intensive process. A queue on a single server can
>>    occupy at most one CPU, so will benefit more from a faster CPU than from
>>    many CPUs.
>> 
>>     > Did u and Tim received my scripts ?
>> 
>>    Not yet, but hopefully someone who knows more about the Pika client than
>>    me will be able to comment.
>> 
>>     > I learnt that basic_consume is better choice than basic_get since
>>    server
>>     > will directly send messages to listening consumer without consumer
>>     > polling it.
>> 
>>    Yes, and the asynchronous choice is typically faster.
>> 
>> 
>>    If you are still stuck with very low throughput then I would recommend
>>    you try the MulticastMain utility, included in the RabbitMQ Java client.
>>    It includes support for many of the options that you want to compare,
>> 
>> 
>> 
>>    -Emile
>> 
>> 
>> 
>> 
>> 
>> 
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.rabbitmq.com
>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> 
> 
> -- 
> Simon MacMullen
> RabbitMQ, Pivotal