[rabbitmq-discuss] Question on throughput with RabbitMQ-3.1.1

Tue Jun 25 14:50:05 BST 2013

Hi Michel ,

Resending the code again. Not sure why you don't see but I have sent it
around 1 hr ago.

Also a good advise on efficient publisher confirm and consumer ack would be
highly appreciated.

//P

On Tue, Jun 25, 2013 at 3:24 PM, Laing, Michael
<michael.laing at nytimes.com>wrote:

> I know a lot about the pika client but I don't see any code.
>
> On my old mac laptop, using the pika tornado adapter (or the libev adapter
> not yet released), I can publish ~20K msgs/sec (4KBytes each), saturating
> my network adapter. My servers are small, and process msgs more slowly, so
> network buffers fill up, and TCP backpressure is exerted, but everything
> completes as expected.
>
> Other pika async adapters run at about 3K msgs/sec in similar runs on my
> mac; I don't work at all with the pika BlockingConnection so you are on
> your own if you are using that.
>
> ml
>
>
> On Tue, Jun 25, 2013 at 8:59 AM, Emile Joubert <emile at rabbitmq.com> wrote:
>
>>
>> Hi,
>>
>> On 25/06/13 13:38, Priyanki Vashi wrote:
>>
>> > I have very high end server with 20 CPUs and 120 GB of RAM so I think
>> > resources wise it's not the bottleneck.
>>
>> And what about the network speed? I would still try an independent
>> bandwidth test from network to disk to give you a comparison reference.
>>
>> The numbers you quoted are about 2 orders of magnitude lower than I
>> would expect for that hardware. Bear in mind that the queue process is
>> typically the most CPU-intensive process. A queue on a single server can
>> occupy at most one CPU, so will benefit more from a faster CPU than from
>> many CPUs.
>>
>> > Did u and Tim received my scripts ?
>>
>> Not yet, but hopefully someone who knows more about the Pika client than
>> me will be able to comment.
>>
>> > I learnt that basic_consume is better choice than basic_get since server
>> > will directly send messages to listening consumer without consumer
>> > polling it.
>>
>> Yes, and the asynchronous choice is typically faster.
>>
>>
>> If you are still stuck with very low throughput then I would recommend
>> you try the MulticastMain utility, included in the RabbitMQ Java client.
>> It includes support for many of the options that you want to compare,
>>
>>
>>
>> -Emile
>>
>>
>>
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.rabbitmq.com
>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130625/2f557337/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Producer_Consumer_Scripts_latest.zip
Type: application/zip
Size: 6584 bytes
Desc: not available
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130625/2f557337/attachment.zip>