[rabbitmq-discuss] Question on throughput with RabbitMQ-3.1.1

Tue Jun 25 16:30:06 BST 2013

Hi Micheal ,

Thanks for u r quick reply.
I have already worked on scripts quite a lot that I have shared here quite
so it would help me if you can just have a look at them and give
suggestions based on them.

In the beginning, I had also gone through the pika link, which you sent me
but I could not grasp the basic concept.

What is the concept of blocking connection ?
What difference really it will make over other type of connection ?

Some theory on this would be helpful.

Thanks !

//P

On Tue, Jun 25, 2013 at 4:48 PM, Laing, Michael
<michael.laing at nytimes.com>wrote:

> Read this carefully: https://pika.readthedocs.org
>
> In particular, look at the 'Asynchronous publisher example' and get it to
> work - then try modifying it to publish as fast possible, remove asynch
> confirms, etc.
>
> Then install tornado and try using the TornadoConnection instead of the
> SelectConnection employed in the example.
>
> ml
>
>
> On Tue, Jun 25, 2013 at 10:13 AM, PRIYANKI VASHI <vashi.priyanki at gmail.com
> > wrote:
>
>> Yes I use blocking connection.
>>
>> I am comparitively new to python as well so Can please u explain the
>> difference between blockin connection n tornedo adaptor connection ?
>>
>> Can u share some example with tornedo type connection ?
>>
>> Sent from my iPhone
>>
>> On 25 jun 2013, at 15:24, "Laing, Michael" <michael.laing at nytimes.com>
>> wrote:
>>
>> I know a lot about the pika client but I don't see any code.
>>
>> On my old mac laptop, using the pika tornado adapter (or the libev
>> adapter not yet released), I can publish ~20K msgs/sec (4KBytes each),
>> saturating my network adapter. My servers are small, and process msgs more
>> slowly, so network buffers fill up, and TCP backpressure is exerted, but
>> everything completes as expected.
>>
>> Other pika async adapters run at about 3K msgs/sec in similar runs on my
>> mac; I don't work at all with the pika BlockingConnection so you are on
>> your own if you are using that.
>>
>> ml
>>
>>
>> On Tue, Jun 25, 2013 at 8:59 AM, Emile Joubert <emile at rabbitmq.com>wrote:
>>
>>>
>>> Hi,
>>>
>>> On 25/06/13 13:38, Priyanki Vashi wrote:
>>>
>>> > I have very high end server with 20 CPUs and 120 GB of RAM so I think
>>> > resources wise it's not the bottleneck.
>>>
>>> And what about the network speed? I would still try an independent
>>> bandwidth test from network to disk to give you a comparison reference.
>>>
>>> The numbers you quoted are about 2 orders of magnitude lower than I
>>> would expect for that hardware. Bear in mind that the queue process is
>>> typically the most CPU-intensive process. A queue on a single server can
>>> occupy at most one CPU, so will benefit more from a faster CPU than from
>>> many CPUs.
>>>
>>> > Did u and Tim received my scripts ?
>>>
>>> Not yet, but hopefully someone who knows more about the Pika client than
>>> me will be able to comment.
>>>
>>> > I learnt that basic_consume is better choice than basic_get since
>>> server
>>> > will directly send messages to listening consumer without consumer
>>> > polling it.
>>>
>>> Yes, and the asynchronous choice is typically faster.
>>>
>>>
>>> If you are still stuck with very low throughput then I would recommend
>>> you try the MulticastMain utility, included in the RabbitMQ Java client.
>>> It includes support for many of the options that you want to compare,
>>>
>>>
>>>
>>> -Emile
>>>
>>>
>>>
>>> _______________________________________________
>>> rabbitmq-discuss mailing list
>>> rabbitmq-discuss at lists.rabbitmq.com
>>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>>
>>
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.rabbitmq.com
>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>
>>
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.rabbitmq.com
>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>
>>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130625/1743d4e4/attachment.htm>