[rabbitmq-discuss] Question on throughput with RabbitMQ-3.1.1

Tue Jun 25 15:48:30 BST 2013

Read this carefully: https://pika.readthedocs.org

In particular, look at the 'Asynchronous publisher example' and get it to
work - then try modifying it to publish as fast possible, remove asynch
confirms, etc.

Then install tornado and try using the TornadoConnection instead of the
SelectConnection employed in the example.

ml

On Tue, Jun 25, 2013 at 10:13 AM, PRIYANKI VASHI
<vashi.priyanki at gmail.com>wrote:

> Yes I use blocking connection.
>
> I am comparitively new to python as well so Can please u explain the
> difference between blockin connection n tornedo adaptor connection ?
>
> Can u share some example with tornedo type connection ?
>
> Sent from my iPhone
>
> On 25 jun 2013, at 15:24, "Laing, Michael" <michael.laing at nytimes.com>
> wrote:
>
> I know a lot about the pika client but I don't see any code.
>
> On my old mac laptop, using the pika tornado adapter (or the libev adapter
> not yet released), I can publish ~20K msgs/sec (4KBytes each), saturating
> my network adapter. My servers are small, and process msgs more slowly, so
> network buffers fill up, and TCP backpressure is exerted, but everything
> completes as expected.
>
> Other pika async adapters run at about 3K msgs/sec in similar runs on my
> mac; I don't work at all with the pika BlockingConnection so you are on
> your own if you are using that.
>
> ml
>
>
> On Tue, Jun 25, 2013 at 8:59 AM, Emile Joubert <emile at rabbitmq.com> wrote:
>
>>
>> Hi,
>>
>> On 25/06/13 13:38, Priyanki Vashi wrote:
>>
>> > I have very high end server with 20 CPUs and 120 GB of RAM so I think
>> > resources wise it's not the bottleneck.
>>
>> And what about the network speed? I would still try an independent
>> bandwidth test from network to disk to give you a comparison reference.
>>
>> The numbers you quoted are about 2 orders of magnitude lower than I
>> would expect for that hardware. Bear in mind that the queue process is
>> typically the most CPU-intensive process. A queue on a single server can
>> occupy at most one CPU, so will benefit more from a faster CPU than from
>> many CPUs.
>>
>> > Did u and Tim received my scripts ?
>>
>> Not yet, but hopefully someone who knows more about the Pika client than
>> me will be able to comment.
>>
>> > I learnt that basic_consume is better choice than basic_get since server
>> > will directly send messages to listening consumer without consumer
>> > polling it.
>>
>> Yes, and the asynchronous choice is typically faster.
>>
>>
>> If you are still stuck with very low throughput then I would recommend
>> you try the MulticastMain utility, included in the RabbitMQ Java client.
>> It includes support for many of the options that you want to compare,
>>
>>
>>
>> -Emile
>>
>>
>>
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.rabbitmq.com
>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130625/df305182/attachment.htm>