[rabbitmq-discuss] Question on throughput with RabbitMQ-3.1.1

Tue Jun 25 14:24:14 BST 2013

I know a lot about the pika client but I don't see any code.

On my old mac laptop, using the pika tornado adapter (or the libev adapter
not yet released), I can publish ~20K msgs/sec (4KBytes each), saturating
my network adapter. My servers are small, and process msgs more slowly, so
network buffers fill up, and TCP backpressure is exerted, but everything
completes as expected.

Other pika async adapters run at about 3K msgs/sec in similar runs on my
mac; I don't work at all with the pika BlockingConnection so you are on
your own if you are using that.

ml

On Tue, Jun 25, 2013 at 8:59 AM, Emile Joubert <emile at rabbitmq.com> wrote:

>
> Hi,
>
> On 25/06/13 13:38, Priyanki Vashi wrote:
>
> > I have very high end server with 20 CPUs and 120 GB of RAM so I think
> > resources wise it's not the bottleneck.
>
> And what about the network speed? I would still try an independent
> bandwidth test from network to disk to give you a comparison reference.
>
> The numbers you quoted are about 2 orders of magnitude lower than I
> would expect for that hardware. Bear in mind that the queue process is
> typically the most CPU-intensive process. A queue on a single server can
> occupy at most one CPU, so will benefit more from a faster CPU than from
> many CPUs.
>
> > Did u and Tim received my scripts ?
>
> Not yet, but hopefully someone who knows more about the Pika client than
> me will be able to comment.
>
> > I learnt that basic_consume is better choice than basic_get since server
> > will directly send messages to listening consumer without consumer
> > polling it.
>
> Yes, and the asynchronous choice is typically faster.
>
>
> If you are still stuck with very low throughput then I would recommend
> you try the MulticastMain utility, included in the RabbitMQ Java client.
> It includes support for many of the options that you want to compare,
>
>
>
> -Emile
>
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130625/cc3c436c/attachment.htm>