[rabbitmq-discuss] Question on throughput with RabbitMQ-3.1.1

[Priyanki Vashi]

Hi Emile ,

Thank you for your reply. I had also looked at the link you shared in the
reply.
I understand that throughout is a function of different parameters and it
would behave based on configuration of these parameters.

But since I am doing a detailed performance study, my goal will be to try
different parameter configuration and see the difference in throughout ,
processor load etc. etc so that I finally propose a suitable configuration.
My requirement is to atleast have 10,000 msg/sec. For maintaining this
throughput requirement with zero message loss and 100% availability of
broker, I started with 4 node cluster with durable queues and exchanges.
Since I knew that persistent messages would trigger disk activity, first I
want to try with non-persistent messages and see what is throghput,
processor utilization etc. etc.

I have very high end server with 20 CPUs and 120 GB of RAM so I think
resources wise it's not the bottleneck.

And as Tim pointed out before, I am also suspecting that bottleneck might
be in the way I am using publisher/confirm and consumer ack. I have shared
my scripts also now.

Did u and Tim received my scripts ?
It would really help me if you guys can have quick look at attached scripts
and guide me if the way I am using publisher confirm and consumer ack is
efficient and correct or not for securing message deliveries both by
rabbitMQ server and then by consumer/s.

I also have configured prefetch count value but still I don't see any
difference in my throughout bcoz may be bottleneck is in my producer.

Also, I tried to simulate four producers (each opening up it's own TCP
connection to server , each have it's own channel and exchange) but I still
don't get higher throughput than 300 msg/sec. I have a single python
script, which starts four producers and it's only one process.

Here I am suspecting that since it's just a single process, it still behave
the same as single producer. If I run the same script again ,y throughout
doubles which is 600 msg/sec and I see that there are two processes each
connected with one producer script.

I can see through web interface of rabbitMQ that my automatic producer
script works since connection, channel gets updated accordingly.

I also wanted to know what is internal mechanism within basic_publish and
basic_consume.
I learnt that basic_consume is better choice than basic_get since server
will directly send messages to listening consumer without consumer polling
it.

I hope, I have explained myself well and in detail manner for you guys to
able to help/guide me.

Thanks !
//Priyanki.


On Tue, Jun 25, 2013 at 1:06 PM, Emile Joubert <emile at rabbitmq.com> wrote:

>
>
> Hi,
>
> On 24/06/13 16:55, Priyanki Vashi wrote:
> > Also configured HAproxy TCP Load balancer so that I can just provide
> > single port to connect to the Cluster.
>
> A mirrored queue will be slower than a queue on a single server of the
> same spec.
>
> > -1 producer and No consumer and no queue binded to exchange
>
> In this case the broker throws messages away without queueing so it
> should be extremely fast.
>
> > Also what points to take care, in order to improve throughput ?
>
> There are several factors that could have a bearing on the overall
> throughput. Here is a similar question I answered  recently:
>
> http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2013-March/026132.html
>
> The numbers you quote are very low. For comparison, measure the network
> and IO bandwidth by piping traffic across the network to disk (e.g.
> using netcat) to determine what performance your servers are capable of.
>
>
>
> -Emile
>
>
>
>
>
>
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130625/60c88792/attachment.htm>