[rabbitmq-discuss] Performance and scalability measurement with my publisher-subscriber clients testing

[Marek Majkowski]

On Wed, Mar 9, 2011 at 10:26, Deepak Vijayvergiy <vdeepak at yahoo-inc.com> wrote:
> I have created the framework for publisher-subscriber and created publisher and consumer clients for basic testing as well. Now, I also want to test the performance as well, as in, I want to give the numbers like -
> (1) How many messages, my publisher client will be able to publish per second or rate?
> (2) How many messages my consumer will be able to consume (per second) .. rate? (a) with messages already in the queue (b) parallely publishing as well
>
> Earlier I have tried doing same by some time functions before and after 1 and last mess respectively, but I don't think that was a proper way. Please let me know if you have done the same measurement?

We do have a tool for doing simple measurements, see MulticastMain
in our java client.  Sample usage:
   http://www.rabbitmq.com/ec2.html#testing
(dig the mailing list archives for more examples)

Doing benchmarks properly, even as simple as you described, is pretty hard.

With 1) you will measure performance of the client. Unless you're using
publisher-acks.

2) It all depends, if data in-memory or on the disk, is it cached by the OS,
how big is the queue, how many queues are handled by rabbit and so on.

2b) That one is quite interesting, but you have to be very careful to keep
the queue balanced. Publishing is usually faster than consuming.

Rabbit is not a rocket science and there are some significant bottlenecks.
If you start swapping to disk or if you use persistent messages -
things will be much slower.

Usually the best approach is to test the whole solution end-to-end
rather than focusing on dry RabbitMQ benchmarks. That way you can
be sure that the numbers actually mean something you're interested
in. Raw benchmarks like publishes/sec or consumes/sec doesn't
say anything useful.

Cheers,
  Marek