[rabbitmq-discuss] Stress testing RabbitMQ

Sat May 14 13:51:54 BST 2011

Hi Matthew,

Here's my naîve implementation of the exchange: 

	https://github.com/videlalvaro/rabbitmq-alternate-fanout

The binding storage seems to be suboptimal. I think the main difference with the default fanout is how it retrieves the list of matches for the *route* call.

Then this repo has the code for the benchmarks. I've just updated it, with the instructions I used on my machine: 

	https://github.com/videlalvaro/massive_fanout

To get the memory *explosion* thing happening, I had to set up the memory_high_watermark to 0.1 for my 4gb Mac. In the real world nobody will do that I guess.

I tried with 5000 queues. To do that perform this call:

	alternate_fanout:start(5000).

The README of the massive_fanout repo explain the usage.

As a side note, from my point of view my benchmarks are not scientific at all, I would compare them with those useless "hello world" benchmarks for web frameworks. So as RabbitMQ user I wouldn't be concerned about the results. It was interesting for me to run them and I think I learned quite a bit of RabbitMQ internals.

-Alvaro

On May 14, 2011, at 1:57 PM, Matthew Sackman wrote:

> On Sat, May 14, 2011 at 12:32:41PM +0100, Matthew Sackman wrote:
>> I have not looked into your benchmark, however, what size msgs are you
>> using? I suspect that if the binary fragments end up under 64 bytes then
>> that is the cause of memory explosion: each msg is being _copied_ rather
>> than reference counted to each queue. If you try with msgs with a bigger
>> payload, I wonder whether you see the same behaviour.
> 
> Hmm, I've changed my mind on that. Even with 64-byte binaries being
> copied to 8k queues, we're only talking 0.5MB, which is very unlikely to
> be an issue unless the binary alloctor is horribly slow, which I'm
> fairly sure it's not.
> 
> Matthew
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

Sent form my Nokia 1100