[rabbitmq-discuss] Stress testing RabbitMQ

Sat May 14 15:02:05 BST 2011

Matthias,

Yes, the binding add/removal is slow. In that case I thought that is more common to be routing messages than adding removing bindings.

Also my goal was to easily get an implementation to see if the route part could be improved by skipping the mnesia:dirty_select. One reason I thought Mnesia could be a problem was because I started getting logs like: Mnesia(rabbit at mrhyde): ** WARNING ** Mnesia is overloaded…

My reasoning was that scanning the table every time a message is routed could be slower than just returning a somehow cached list of bindings, which is what I tried to do.

As you say if the bulk of the job for message routing is done somewhere else, then of course optimizing here won't matter that much. Is like when people get their uber fast web frameworks for hello world benchmarks but then writing or reading form MySQL makes the app slow.

BTW, thanks for taking the time of testing all this.

Alvaro

On May 14, 2011, at 3:42 PM, Matthias Radestock wrote:

> Alvaro,
> 
> Alvaro Videla wrote:
>> Here's my naîve implementation of the exchange:  https://github.com/videlalvaro/rabbitmq-alternate-fanout
>> The binding storage seems to be suboptimal. I think the main
>> difference with the default fanout is how it retrieves the list of
>> matches for the *route* call.
> 
> The representation chosen there results in O(n^2) time complexity for binding addition and removal.
> 
> Btw, for massive fanout the bulk of the time and space complexity is likely in the distribution of the messages to the queues, storage of the messages in the queues, and sending of the messages to consumers. The routing logic should only take up a fraction of the overall time, so by Amdahl's law any improvements there will have little impact on the overall performance.
> 
> Matthias.

Sent form my Nokia 1100