[rabbitmq-discuss] Is RabbitMQ indeed "hype ware"?

[Ben Hood]

Roberto,

On Sat, Aug 29, 2009 at 3:20 PM, Dikkie Hagel<dikkiehagel at hotmail.com> wrote:
> We've started our development with ActiveMQ. Everything seemed to work fine
> until we started connecting with 1000 consumers to a single topic. ActiveMQ
> could not handle this at all, complaining about "too many open filehandles".
> Others seem to have encountered this same issue as well (see second life's
> review), but we've not been able to google a solution.

Were you not able to use ulimit to increase the OS limit for open file handles?

> We then turned to RabbitMQ, and after fixing the Perl and ActionScript AMQP
> interfaces (http://github.com/joodie) we now almost have a working prototype
> for our environment, ready for some first performance tests.

Interesting. Can you tell us what fixes had to be made? I've seen that
as3-amqp was forked yesterday, but I don't know if that is what you
are actually referring to. I've noticed that the same github user also
forked net-amqp as well. Is this related?

> ============== START OF QUOTE =================
> This is due to the configuration out of the box for ActiveMQ - its very
> straightforward for ActiveMQ to handle the same load as RabbitMQ.
> There are some significant benefits in using ActiveMQ over RabbitMQ.
> ActiveMQ has no limit on the depth (number of messages) in a Queue -
> other than disk space. RabbitMQ runs out of memory.

In Rabbit 1.6 this is true. Once you run out of physical memory it
will start running slowly, and then after that you will hit the wall
when you exceed virtual memory. Hence the major feature of the
upcoming release is queue paging.

> Also - there's performance benefits too - although sending messages can
> be a lot faster to RabbitMQ (this is due to RabbitMQ caching messages in
> memory) - consuming messages is considerably slower.

RabbitMQ endeavours to deliver messages as opposed to queueing them,
so I'm not too sure that there is something methodologically wrong
there. However, it may be the case that ActiveMQ benchmarks better
than RabbitMQ, for this or other reasons. I guess the simplest way to
find out is to just run some benchmarks on both systems and see what
transpires.

> In reality, RabbitMQ is just hype ware - not really suitable for
> production scenarios. Although its getting a lot of traction, it only
> suitable for cases where consumers will always keep up with producers.

As I said previously, not being able to page is a known issue in Rabbit 1.6.

Whether or not is suitable for production is not up to me to say.
After all, it is just some open source software that you can download,
run and test for yourself.

I guess another way to find out is to ask the community and google for
archives and blogs for people indicating that Rabbit doesn't work.

> Now that's a pretty scary assesment there. "Hype ware"... Perhaps RabbitMQ's
> engineers can reply to this, and reassure us that our shift in favour of
> RabbitMQ is a sound decision? (we plan to deploy it on systems with plenty
> gigs of RAM).

Hypeware is cool name :-)

As for your production issues, it's kind of hard to respond to an
issue that hasn't arisen yet. I would just test it and find out for
yourself. Do the same with ActiveMQ and just choose the system that
suits you best.

Also, I think the pending 1.7 release will mitigate the issue of true
store and forward.

So if you app really does have some heavy paging requirements it would
be interesting to test out the new persister mechanism to see if
actually works .... would be cool to see :-)

HTH,

Ben