[rabbitmq-discuss] RabbitMQ QUEUE MESSAGES

Thu Jul 11 16:40:46 BST 2013

NuPogodi:

> Taking into account that correct queue implementation should be of O(1) time
> complexity should the issues with RabbitMQ performance when queues grow
> treated as a bug that will be fixed or as a "feature" that is not going to
> be addressed because Rabbit focuses on efficient message routing and not
> efficient queuing?

O(1) appends and other queue characteristics in CS literature do not take into account
the fact that your consumers can fall well behind your producers and RAM is a limited resource.
Both are purely operational issues.

In infrastructure tools such as RabbitMQ, they are very real. When a queue starts to grow,
RabbitMQ tries to be reasonably smart and move messages to disk.
In the default message index implementation there is some per message memory cost.

There is a plugin, rabbitmq toke, that uses Tokyo Cabinet for message index. Many installations,
however, operate perfectly fine with the default impl and the Tokyo Cabinet dependency complicates deployment, so it's not used by default.

It would not say it's fair to claim that RabbitMQ "focuses on efficient routing and not efficient queueing" because the issue is not of "efficient queueing" but balancing
performance difference of disk vs RAM in situations when a node is physically low on RAM.
If you are not low on RAM, you likely won't encounter any issues.

It's something that can be improved, but that's not entirely trivial to do well.
--
MK