[rabbitmq-discuss] memory consumption during message delivery

[Cermak, Marek]

Hi everybody,
I did some measurements of memory consumption of RMQ (v3.0.2) during sending and delivering messages and found an interesting behaviour during delivery. In short: messages were stored in mnesia transient storage on disk and when a consumer starts to consume them all of them were read to memory in one chunk which was almost deadly for RMQ and the server.

The scenario was 1+2+8+10 pairs of bound direct exchanges and queues each received messages of particular size (1kB, 64 kB, 1 MB and 8MB) and after a while consumer was started to 'empty' each of the queues. The consumer of the queue with biggest messages triggered the avalanche of data being transferred from disk to memory. Similar behaviour can be seen always when you fill the queue up to the roof with bigger messages and start the consumer. I've created the chart where you can see it clearly.

Tested with 1G of memory, 64b Debian GNU/linux, Erlang R15B, RMQ v3.0.2.

Is it an expected behaviour? Can it be somehow tuned/adjusted? Thanks.

Marek


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130403/73bf1903/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: combo_020.gif
Type: image/gif
Size: 29688 bytes
Desc: combo_020.gif
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130403/73bf1903/attachment.gif>