[rabbitmq-discuss] The RabbitMQ uses very much memory in the cluster pattern

[Matthias Radestock]

On 14/08/12 07:14, Liu Hao wrote:
>       The connections and channels are actually too much. I decrease the
> connections and channels. Now, I have 40 consumer connections (one
> connection with one channel), and 50 producer connections (one
> connection with 10 channels). The memory is the same , acquires a lot.
>
>       But I find an interesting fact that If I use 50 producer
> connections (one connection with only one channel) , the memory will be
> under 2G, but the most connections are flowed and the publish rate is
> too slow.
>
>       This is just a test demo, and one message is 10KB.
>
>       The command report is so big(35M), and I give the beginning and
> the end of the output to you as the attachment.

I think you are simply pushing rabbit beyond the limit of its 
capability. Internal flow control happens on a per-process-link basis, 
so when you increase the number of publishing channels that corresponds 
to a linear increase in the amount of internal buffer space that is 
potentially required. To the point where all memory is taken up by 
messages sitting in these buffers.

Publishing across nodes carries an extra cost, so the buffers will fill 
up at lower publishing rates.

If with 50 producer connections x 1 channel you see most connections 
flowed, then that is an indications that rabbit is already operating at 
capacity but is still able to keep overall memory use under control. 
Adding more producer connections/channels will not increase the 
sustainable sending rate but will degrade rabbit's ability to control 
memory use.


Btw, I suggest you upgrade to the latest rabbit version - the flow 
control code has changed somewhat and there have been some performance 
improvements.

Regards,

Matthias.