[rabbitmq-discuss] beam instances and growing memory usage
matthias at lshift.net
Sat Nov 17 08:46:15 GMT 2007
> When we did a complete site restart on Tuesday night, Wednesday worked
> flawlessly. We rebuilt the cluster (disk based nodes) and created a
> total of 7 brokers on the box.
When you are saying "disk based nodes", do you mean that all broker
nodes are configured as disk nodes? That is not good for performance. If
you have all nodes on a single machine and they all write to the same
disk then there is no point having more than one disk node.
> When I was watching the instances on Wednesday, each instance was
> taking up about 0.1% of the server ram, according to top. I checked
> yesterday (Thursday) and each instance was taking 2.0%. I just looked
> now (Friday) and it's taking 2.9%. Durable is set to false - so it
> doesn't appear that things should be persisting anywhere. Any ideas
> why it'd be growing over time?
The growth is not necessarily anything to worry about. You still have
plenty of memory left and the growth might stop after a while. Keep an
eye on it.
It is worth checking the queue sizes. Connect to one of the nodes in
your cluster - any node will do - and run rabbit_amqqueue:stat_all(), e.g.
$ erl -sname debug -hidden -remsh rabbit at tanto
Erlang (BEAM) emulator version 5.5.5 [source] [async-threads:0] [hipe]
Eshell V5.5.5 (abort with ^G)
(rabbit at tanto)1> rabbit_amqqueue:stat_all().
(rabbit at tanto)2> CTRL-c CTRL-c
That will list all the queues in the system, their message queue length
and their number of consumers.
On another topic, from your 'top' reports I notice that although the CPU
load is unevenly distributed over the nodes, the distribution changes
over time and the cumulative CPU times are much more even. So it looks
like your load-balancing logic is working fine over the long run.
Curiously though there are two nodes in your latter report which have
accumulated much less CPU time.
More information about the rabbitmq-discuss