[rabbitmq-discuss] Cluster Memory Usage

Mon Nov 21 22:16:16 GMT 2011

On Mon, Nov 21, 2011 at 3:41 PM, Matthias Radestock
<matthias at rabbitmq.com> wrote:
>
> Have you ever seen the excessive memory use when there were no / very few
> messages in the queue?

Yes.  We've seen this problem when there are only a few dozen messages
and when there
are hundreds of thousands.

Basically, it seems like the slave is always taking up 2-5x the amount
of memory that the
master uses.

>
> Also, what CPU usage do you see when memory use is excessive? On the slave
> you are probably never going to be able to utilise much more than one core,
> since there is only one queue and no connections. So if CPU usage is getting
> near maxing out one core then the system will be operating at the limit.

I do see short periods where cpu utilization goes to 100% for the
beam.smp, but it's not consistently at 100%.  The average cpu
utilization over the life of the process is about 48% right now.  The
system itself is not otherwise overloaded, sitting at about 70% idle
for today.

Looking at the two servers, the slave shows that it's beam.smp has
spent 12 more hours on cpu than the beam.smp on the master.  This
doesn't make sense to me if the slave is doing less work because it's
only handling traffic coming from the master.

master:
UID        PID  PPID  C    SZ   RSS PSR STIME TTY          TIME CMD
rabbitmq 15213 15211 30 246127 550868 4 Nov18 ?        23:30:42
/usr/lib64/erlang/erts-5.8.4/bin/beam.smp -W w -K true -A30 -P 1048576
-- -root /usr/lib64/erlan
rabbitmq 15331 15213  0  2634   452   2 Nov18 ?        00:00:00 inet_gethost 4
rabbitmq 15332 15331  0  4207   748   2 Nov18 ?        00:00:00 inet_gethost 4
rabbitmq 15333 15213  0   950   356   3 Nov18 ?        00:00:00
/usr/lib64/erlang/lib/os_mon-2.2.6/priv/bin/cpu_sup
rabbitmq 21654     1  0  2643   492   2 Nov17 ?        00:00:00
/usr/lib64/erlang/erts-5.8.4/bin/epmd -daemon

slave:
UID        PID  PPID  C    SZ   RSS PSR STIME TTY          TIME CMD
rabbitmq 27918     1  0  2643   364   4 Nov18 ?        00:00:00
/usr/lib64/erlang/erts-5.8.4/bin/epmd -daemon
rabbitmq 27965 27964 47 906682 2334424 5 Nov18 ?       1-12:05:24
/usr/lib64/erlang/erts-5.8.4/bin/beam.smp -W w -K true -A30 -P 1048576
-- -roo
rabbitmq 28065 27965  0  2634   452   7 Nov18 ?        00:00:00 inet_gethost 4
rabbitmq 28066 28065  0  4207   744   7 Nov18 ?        00:00:00 inet_gethost 4
rabbitmq 28067 27965  0   950   360   7 Nov18 ?        00:00:00
/usr/lib64/erlang/lib/os_mon-2.2.6/priv/bin/cpu_sup

Travis

-- 
Travis Campbell
travis at ghostar.org