[rabbitmq-discuss] memory usage reporting
Kyle O'Donnell
kyleo at 0b10.mx
Wed Apr 24 18:13:41 BST 2013
So I had a hunch it was erlang R15 so I tried R14 and I am not seeing the same behaviour, and my app works just fine:
Has anyone seen similar issues using R15? Does rabbit actually support R15?
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
22185 prod 20 0 2218m 164m 2648 S 1 0.5 2:03.38 /x/s/erlang-R14B04/lib/erlang/erts-5.8.5/bin/beam.smp -W w -K true -A30 -P 1048576 -- -root /x/s/
$ rabbitmqctl status
Status of node 'rabbit at p1-amqp-j101' ...
[{pid,22185},
{running_applications,
[{rabbitmq_management_visualiser,"RabbitMQ Visualiser","0.0.0"},
{rabbitmq_management,"RabbitMQ Management Console","0.0.0"},
{rabbitmq_management_agent,"RabbitMQ Management Agent","0.0.0"},
{rabbit,"RabbitMQ","3.0.4"},
{os_mon,"CPO CXC 138 46","2.2.7"},
{rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","0.0.0"},
{webmachine,"webmachine","1.9.1-rmq0.0.0-git52e62bc"},
{mochiweb,"MochiMedia Web Server","2.3.1-rmq0.0.0-gitd541e9a"},
{xmerl,"XML parser","1.2.10"},
{inets,"INETS CXC 138 49","5.7.1"},
{mnesia,"MNESIA CXC 138 12","4.5"},
{amqp_client,"RabbitMQ AMQP Client","0.0.0"},
{sasl,"SASL CXC 138 11","2.1.10"},
{stdlib,"ERTS CXC 138 10","1.17.5"},
{kernel,"ERTS CXC 138 10","2.14.5"}]},
{os,{unix,linux}},
{erlang_version,
"Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:30] [hipe] [kernel-poll:true]\n"},
{memory,
[{total,126538808},
{connection_procs,77192},
{queue_procs,322592},
{plugins,247664},
{other_proc,30329304},
{mnesia,92032},
{mgmt_db,195648},
{msg_index,302840},
{other_ets,1738912},
{binary,31832},
{code,19051379},
{atom,1887705},
{other_system,72261708}]},
{vm_memory_high_watermark,0.4},
{vm_memory_limit,13496687001},
{disk_free_limit,1000000000},
{disk_free,21999628288},
{file_descriptors,
[{total_limit,130970},
{total_used,5},
{sockets_limit,117871},
{sockets_used,3}]},
{processes,[{limit,1048576},{used,230}]},
{run_queue,0},
{uptime,506}]
...done.
----- Original Message -----
From: "Kyle O'Donnell" <kyleo at 0b10.mx>
To: "Simon MacMullen" <simon at rabbitmq.com>
Cc: "Discussions about RabbitMQ" <rabbitmq-discuss at lists.rabbitmq.com>
Sent: Wednesday, April 24, 2013 12:50:16 PM
Subject: Re: [rabbitmq-discuss] memory usage reporting
When I attached to the erlang VM it was indeed reporting ~15GB of memory usage
Ubuntu 12.04
Compiled erlang from source
here are my config options:
./configure --prefix=/x/s/erlang-R15B03 --enable-smp-support --enable-hipe --enable-threads --with-ssl
----- Original Message -----
From: "Simon MacMullen" <simon at rabbitmq.com>
To: "Discussions about RabbitMQ" <rabbitmq-discuss at lists.rabbitmq.com>
Cc: "Kyle O'Donnell" <kyleo at 0b10.mx>
Sent: Wednesday, April 24, 2013 12:43:44 PM
Subject: Re: [rabbitmq-discuss] memory usage reporting
Hmm. I am curious. On my machine (Ubuntu 10.04) the amount of memory
reported used in "rabbitmqctl status" matches up much more closely with
RES than with VIRT (which is roughly what you'd expect).
In your case the 17.9GB VIRT vs 269MB RES is too much to be explained by
paged-out memory since you have 5GB of swap that is not used). So I
would expect it to be memory-mapped files or other mysterious things.
However, Erlang certainly thinks that your RabbitMQ process is really
using 15.5GB of memory; the memory breakdown looks plausible and those
numbers do not at all correspond to memory-mapped files or whatever.
So I am not surprised the memory alarm has gone off, what surprises me
is that the OS is reporting so little real memory in use. Which OS is
it? How did you install Erlang?
Cheers, Simon
On 24/04/13 14:49, Kyle O'Donnell wrote:
> Hi,
>
> I've built rabbit (2.8.7 and 3.0.4) from source using R15B03, here
> are the build time options:
>
> make RABBITMQ_BASE=/x/s/rabbitmq-server-3.0.4
> RABBITMQ_LOG_BASE=/x/s/rabbitmq-server-3.0.4/var/log
> RABBITMQ_CONFIG_FILE=/x/s/rabbitmq-server-3.0.4/etc/rabbitmq
> RABBITMQ_MNESIA_DIR=/x/s/rabbitmq-server-3.0.4/var/lib/mnesia
> TARGET_DIR=/x/s/rabbitmq-server-3.0.4
> SBIN_DIR=/x/s/rabbitmq-server-3.0.4/sbin
> MAN_DIR=/x/s/rabbitmq-server-3.0.4/man
>
> When I start my app rabbit quickly reports that it's reached the high
> watermark for memory usage, and it counts virtual memory not resident
> memory:
>
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> 29723 prod 20 0 17.9g 269m 2752 S 4 0.8 2:40.80
> /x/s/erlang-R15B03/lib/erlang/erts-5.9.3.1/bin/beam.smp -W w -K true
> -A30 -P 1048576 -- -root /x/s/erlang-R15B03/lib/erlang -progname erl
> -- -home /home/prod -- -pa /x/s/rabbitmq
>
> There is plenty of free memory on the machine
>
> $ free -m total used free shared buffers
> cached Mem: 32178 2992 29186 0 242
> 1929 -/+ buffers/cache: 820 31358 Swap: 5719
> 0 5719
>
> Is this normal? Curious why rabbit cares about the virtual memory
> size when there is so much actual memory available.
>
> below is the output from rabbitmqctl status
>
> Thanks, Kyle
>
>
> rabbitmqctl status Status of node 'rabbit at p1-amqp-j101' ...
> [{pid,29723}, {running_applications,
> [{rabbitmq_management_visualiser,"RabbitMQ Visualiser","0.0.0"},
> {rabbitmq_management,"RabbitMQ Management Console","0.0.0"},
> {rabbitmq_management_agent,"RabbitMQ Management Agent","0.0.0"},
> {rabbit,"RabbitMQ","3.0.4"}, {mnesia,"MNESIA CXC 138 12","4.7.1"},
> {os_mon,"CPO CXC 138 46","2.2.10"}, {rabbitmq_web_dispatch,"RabbitMQ
> Web Dispatcher","0.0.0"},
> {webmachine,"webmachine","1.9.1-rmq0.0.0-git52e62bc"},
> {mochiweb,"MochiMedia Web Server","2.3.1-rmq0.0.0-gitd541e9a"},
> {xmerl,"XML parser","1.3.2"}, {inets,"INETS CXC 138 49","5.9.2"},
> {amqp_client,"RabbitMQ AMQP Client","0.0.0"}, {sasl,"SASL CXC 138
> 11","2.2.1"}, {stdlib,"ERTS CXC 138 10","1.18.3"}, {kernel,"ERTS
> CXC 138 10","2.15.3"}]}, {os,{unix,linux}}, {erlang_version, "Erlang
> R15B03 (erts-5.9.3.1) [source] [64-bit] [smp:4:4] [async-threads:30]
> [hipe] [kernel-poll:true]\n"}, {memory, [{total,15536914264},
> {connection_procs,89537888}, {queue_procs,18333416},
> {plugins,1170400}, {other_proc,6548309}, {mnesia,2538168},
> {mgmt_db,32239752}, {msg_index,615120}, {other_ets,3557064},
> {binary,15276878352}, {code,20248995}, {atom,801697},
> {other_system,84445103}]}, {vm_memory_high_watermark,0.4},
> {vm_memory_limit,13496687001}, {disk_free_limit,1000000000},
> {disk_free,22007357440}, {file_descriptors, [{total_limit,130970},
> {total_used,1828}, {sockets_limit,117871}, {sockets_used,1826}]},
> {processes,[{limit,1048576},{used,20875}]}, {run_queue,0},
> {uptime,1620}] ...done.
> _______________________________________________ rabbitmq-discuss
> mailing list rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
--
Simon MacMullen
RabbitMQ, VMware
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss at lists.rabbitmq.com
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
More information about the rabbitmq-discuss
mailing list