[rabbitmq-discuss] memory usage reporting

[Kyle O'Donnell]

Matthias,

I don't know if there is anything confidential in the report output.  Is there something specific I can send you (the output is ~10k lines)?

I don't know specifically what our app is doing, though I can ask the development team specific questions for you... If I can simplify/test it I'll let you know.

I think there are 2 possible issues, 1) erlang might be looking in the wrong place to get its memory usage information and 2) something is causing erlang to reserve large chunks of virtual memory.

I also tried one last thing... the build I had of R14 was about 12months old.  I wanted to make sure it was using the exact same configure options my R15 and R16 builds.  I recompiled R14, and rabbit and I am not seeing the huge virtual memory usage:

notice the resident memory usage and the memory reported by rabbit status are very close [120mb vs 166mb]

also notice that the amount of memory allocated to 'binary' is almost nil, whereas with R15/R16 it was huge..


  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                         
26856 prod      20   0 2217m 166m 2640 S    1  0.5   2:03.78 beam.smp                                                                                         

Status of node 'rabbit at p1-amqp-j101' ...
[{pid,26856},
 {running_applications,
     [{rabbitmq_management_visualiser,"RabbitMQ Visualiser","0.0.0"},
      {rabbitmq_management,"RabbitMQ Management Console","0.0.0"},
      {rabbitmq_management_agent,"RabbitMQ Management Agent","0.0.0"},
      {rabbit,"RabbitMQ","3.0.4"},
      {mnesia,"MNESIA  CXC 138 12","4.5"},
      {os_mon,"CPO  CXC 138 46","2.2.7"},
      {rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","0.0.0"},
      {webmachine,"webmachine","1.9.1-rmq0.0.0-git52e62bc"},
      {mochiweb,"MochiMedia Web Server","2.3.1-rmq0.0.0-gitd541e9a"},
      {xmerl,"XML parser","1.2.10"},
      {inets,"INETS  CXC 138 49","5.7.1"},
      {amqp_client,"RabbitMQ AMQP Client","0.0.0"},
      {sasl,"SASL  CXC 138 11","2.1.10"},
      {stdlib,"ERTS  CXC 138 10","1.17.5"},
      {kernel,"ERTS  CXC 138 10","2.14.5"}]},
 {os,{unix,linux}},
 {erlang_version,
     "Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:30] [hipe] [kernel-poll:true]\n"},
 {memory,
     [{total,126532760},
      {connection_procs,225328},
      {queue_procs,414736},
      {plugins,116176},
      {other_proc,30207680},
      {mnesia,97560},
      {mgmt_db,165176},
      {msg_index,302840},
      {other_ets,1744392},
      {binary,34256},
      {code,19111543},
      {atom,1887705},
      {other_system,72225368}]},
 {vm_memory_high_watermark,0.4},
 {vm_memory_limit,13496687001},
 {disk_free_limit,1000000000},
 {disk_free,21576155136},
 {file_descriptors,
     [{total_limit,130970},
      {total_used,5},
      {sockets_limit,117871},
      {sockets_used,3}]},
 {processes,[{limit,1048576},{used,264}]},
 {run_queue,0},
 {uptime,578}]
...done.


----- Original Message -----
From: "Matthias Radestock" <matthias at rabbitmq.com>
To: "Kyle O'Donnell" <kyleo at 0b10.mx>
Cc: "Discussions about RabbitMQ" <rabbitmq-discuss at lists.rabbitmq.com>
Sent: Wednesday, April 24, 2013 5:08:30 PM
Subject: Re: [rabbitmq-discuss] memory usage reporting

Kyle,

On 24/04/13 21:49, Kyle O'Donnell wrote:
> R16B seems to behave the same way R15B03 does:

Interesting. Do you have a 'rabbitmqctl report' to go with that?

What is your app actually doing? Looks like it opens a fair number of 
connections, and publish quite a lot of (or some very large) messages. 
Presumably it's consuming them too. The difference in behaviour could be 
due to very small scheduling differences in the VM, or, if you are 
running the producers and consumers on the same machine as rabbit, 
differences in Erlang CPU usage and consequent O/S level scheduling.

Can you condense the behaviour into a test case that we can run against 
different erlang versions and observe the difference in behaviour?

Regards,

Matthias.