[rabbitmq-discuss] rabbitmq gets high load

[Doug]

Thanks Tim.

This is the info in sasl log (many of these):

=SUPERVISOR REPORT==== 17-Jan-2013::17:00:31 ===
     Supervisor: {<0.31128.1705>,
                                           rabbit_channel_sup_sup}
     Context:    shutdown_error
     Reason:     noproc
     Offender:   [{pid,<0.31130.1705>},
                  {name,channel_sup},
                  {mfa,{rabbit_channel_sup,start_link,[]}},
                  {restart_type,temporary},
                  {shutdown,infinity},
                  {child_type,supervisor}]


=SUPERVISOR REPORT==== 17-Jan-2013::17:02:41 ===
     Supervisor: {<0.30250.1707>,
                                           rabbit_channel_sup_sup}
     Context:    shutdown_error
     Reason:     shutdown
     Offender:   [{pid,<0.30252.1707>},
                  {name,channel_sup},
                  {mfa,{rabbit_channel_sup,start_link,[]}},
                  {restart_type,temporary},
                  {shutdown,infinity},
                  {child_type,supervisor}]


And this is the status report:

Status of node 'testmq-master at ubuntu' ...
[{pid,17994},
 {running_applications,[{rabbit,"RabbitMQ","2.8.2"},
                        {os_mon,"CPO  CXC 138 46","2.2.9"},
                        {sasl,"SASL  CXC 138 11","2.2.1"},
                        {mnesia,"MNESIA  CXC 138 12","4.7"},
                        {stdlib,"ERTS  CXC 138 10","1.18.1"},
                        {kernel,"ERTS  CXC 138 10","2.15.1"}]},
 {os,{unix,linux}},
 {erlang_version,"Erlang R15B01 (erts-5.9.1) [source] [64-bit]
[smp:8:8] [async-threads:30] [kernel-poll:true]\n"},
 {memory,[{total,394953040},
          {processes,313197287},
          {processes_used,313056817},
          {system,81755753},
          {atom,1788105},
          {atom_used,1770587},
          {binary,16938376},
          {code,11936253},
          {ets,5830120}]},
 {vm_memory_high_watermark,0.6},
 {vm_memory_limit,10114781184},
 {disk_free_limit,16857968640},
 {disk_free,206313144320},
 {file_descriptors,[{total_limit,65435},
                    {total_used,151},
                    {sockets_limit,58889},
                    {sockets_used,137}]},
 {processes,[{limit,1048576},{used,2867}]},
 {run_queue,13},
 {uptime,8705279}]
...done.



2013/1/17 Tim Watson <tim at rabbitmq.com>:
> Hi Doug
>
> On 17 Jan 2013, at 04:29, Doug wrote:
>
>>  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
>> 17994 rabbitmq  20   0 11.6g  10g 1764 S  367 66.2 416834:07 beam.smp
>>
>> As you see, rabbitmq process has lots of system load (367% CPU, 66.2% memory).
>
> Certainly looks like it. That level of CPU utilisation is a bit surprising.
>
>> It even can't list the queue now:
>>
>> root at ubuntu:~# rabbitmqctl -p /pay list_queues
>> Listing queues ...
>> ^C
>> BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded
>>       (v)ersion (k)ill (D)b-tables (d)istribution
>>
>>
>> So how to handle this case? Thanks.
>
> You've not given us much to go on here. Normally I'd ask you to run the output of `rabbitmqctl report` but that's likely to lock up just as list_queues is doing. Can you post the regular as sasl rabbit logs please? If they're too bit to post, a /paste/ url will do fine and of course you may wish to scrub the logs to remove any private information if that's relevant.
>
> Also what kind of /usage/ is this rabbit subject to at the time? Is it serving a large number of producers, consumers, both? Have you looked at any stats around the time this starts happening in order to see what the broker is up to?
>
> Cheers,
> Tim
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss