[rabbitmq-discuss] cpu & network usage

Wed Dec 4 23:11:16 GMT 2013

Hello there,

     As a performance optimization on our rabbit mq cluster (of 2 nodes), I
removed following from a high volume non-durable queue

   1. AMQP_MANDATORY flag during publish and
   2. AMQP_AUTOACK during consume.

 As a result the number of messages on that queue went down from 150 to 15
which is fantastic and expected. But this also increased overall cpu usage
from 22% to 32% with an increased network traffic from 30k packets/sec to
40k packets/sec. The ingress & egress rate stayed the same before and after
the change i.e. around 300 msgs/sec.

  So, my question is why would network traffic increase when I make all
calls asynchronous when the ingress & egress rate is same ?

More info about the cluster:
shri at mq1:~$ sudo rabbitmqctl status
Status of node rabbit at mq1 ...
[{pid,11689},
 {running_applications,
     [{rabbitmq_management,"RabbitMQ Management Console","3.0.2"},
      {rabbitmq_management_agent,"RabbitMQ Management Agent","3.0.2"},
      {rabbit,"RabbitMQ","3.0.2"},
      {os_mon,"CPO  CXC 138 46","2.2.4"},
      {rabbitmq_mochiweb,"RabbitMQ Mochiweb Embedding","3.0.2"},
      {webmachine,"webmachine","1.9.1-rmq3.0.2-git52e62bc"},
      {mochiweb,"MochiMedia Web Server","2.3.1-rmq3.0.2-gitd541e9a"},
      {mnesia,"MNESIA  CXC 138 12","4.4.12"},
      {amqp_client,"RabbitMQ AMQP Client","3.0.2"},
      {xmerl,"XML parser","1.2.3"},
      {inets,"INETS  CXC 138 49","5.2"},
      {sasl,"SASL  CXC 138 11","2.1.8"},
      {stdlib,"ERTS  CXC 138 10","1.16.4"},
      {kernel,"ERTS  CXC 138 10","2.13.4"}]},
 {os,{unix,linux}},
 {erlang_version,
     "Erlang R13B03 (erts-5.7.4) [source] [64-bit] [smp:12:12] [rq:12]
[async-threads:30] [hipe] [kernel-poll:true]\n"},
 {memory,
     [{total,469535752},
      {connection_procs,17995944},
      {queue_procs,11675696},
      {plugins,230048},
      {other_proc,126014264},
      {mnesia,18446743803509563784},
      {mgmt_db,12530864},
      {msg_index,1934440},
      {other_ets,-18446743803280937440},
      {binary,5524296},
      {code,18395964},
      {atom,3440321},
      {other_system,43167571}]},
 {vm_memory_high_watermark,0.4},
 {vm_memory_limit,5047128883},
 {disk_free_limit,1000000000},
 {disk_free,482189381632},
 {file_descriptors,
     [{total_limit,65436},
      {total_used,309},
      {sockets_limit,58890},
      {sockets_used,298}]},
 {processes,[{limit,1048576},{used,2992}]},
 {run_queue,7},
 {uptime,23771203}]
...done.

Thank you,
Shri
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20131204/15110bbd/attachment.html>