Actually is causing me headaches because the memory of the node keeps rising to beyond the watermark level and that freezes all my rabbitmq cluster. For instance right node I have a node with 4.3GB used with a watermark of 4.7GB and all ndoe queues are updated (empties). I&#39;ve pated the node status and queue details below.<br>
<br>Any idea how to solve this?<br><br>curl .../api/nodes/rabbit@i-000000ed-wsm<br>{<br>�� &quot;name&quot;:&quot;rabbit@i-000000ed-wsm&quot;,<br>�� &quot;type&quot;:&quot;ram&quot;,<br>�� &quot;running&quot;:true,<br>�� &quot;os_pid&quot;:&quot;6451&quot;,<br>
�� &quot;mem_ets&quot;:54743192,<br>�� &quot;mem_binary&quot;:206280808,<br>�� &quot;mem_proc&quot;:4284714760,<br>�� &quot;mem_proc_used&quot;:4284698594,<br>�� &quot;mem_atom&quot;:2107665,<br>�� &quot;mem_atom_used&quot;:2076476,<br>
�� &quot;mem_code&quot;:15225838,<br>�� &quot;fd_used&quot;:42,<br>�� &quot;fd_total&quot;:131072,<br>�� &quot;sockets_used&quot;:14,<br>�� &quot;sockets_total&quot;:117872,<br>�� &quot;mem_used&quot;:4645571200,<br>�� &quot;mem_limit&quot;:5017685196,<br>
�� &quot;mem_alarm&quot;:false,<br>�� &quot;proc_used&quot;:618,<br>�� &quot;proc_total&quot;:32768,<br>�� &quot;statistics_level&quot;:&quot;fine&quot;,<br>�� &quot;erlang_version&quot;:&quot;R15B&quot;,<br>�� &quot;uptime&quot;:2203242331,<br>
�� &quot;run_queue&quot;:5,<br>�� &quot;processors&quot;:4,<br>�� &quot;exchange_types&quot;:[<br>����� {<br>�������� &quot;name&quot;:&quot;topic&quot;,<br>�������� &quot;description&quot;:&quot;AMQP topic exchange, as per the AMQP specification&quot;,<br>
�������� &quot;enabled&quot;:true<br>����� },<br>����� {<br>�������� &quot;name&quot;:&quot;fanout&quot;,<br>�������� &quot;description&quot;:&quot;AMQP fanout exchange, as per the AMQP specification&quot;,<br>�������� &quot;enabled&quot;:true<br>
����� },<br>����� {<br>�������� &quot;name&quot;:&quot;direct&quot;,<br>�������� &quot;description&quot;:&quot;AMQP direct exchange, as per the AMQP specification&quot;,<br>�������� &quot;enabled&quot;:true<br>����� },<br>
����� {<br>�������� &quot;name&quot;:&quot;headers&quot;,<br>�������� &quot;description&quot;:&quot;AMQP headers exchange, as per the AMQP specification&quot;,<br>�������� &quot;enabled&quot;:true<br>����� }<br>�� ],<br>�� &quot;auth_mechanisms&quot;:[<br>
����� {<br>�������� &quot;name&quot;:&quot;PLAIN&quot;,<br>�������� &quot;description&quot;:&quot;SASL PLAIN authentication mechanism&quot;,<br>�������� &quot;enabled&quot;:true<br>����� },<br>����� {<br>�������� &quot;name&quot;:&quot;AMQPLAIN&quot;,<br>
�������� &quot;description&quot;:&quot;QPid AMQPLAIN mechanism&quot;,<br>�������� &quot;enabled&quot;:true<br>����� },<br>����� {<br>�������� &quot;name&quot;:&quot;RABBIT-CR-DEMO&quot;,<br>�������� &quot;description&quot;:&quot;RabbitMQ Demo challenge-response authentication mechanism&quot;,<br>
�������� &quot;enabled&quot;:false<br>����� }<br>�� ],<br>�� &quot;applications&quot;:[<br>����� {<br>�������� &quot;name&quot;:&quot;amqp_client&quot;,<br>�������� &quot;description&quot;:&quot;RabbitMQ AMQP Client&quot;,<br>
�������� &quot;version&quot;:&quot;2.7.1&quot;<br>����� },<br>����� {<br>�������� &quot;name&quot;:&quot;inets&quot;,<br>�������� &quot;description&quot;:&quot;INETS� CXC 138 49&quot;,<br>�������� &quot;version&quot;:&quot;5.8&quot;<br>
����� },<br>����� {<br>�������� &quot;name&quot;:&quot;kernel&quot;,<br>�������� &quot;description&quot;:&quot;ERTS� CXC 138 10&quot;,<br>�������� &quot;version&quot;:&quot;2.15&quot;<br>����� },<br>����� {<br>�������� &quot;name&quot;:&quot;mnesia&quot;,<br>
�������� &quot;description&quot;:&quot;MNESIA� CXC 138 12&quot;,<br>�������� &quot;version&quot;:&quot;4.6&quot;<br>����� },<br>����� {<br>�������� &quot;name&quot;:&quot;mochiweb&quot;,<br>�������� &quot;description&quot;:&quot;MochiMedia Web Server&quot;,<br>
�������� &quot;version&quot;:&quot;1.3-rmq2.7.1-git&quot;<br>����� },<br>����� {<br>�������� &quot;name&quot;:&quot;os_mon&quot;,<br>�������� &quot;description&quot;:&quot;CPO� CXC 138 46&quot;,<br>�������� &quot;version&quot;:&quot;2.2.8&quot;<br>
����� },<br>����� {<br>�������� &quot;name&quot;:&quot;rabbit&quot;,<br>�������� &quot;description&quot;:&quot;RabbitMQ&quot;,<br>�������� &quot;version&quot;:&quot;2.7.1&quot;<br>����� },<br>����� {<br>�������� &quot;name&quot;:&quot;rabbitmq_management&quot;,<br>
�������� &quot;description&quot;:&quot;RabbitMQ Management Console&quot;,<br>�������� &quot;version&quot;:&quot;2.7.1&quot;<br>����� },<br>����� {<br>�������� &quot;name&quot;:&quot;rabbitmq_management_agent&quot;,<br>�������� &quot;description&quot;:&quot;RabbitMQ Management Agent&quot;,<br>
�������� &quot;version&quot;:&quot;2.7.1&quot;<br>����� },<br>����� {<br>�������� &quot;name&quot;:&quot;rabbitmq_management_visualiser&quot;,<br>�������� &quot;description&quot;:&quot;RabbitMQ Visualiser&quot;,<br>�������� &quot;version&quot;:&quot;2.7.1&quot;<br>
����� },<br>����� {<br>�������� &quot;name&quot;:&quot;rabbitmq_mochiweb&quot;,<br>�������� &quot;description&quot;:&quot;RabbitMQ Mochiweb Embedding&quot;,<br>�������� &quot;version&quot;:&quot;2.7.1&quot;<br>����� },<br>
����� {<br>�������� &quot;name&quot;:&quot;sasl&quot;,<br>�������� &quot;description&quot;:&quot;SASL� CXC 138 11&quot;,<br>�������� &quot;version&quot;:&quot;2.2&quot;<br>����� },<br>����� {<br>�������� &quot;name&quot;:&quot;stdlib&quot;,<br>
�������� &quot;description&quot;:&quot;ERTS� CXC 138 10&quot;,<br>�������� &quot;version&quot;:&quot;1.18&quot;<br>����� },<br>����� {<br>�������� &quot;name&quot;:&quot;webmachine&quot;,<br>�������� &quot;description&quot;:&quot;webmachine&quot;,<br>
�������� &quot;version&quot;:&quot;1.7.0-rmq2.7.1-hg&quot;<br>����� }<br>�� ]<br>}<br><br><br><br>rabbitmqctl status<br>Status of node &#39;rabbit@i-00000140-csm&#39; ...<br>[{pid,739},<br>�{running_applications,<br>���� [{rabbitmq_management_visualiser,&quot;RabbitMQ Visualiser&quot;,&quot;2.7.1&quot;},<br>
����� {rabbitmq_management,&quot;RabbitMQ Management Console&quot;,&quot;2.7.1&quot;},<br>����� {rabbitmq_mochiweb,&quot;RabbitMQ Mochiweb Embedding&quot;,&quot;2.7.1&quot;},<br>����� {webmachine,&quot;webmachine&quot;,&quot;1.7.0-rmq2.7.1-hg&quot;},<br>
����� {amqp_client,&quot;RabbitMQ AMQP Client&quot;,&quot;2.7.1&quot;},<br>����� {rabbitmq_management_agent,&quot;RabbitMQ Management Agent&quot;,&quot;2.7.1&quot;},<br>����� {rabbit,&quot;RabbitMQ&quot;,&quot;2.7.1&quot;},<br>
����� {os_mon,&quot;CPO� CXC 138 46&quot;,&quot;2.2.8&quot;},<br>����� {sasl,&quot;SASL� CXC 138 11&quot;,&quot;2.2&quot;},<br>����� {mochiweb,&quot;MochiMedia Web Server&quot;,&quot;1.3-rmq2.7.1-git&quot;},<br>����� {inets,&quot;INETS� CXC 138 49&quot;,&quot;5.8&quot;},<br>
����� {mnesia,&quot;MNESIA� CXC 138 12&quot;,&quot;4.6&quot;},<br>����� {stdlib,&quot;ERTS� CXC 138 10&quot;,&quot;1.18&quot;},<br>����� {kernel,&quot;ERTS� CXC 138 10&quot;,&quot;2.15&quot;}]},<br>�{os,{unix,linux}},<br>
�{erlang_version,<br>���� &quot;Erlang R15B (erts-5.9) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false]\n&quot;},<br>�{memory,<br>���� [{total,4404012432},<br>����� {processes,4148260149},<br>����� {processes_used,4148259728},<br>
����� {system,255752283},<br>����� {atom,561761},<br>����� {atom_used,550271},<br>����� {binary,81414464},<br>����� {code,15193946},<br>����� {ets,86119376}]},<br>�{vm_memory_high_watermark,0.799999999904651},<br>�{vm_memory_limit,6712184012}]<br>
...done.<br><br><br><br>Here I&#39;m attaches the queue details. The first two queues consumes 3.5GB<br><a href="https://gist.github.com/2649372">https://gist.github.com/2649372</a><br><br><br><br><div class="gmail_quote">
On Tue, May 8, 2012 at 8:22 AM, Matthias Radestock <span dir="ltr">&lt;<a href="mailto:matthias@rabbitmq.com" target="_blank">matthias@rabbitmq.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Pablo,<div class="im"><br>
<br>
On 07/05/12 19:28, Pablo Molnar wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I&#39;m trying to figure out why the memory consumption of some queues<br>
are �extremely high although there is almost none messages ready to consume.<br>
For instance this queue has only 3 messages but consumes 2.4GB of<br>
RAM.<br>
</blockquote>
<br></div>
That is not necessarily a problem, in particular ...<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5">
{<br>
 � &quot;exclusive_consumer_tag&quot;:&quot;&quot;,<br>
 � &quot;messages_ready&quot;:0,<br>
 � &quot;messages_unacknowledged&quot;:3,<br>
 � &quot;messages&quot;:3,<br>
 � &quot;consumers&quot;:25,<br>
 � &quot;memory&quot;:2586777216,<br>
 � &quot;slave_nodes&quot;:[<br>
<br>
 � ],<br>
 � &quot;backing_queue_status&quot;:{<br>
 � � �&quot;q1&quot;:0,<br>
 � � �&quot;q2&quot;:0,<br>
 � � �&quot;delta&quot;:[<br>
 � � � � &quot;delta&quot;,<br>
 � � � � &quot;undefined&quot;,<br>
 � � � � 0,<br>
 � � � � &quot;undefined&quot;<br>
 � � �],<br>
 � � �&quot;q3&quot;:0,<br>
 � � �&quot;q4&quot;:0,<br>
 � � �&quot;len&quot;:0,<br>
 � � �&quot;pending_acks&quot;:3,<br>
 � � �&quot;target_ram_count&quot;:0,<br>
 � � �&quot;ram_msg_count&quot;:0,<br>
 � � �&quot;ram_ack_count&quot;:0,<br>
 � � �&quot;next_seq_id&quot;:72528926,<br>
 � � �&quot;persistent_count&quot;:3,<br>
 � � �&quot;avg_ingress_rate&quot;:42.<u></u>456013780009044,<br>
 � � �&quot;avg_egress_rate&quot;:42.<u></u>456013780009044,<br>
 � � �&quot;avg_ack_ingress_rate&quot;:42.<u></u>456013780009044,<br>
 � � �&quot;avg_ack_egress_rate&quot;:42.<u></u>456013780009044<br></div></div>
 � }, �...<br>
</blockquote>
<br>
...this is a slow-moving, but not idle queue. I suspect what happened is that at some point the queue did have lots of messages in it, hence requiring lots of memory. Those messages subsequently got drained, but the since the queue is slow moving it can take a while for Erlang&#39;s gc to reclaim memory.<br>

<br>
Unless this is causing any other problems I wouldn&#39;t worry about it.<div><div class="h5"><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
rabbitmqctl status<br>
Status of node &#39;rabbit@i-00000140-csm&#39; ...<br>
[{pid,20106},<br>
�{running_applications,<br>
 � � [{rabbitmq_management_<u></u>visualiser,&quot;RabbitMQ Visualiser&quot;,&quot;2.7.1&quot;},<br>
 � � �{rabbitmq_management,&quot;RabbitMQ Management Console&quot;,&quot;2.7.1&quot;},<br>
 � � �{rabbitmq_mochiweb,&quot;RabbitMQ Mochiweb Embedding&quot;,&quot;2.7.1&quot;},<br>
 � � �{webmachine,&quot;webmachine&quot;,&quot;1.7.<u></u>0-rmq2.7.1-hg&quot;},<br>
 � � �{amqp_client,&quot;RabbitMQ AMQP Client&quot;,&quot;2.7.1&quot;},<br>
 � � �{rabbitmq_management_agent,&quot;<u></u>RabbitMQ Management Agent&quot;,&quot;2.7.1&quot;},<br>
 � � �{rabbit,&quot;RabbitMQ&quot;,&quot;2.7.1&quot;},<br>
 � � �{os_mon,&quot;CPO �CXC 138 46&quot;,&quot;2.2.8&quot;},<br>
 � � �{sasl,&quot;SASL �CXC 138 11&quot;,&quot;2.2&quot;},<br>
 � � �{mochiweb,&quot;MochiMedia Web Server&quot;,&quot;1.3-rmq2.7.1-git&quot;},<br>
 � � �{inets,&quot;INETS �CXC 138 49&quot;,&quot;5.8&quot;},<br>
 � � �{mnesia,&quot;MNESIA �CXC 138 12&quot;,&quot;4.6&quot;},<br>
 � � �{stdlib,&quot;ERTS �CXC 138 10&quot;,&quot;1.18&quot;},<br>
 � � �{kernel,&quot;ERTS �CXC 138 10&quot;,&quot;2.15&quot;}]},<br>
�{os,{unix,linux}},<br>
�{erlang_version,<br>
 � � &quot;Erlang R15B (erts-5.9) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false]\n&quot;},<br>
�{memory,<br>
 � � [{total,4736488576},<br>
 � � �{processes,3832122172},<br>
 � � �{processes_used,3832066481},<br>
 � � �{system,904366404},<br>
 � � �{atom,561761},<br>
 � � �{atom_used,551430},<br>
 � � �{binary,98523592},<br>
 � � �{code,15193946},<br>
 � � �{ets,717445688}]},<br>
�{vm_memory_high_watermark,0.<u></u>6999999999761628},<br>
�{vm_memory_limit,<a href="tel:5873161011" value="+15873161011" target="_blank">5873161011</a>}]<br>
...done.<br>
</blockquote>
<br></div></div>
You may want to upgrade to the latest Erlang (R15B01) and rabbit (2.8.2), though I doubt that will make a difference.<br>
<br>
<br>
Regards,<br>
<br>
Matthias.<br>
</blockquote></div><br>