[rabbitmq-discuss] exessive memory usage and blocked connection, even when no messages left

Wed May 18 21:47:45 BST 2011

Eventually (a couple of hours without any connections) the node
returned to normal memory usage, so I launched a new batch of tests. I
saw a sustained load of 20K published/delivered/acked for a few
minutes, with the memory usage of each node increasing rapidly. After
perhaps five minutes the cluster rotted completely. No messages
published, nothing delivered, all connections are blocked.

20K messages over four nodes, that shouldn't be a problem? From my
point of view it looks like RabbitMQ is leaking memory badly, but that
couldn't be, could it?

At this point there is no message on any queue, according to the web
UI, but each node is still using over 3Gb RAM.

Theo

On May 18, 6:15 pm, Theo <icon... at gmail.com> wrote:
> 2.4.1, should have been more specific. the OS is ubuntu 10.04, btw.
>
> there's a lot of warnings in the log about memory usage, but it's more
> or less the same thing over and over again:
>
> =INFO REPORT==== 18-May-2011::14:37:20 ===
> vm_memory_high_watermark set. Memory used:3327376104 allowed:
> 3006570496
>
> =INFO REPORT==== 18-May-2011::14:37:20 ===
>     alarm_handler: {set,
> {{vm_memory_high_watermark,rabbit at rfmmqstaging01},[]}}
>
> then I found this, not sure if it's important (pasted it into a gist):
>
> https://gist.github.com/978897
>
> this is the version of erlang:
>
> Erlang R13B03 (erts-5.7.4) [source] [64-bit] [smp:8:8] [rq:8] [async-
> threads:0] [hipe] [kernel-poll:false]
>
> I've been watching the management UI since I posted my first message
> (almost an hour ago) and the memory usage hasn't gone down. can a node
> get bad? the other nodes look ok, but the one that's in the red in the
> screenshot is still in the red. no connections to the server for an
> hour.
>
> T#
>
> On May 18, 6:01 pm, Matthew Sackman <matt... at rabbitmq.com> wrote:> Hi Theo,
>
> > On Wed, May 18, 2011 at 08:22:13AM -0700, Theo wrote:
> > > More specifics on the setup: the cluster consists of 4 EC2 instances
> > > with 8 CPUs and 7 Gb RAM each (I forget the exact instance name)
> > > running RabbitMQ 2.4.
>
> > Is that 2.4.0 or 2.4.1? If not 2.4.1, please upgrade. Which version of
> > Erlang are you running? Is there anything of note other than connections
> > and memory alarms in any of the logs?
>
> > Matthew
> > _______________________________________________
> > rabbitmq-discuss mailing list
> > rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss