[rabbitmq-discuss] Erlang crashes trying to allocate 583848200 bytes of memory

[Dom54]

On Jan 24, 9:36 pm, Jerry Kuch <jer... at vmware.com> wrote:
Thanks Jerry for the reply.
My answers/comments inline.
Ciao
    Dom

> Hi, Dom...
>
> > we have a similar problem that can be summarised as follows.  Our goal
> > is to compare RabbitMQ (the last stable version) to ActiveMQ.  To this
> > end we set up two minimal and similar testing environments having both
> > 1 publisher, 1 subscriber and 1 message broker (RabbitMQ/ ActiveMQ)
> > server.  In both testing environments the message broker runs under a
> > Windows Hyper-V virtual machine (hosted on a Windows Server 23008 64
> > bits) that has as OS a 32 bit Windows Server 2008 (if I'm correct we
> > allocated 2 GB, perhpas 4 GB, to the VM and 1 CPU).  The
> > publishers/subscribers are run under Windows XP/Vista and are on the
> > same LAN as the broker VM.
>
> This ought to be an entirely reasonable set up.  Apologies that you're
> having trouble with it.  There's definitely something awry here and
> current evidence points toward it not being yiour fault.
>
> > On the subscriber side it connects to the broker waiting for messages.
> > When a new chuck started message is received, it saves the received
> > timestamp and then counts how many messages it has received till the
> > indication of the chunk end message. At this point it reads the system
> > clock and calculates how much time has elapsed since the chunck start
> > on the publisher side (as reported in the received timestamp) and the
> > arrival on the chunk end message on the subscriber side.  The
> > subscriber then logs all these data (start timestamp, end timestamp,
> > elapsed time, number of messages), and is ready to process a new
> > message chunk.  Anyway further details and the Java code is available
> > in the indicated RabbitMQ newsgroup's message.  In our testing
> > environment, therefore, there are no durable queues, no permanent
> > messages, a single subscriber, ...
>
> Our of curiosity when and how is your consumer ACK-ing the messages it
> gets?  Do you have auto-ACK set?  Or are you explicitly ACK-ing them,
> either individually or as a range?

We tried both using auto-ACK and explicit ACK on each received
message. The result was exactly the same.
BTW our VM has a 2 GB RAM. The RbbitMQ exchange type is a direct one,
and there only 1 consumer queue bound to that exchange.
You can find all our test code here (http://lists.rabbitmq.com/
pipermail/rabbitmq-discuss/2011-January/
010818.html).

>
> > What we experienced are crashes of the RabbitMQ server due to the
> > memory allocation issues. Indeed, the Windows Task Manager shows the
> > RabbitMQ process increasing memory, until it crashes.  On the RabbitMQ
> > forum someone suggested to my colleague Cristoforo to reduce the
> > vm_memory_high_watermark value from its default 0.40 value.  We tried
> > with 0.25, 0.20 and 0.10. The crash problem doesn't disappear unless
> > the subscriber runs on a fast machine (e.g. putting the subscriber on
> > the same VM as the message broker seems to have less crashing issues).
> > With ActiveMQ we don't experience similar issues at all.  We are not
> > using Erlnag under a 64 bit OS (our VM, as stated, is a 32 bit Windows
> > Server 2008), and the vm_memory_high_watermark has been lowered enough
> > (we think).
>
> So your observations are that when you have a fast consumer, your
> broker fares better?

Yes. To our understanding the RabbitMQ broker (and its supporting
Erlang environment) keeps to be delivered messages in memory; if the
consumer is not fast enough the Erlang memory footprint grows till the
broker crash.

>
> > Our conclusion is that the Windows Erlang implementation has some
> > serious issues on memory management under Windows, or that we have not
> > configured Erlang/RabbitMQ in a proper way.
>
> At the moment, it looks like the former is very likely true.  Some on
> the Rabbit team have had some interaction with Erlang's maintainers on
> the issue, but we don't have a good answer for you yet.
>
> > Sorry for the long comment, but I think it was necessary to describe a
> > testing scenario that, in my opinion, can shed some lights on the
> > flagged issue.
>
> No apologies necessary...  your feedback is helpful, and has helped us
> confirm that something is awry on Windows.  The Rabbit user base seems
> rather tilted toward Unix so when there are problems specific to
> running on Windows, we don't always find out as quickly as we might
> like...
>
> Best regards,
> Jerry
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss