[rabbitmq-discuss] Erlang crashes trying to allocate 583848200 bytes of memory
D.Rotondi at Computer.Org
Tue Jan 25 10:30:42 GMT 2011
On Jan 24, 9:36 pm, Jerry Kuch <jer... at vmware.com> wrote:
Thanks Jerry for the reply.
My answers/comments inline.
> Hi, Dom...
> > we have a similar problem that can be summarised as follows. Our goal
> > is to compare RabbitMQ (the last stable version) to ActiveMQ. To this
> > end we set up two minimal and similar testing environments having both
> > 1 publisher, 1 subscriber and 1 message broker (RabbitMQ/ ActiveMQ)
> > server. In both testing environments the message broker runs under a
> > Windows Hyper-V virtual machine (hosted on a Windows Server 23008 64
> > bits) that has as OS a 32 bit Windows Server 2008 (if I'm correct we
> > allocated 2 GB, perhpas 4 GB, to the VM and 1 CPU). The
> > publishers/subscribers are run under Windows XP/Vista and are on the
> > same LAN as the broker VM.
> This ought to be an entirely reasonable set up. Apologies that you're
> having trouble with it. There's definitely something awry here and
> current evidence points toward it not being yiour fault.
> > On the subscriber side it connects to the broker waiting for messages.
> > When a new chuck started message is received, it saves the received
> > timestamp and then counts how many messages it has received till the
> > indication of the chunk end message. At this point it reads the system
> > clock and calculates how much time has elapsed since the chunck start
> > on the publisher side (as reported in the received timestamp) and the
> > arrival on the chunk end message on the subscriber side. The
> > subscriber then logs all these data (start timestamp, end timestamp,
> > elapsed time, number of messages), and is ready to process a new
> > message chunk. Anyway further details and the Java code is available
> > in the indicated RabbitMQ newsgroup's message. In our testing
> > environment, therefore, there are no durable queues, no permanent
> > messages, a single subscriber, ...
> Our of curiosity when and how is your consumer ACK-ing the messages it
> gets? Do you have auto-ACK set? Or are you explicitly ACK-ing them,
> either individually or as a range?
We tried both using auto-ACK and explicit ACK on each received
message. The result was exactly the same.
BTW our VM has a 2 GB RAM. The RbbitMQ exchange type is a direct one,
and there only 1 consumer queue bound to that exchange.
You can find all our test code here (http://lists.rabbitmq.com/
> > What we experienced are crashes of the RabbitMQ server due to the
> > memory allocation issues. Indeed, the Windows Task Manager shows the
> > RabbitMQ process increasing memory, until it crashes. On the RabbitMQ
> > forum someone suggested to my colleague Cristoforo to reduce the
> > vm_memory_high_watermark value from its default 0.40 value. We tried
> > with 0.25, 0.20 and 0.10. The crash problem doesn't disappear unless
> > the subscriber runs on a fast machine (e.g. putting the subscriber on
> > the same VM as the message broker seems to have less crashing issues).
> > With ActiveMQ we don't experience similar issues at all. We are not
> > using Erlnag under a 64 bit OS (our VM, as stated, is a 32 bit Windows
> > Server 2008), and the vm_memory_high_watermark has been lowered enough
> > (we think).
> So your observations are that when you have a fast consumer, your
> broker fares better?
Yes. To our understanding the RabbitMQ broker (and its supporting
Erlang environment) keeps to be delivered messages in memory; if the
consumer is not fast enough the Erlang memory footprint grows till the
> > Our conclusion is that the Windows Erlang implementation has some
> > serious issues on memory management under Windows, or that we have not
> > configured Erlang/RabbitMQ in a proper way.
> At the moment, it looks like the former is very likely true. Some on
> the Rabbit team have had some interaction with Erlang's maintainers on
> the issue, but we don't have a good answer for you yet.
> > Sorry for the long comment, but I think it was necessary to describe a
> > testing scenario that, in my opinion, can shed some lights on the
> > flagged issue.
> No apologies necessary... your feedback is helpful, and has helped us
> confirm that something is awry on Windows. The Rabbit user base seems
> rather tilted toward Unix so when there are problems specific to
> running on Windows, we don't always find out as quickly as we might
> Best regards,
> rabbitmq-discuss mailing list
> rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
More information about the rabbitmq-discuss