[rabbitmq-discuss] Windows RabbitMQ Crashes and Blue Screens under Load

Wed Jan 11 18:32:23 GMT 2012

We've let loose one of our testing ninjas on RabbitMQ for load testing, and
we're consistently running into issues when the high memory watermark is
hit.

Windows Server 2003 32-bit , Erlang R15B 32-bit, Rabbit 2.7.1

2,000 Consumers each with their own queue bound to a direct exchange

1 Producer, publishing a 2 MB message to the exchange, once every second,
for a total of 50 seconds

Everything behaves as expected, until the memory footprint hits the high
watermark, at which point:

On a physical machine: ERL process crashes and dump file is created

On a Virtual Machine: Blue Screen of Death is shown and server reboots

VM environment = VMware, Inc.R vCenter Lab Manager 4.0 (4.0.3.1318)

One other note is that we see the same problem with ERL R14B04 and Rabbit
2.7.0.

I have looked through the log file and also turned on the console debug
output, and nothing seems to be jumping out as an error.  If needed, I can
upload the minidump from the Blue Screen and the ERL crash dump file, just
point me where to do it.

Let me know if there is anything else I can do to try and help get this
fixed.

In the rabbit log, there are no errors, and only a few warnings 20 seconds
before the crash:

=INFO REPORT==== 11-Jan-2012::10:55:53 ===

closing TCP connection <0.4405.0> from 10.6.64.104:57830

=WARNING REPORT==== 11-Jan-2012::10:55:53 ===

exception on TCP connection <0.20552.0> from 10.6.64.104:59521

connection_closed_abruptly

In the console output log file for the physical machine, this is the only
message I see:

starting direct_client
...done

starting notify cluster nodes
...done

broker running

Eshell V5.9  (abort with ^G)

(rabbit at QEDLP082)1> 

Crash dump was written to: C:/Documents and
Settings/Administrator.QEDLP/Application Data/RabbitMQ/erl_crash.dump

eheap_alloc: Cannot allocate 6731340 bytes of memory (of type "heap").

in message_loop

win32sysinfo:Erlang has closed.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120111/e1621720/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 7172 bytes
Desc: not available
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120111/e1621720/attachment.bin>