[rabbitmq-discuss] server crashes with very fast consumers

[Matthew Sackman]

Hi Alex,

On Thu, Mar 10, 2011 at 09:48:08PM -0800, alex chen wrote:
> I am testing rabbit server with SSD.  The broker process would crash when many 
> consumers start to consume backed up messages.  Because SSD if very fast, the 
> consuming rate reached more than 200 MB/sec.  After several minutes, the broker 
> process would use all cpu resources (800% on 8 cores) and crash.

I test all the time on an Intel G2 drive. I've never seen it cause a
crash, even in scenarios such as yours. Furthermore, I find it somewhat
unlikely that if the server process did die, it didn't spit out any sort
of error on its way out. There must be /something/ somewhere that
suggests a bit more as to what went wrong, whether it's in syslog, or
dmesg, or rabbit's various logs.

I did however yesterday find and fix a bug where inexplicably a function
that normally takes about 0.5s ended up in some very strange cases
taking > 80s. After a lot of head scratching and debugging, I never
understood why this variance existed. However, I rewrote the function
and it's now much faster all the time. This particular bug has been
through QA and is now on default and should be in the next release, thus
if you want to try compiling from default and repeating your test, we'd
be interested in knowing whether you see any change.

Matthew