[rabbitmq-discuss] Problem concurrency isue with rabbitmq (branch bug21673)

[Matthew Sackman]

On Mon, Feb 15, 2010 at 11:40:20AM -0800, scott w wrote:
> We are also noticing that rabbitmq branch bug 21673 seems to hang after
> placing a severe amount of load on it. I don't know erlang but is there some
> way I can do a thread dump to send to you guys to help determine what the
> problem might be?

Let me guess. You send an awful lot of messages into RabbitMQ, and then
you start reading from the queue, and you find it takes some time before
the first message pops out?

If so, that's entirely expected. What you're seeing is the fact that
writes, being async, allow a large backlog of pending disk activity to
grow, bounded by available RAM. When a read occurs, except in some
special circumstances, you have no choice but to let those writes hit
disk before you can read them - you can't risk a read overtaking a
write.

However, that said, I'm now thinking that there are some situations in
which allowing a read to have a higher priority than a write would be
safe... I shall have a play. If you could confirm that this is the
scenario you're seeing that'd be great. In general, you should be able
to identify this situation by a high(ish) load average, but no CPU
activity and your disks being burnt to a cinder. A 15kRPM SAS Fiber
Channel 8-disk RAID 0 array /may/ help here :P

Matthew