[rabbitmq-discuss] RabbitMQ 2.3.1 high CPU usage & disks operations

[Matthew Sackman]

Hi Raphael,

Thanks for the detailed report. This is an interesting one, and a few
ideas spring to mind.

You seem to be using the x-message-ttl feature now, with message expiry
around 1 day. Do these events happen to correspond with the bulk expiry
of a lot of messages? Certainly 60,000 odd msgs in the broker shouldn't
cause any problems at all, but just want to check that one. Also, when
you say "many queues get deleted", how many are we talking about? The
only thing we can think of is if you're deleting more queues than you
have file descriptors available for the process then you may run into
problems: Rabbit won't crash, but it might thrash fds between different
queues which can overall result in vastly more disk activity than
necessary.

The normal Linux default fd limit (ulimit -n) is 1024, though on OS X,
it's down at 256 by default IIRC. If you're deleting more than about
900 queues at any one go, then I'd suggest that you try increasing that
substantially: having about double the number of fds available than
queues should eliminate any thrashing - if indeed this is the problem.
If this is the problem then by strace, you should be able to see an
awful lot of open and close calls to various files, whereas if this
problem is solved by higher fd ulimit then you should see files stay
open for much longer.

Please let us know how you get on.

Matthew