[rabbitmq-discuss] RabbitMQ Consumes Very Slow

[Matthew Sackman]

Hi Bradford,

On Tue, Jun 01, 2010 at 03:27:51PM -0700, Bradford Stephens wrote:
> I'm having pretty nasty speed issues on Fedora. I'm running Fedora 8,
> 64-bit on EC2. I have a 8core/8GB RAM server. I'm able to insert
> thousands of messages a second, but I can only consume about 3/sec.
> Each is ~1k.

Well for starters, don't use basic.get, set up a consumer using
basic.consume. Secondly, think about whether you need to acknowledge
messages - if they're relatively unimportant messages (i.e. if a client
crashes whilst processing messages, do you want those messages to be
lost, or to automatically be requeued) then you might consider turning
on noAck (looks like you already do in your example code), which helps
speed.

Finally, you're bascially writing a poor version of the QueueingConsumer
(the major difference being that QueueingConsumer permits a jump across
threads, meaning that you can process received messages in a different
thread, and not impact the rate at which you receive messages from the
broker). I'd recommend you try using the QueueingConsumer instead as it
would then mean that you don't block the client whilst you're processing
the messages (in this case just adding messages into a queue -- err
which is what the QueueingConsumer does for you automatically).

As always with performance issues, once you're using the right API and
right algorithms (and asking here is definitely the right thing to do),
careful use of a profiler is a good idea to pick up where any further
bottlenecks are.

Matthew