[rabbitmq-discuss] Low performance when there are many messages in ready or unacked state

[Anahi Hojnadel]

Hi guys!

We have an application that has a publisher which sends messages to a 
consumer which process those messages and then stores them in the DB. 
Everything is working fine up to now, but the number of messages we're 
receiving is getting bigger and performance is quite bad. We started to 
analize where the bottleneck was and it seems to be in rabbit queues.

I've been reading many posts in Rabbit's blog about optimization but I 
couldn't find how to solve the problem.

The main problem is that we're publishing messages faster than we can 
consume them. When the number of messages in ready state (or in unacked 
state) increases, the deliver ratio decrease considerably.

I've tried adding more consumers, sending messages more slowly (with time 
sleep)... without luck.

I've seen that sending an ack for each message make things slower but we 
actually need to ensure they're delivered. Our current ack ratio is too 
low, so every time the number of unacked messages increases, the deliver 
ratio decreases.

Do you have any ideas of what could I try to improve our process 
performance? Or should I just make sure not to send more messages than my 
current ack ratio? Is stacking acks and sending them in batches a safe 
strategy to try out?

Adding more queues is not an option since it will add too much complexity 
to our app.

Thanks!!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20140122/ca23af3f/attachment.html>