[rabbitmq-discuss] Low performance when there are many messages in ready or unacked state

[Alvaro Videla]

Hi,

Have you considered the approaches explained here:
http://www.rabbitmq.com/blog/2012/05/11/some-queuing-theory-throughput-latency-and-bandwidth/

Also I think people could help you a bit more if you shared some
numbers about number of messages, consumers, RabbitMQ and Erlang
versions, and so on.

Regards,

Alvaro

On Wed, Jan 22, 2014 at 9:51 PM, Anahi Hojnadel
<anahi.hojnadel at gmail.com> wrote:
> Hi guys!
>
> We have an application that has a publisher which sends messages to a
> consumer which process those messages and then stores them in the DB.
> Everything is working fine up to now, but the number of messages we're
> receiving is getting bigger and performance is quite bad. We started to
> analize where the bottleneck was and it seems to be in rabbit queues.
>
> I've been reading many posts in Rabbit's blog about optimization but I
> couldn't find how to solve the problem.
>
> The main problem is that we're publishing messages faster than we can
> consume them. When the number of messages in ready state (or in unacked
> state) increases, the deliver ratio decrease considerably.
>
> I've tried adding more consumers, sending messages more slowly (with time
> sleep)... without luck.
>
> I've seen that sending an ack for each message make things slower but we
> actually need to ensure they're delivered. Our current ack ratio is too low,
> so every time the number of unacked messages increases, the deliver ratio
> decreases.
>
> Do you have any ideas of what could I try to improve our process
> performance? Or should I just make sure not to send more messages than my
> current ack ratio? Is stacking acks and sending them in batches a safe
> strategy to try out?
>
> Adding more queues is not an option since it will add too much complexity to
> our app.
>
> Thanks!!
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>