[rabbitmq-discuss] Low performance when there are many messages in ready or unacked state

[Anahi]

Thanks for the info! I read the article, but we can't use that
implementation since it seems to be kind of experimental and we need to
mantain the app as stable as possible,

Regarding the prefetch count, I'm not really sure about how it works in the
.net version. I've tried with this code _channel.BasicQos(0, 50, false) and
the only difference I see is that unacked messages ratio keep constant. But
there's no performance improvement.

We're using: RabbitMQ 3.2.2, Erlang R16B03

*Setting prefetch count before consuming data:*
Deliver 24/s
Ack 20~24/s
Published 1600/s


*Without prefetch count:*
Deliver 10~30/s
Ack 20~30/s
Published 1600/s

When we decreased the publish ratio (by adding sleep after sending each
message) we could increase the deliver ratio up to 90/s, but when the
number of messages within ready state start growing, the deliver ratio
decreases to 20/s again.


Thanks!!!

Anni..


2014/1/22 Alvaro Videla <videlalvaro at gmail.com>

> Hi,
>
> Have you considered the approaches explained here:
>
> http://www.rabbitmq.com/blog/2012/05/11/some-queuing-theory-throughput-latency-and-bandwidth/
>
> Also I think people could help you a bit more if you shared some
> numbers about number of messages, consumers, RabbitMQ and Erlang
> versions, and so on.
>
> Regards,
>
> Alvaro
>
> On Wed, Jan 22, 2014 at 9:51 PM, Anahi Hojnadel
> <anahi.hojnadel at gmail.com> wrote:
> > Hi guys!
> >
> > We have an application that has a publisher which sends messages to a
> > consumer which process those messages and then stores them in the DB.
> > Everything is working fine up to now, but the number of messages we're
> > receiving is getting bigger and performance is quite bad. We started to
> > analize where the bottleneck was and it seems to be in rabbit queues.
> >
> > I've been reading many posts in Rabbit's blog about optimization but I
> > couldn't find how to solve the problem.
> >
> > The main problem is that we're publishing messages faster than we can
> > consume them. When the number of messages in ready state (or in unacked
> > state) increases, the deliver ratio decrease considerably.
> >
> > I've tried adding more consumers, sending messages more slowly (with time
> > sleep)... without luck.
> >
> > I've seen that sending an ack for each message make things slower but we
> > actually need to ensure they're delivered. Our current ack ratio is too
> low,
> > so every time the number of unacked messages increases, the deliver ratio
> > decreases.
> >
> > Do you have any ideas of what could I try to improve our process
> > performance? Or should I just make sure not to send more messages than my
> > current ack ratio? Is stacking acks and sending them in batches a safe
> > strategy to try out?
> >
> > Adding more queues is not an option since it will add too much
> complexity to
> > our app.
> >
> > Thanks!!
> >
> > _______________________________________________
> > rabbitmq-discuss mailing list
> > rabbitmq-discuss at lists.rabbitmq.com
> > https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> >
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "rabbitmq-discuss" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/rabbitmq-discuss/BW62Tn9sQkE/unsubscribe
> .
> To unsubscribe from this group and all its topics, send an email to
> rabbitmq-discuss+unsubscribe at googlegroups.com.
> To post to this group, send email to rabbitmq-discuss at googlegroups.com.
> Visit this group at http://groups.google.com/group/rabbitmq-discuss.
> For more options, visit https://groups.google.com/groups/opt_out.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20140123/d43e75c8/attachment.html>