[rabbitmq-discuss] Extremely unfair consumption under high publish load

Fri Aug 12 05:14:52 BST 2011

Please disregard this complaint.

It turned out that I was bombing rabbitmq with reconnects and
QueueDeclares due to a bug in my code, so no wonder it behaved
indecently.

Morale:
Do not bomb rabbitmq with reconnects and QueueDeclares! Especially
when you're on a cluster.

On Thu, Aug 11, 2011 at 10:56 AM, Eugene Kirpichov <ekirpichov at gmail.com> wrote:
> Hi.
>
> I have a scenario where I'm submitting several hundred thousand
> messages to a bunch of queues and then waiting for them to be
> consumed.
>
> While I'm submitting, the consumption throughput is quite low and
> unfair - some of the queues barely get consumed at all.
>
> After I stop submitting, consumption throughput grows by an order of
> magnitude (this is more or less understandable) and becomes fair (this
> is what bugs me!).
>
> What's wrong? How can I avoid this effect?
>
> Also, is there a way to make rabbitmq to pay more attention to the
> consumers rather than producers? I'd rather be publishing and
> receiving 2k msg/s than publishing 3k and receiving 1k (queueing the
> rest).
>
> http://i.imgur.com/q3TxC.png shows the rates of submitting (bottom)
> and consuming (top) per queue. [ shameless plug: drawn from the
> application log with a one-liner of http://github.com/jkff/timeplot ]
>
> --
> Eugene Kirpichov
> Principal Engineer, Mirantis Inc. http://www.mirantis.com/
> Editor, http://fprog.ru/
>

-- 
Eugene Kirpichov
Principal Engineer, Mirantis Inc. http://www.mirantis.com/
Editor, http://fprog.ru/