[rabbitmq-discuss] Extremely unfair consumption under high publish load

[Eugene Kirpichov]

Hi.

I have a scenario where I'm submitting several hundred thousand
messages to a bunch of queues and then waiting for them to be
consumed.

While I'm submitting, the consumption throughput is quite low and
unfair - some of the queues barely get consumed at all.

After I stop submitting, consumption throughput grows by an order of
magnitude (this is more or less understandable) and becomes fair (this
is what bugs me!).

What's wrong? How can I avoid this effect?

Also, is there a way to make rabbitmq to pay more attention to the
consumers rather than producers? I'd rather be publishing and
receiving 2k msg/s than publishing 3k and receiving 1k (queueing the
rest).

http://i.imgur.com/q3TxC.png shows the rates of submitting (bottom)
and consuming (top) per queue. [ shameless plug: drawn from the
application log with a one-liner of http://github.com/jkff/timeplot ]

-- 
Eugene Kirpichov
Principal Engineer, Mirantis Inc. http://www.mirantis.com/
Editor, http://fprog.ru/