[rabbitmq-discuss] Queue depth and no. of consumers.

Fri Oct 28 15:53:23 BST 2011

On 28/10/11 13:06, michael neidhardt wrote:
> Hi,
> I have a setup with 20 consumers to a single queue. I push approx. 600
> msgs on the queue,
> and all 20 consumers start running at full speed.
> After, say, 20 minutes, I run "rabbitmqctl list_queues name messages
> messages_unacknowledged consumers"
> and see this:
> queuename 540 540 20
>
> This is as expected.

So: you have 540 messages in the queue, *all of which are 
unacknowledged*. That is, they've been consumed but not acked.

> When I get the stats from a queue_declare with passive=True (using
> py_amqplib), I am told that
> there are no messages in queue, and 20 consumers. This is not what I
> expected.

The number returned in queue_declare is the number of messages that 
could be consumed - it's equivalent to messages_ready in rabbitmqctl.

> If I kill all consumers and run the python script again, I am told
> that there are 540 messages and 0 consumers.

When you do this, all the unacknowledged messages get requeued - so now 
they show up as ready again.

> When I start the 20 consumers again, only 2 of them appear to be
> getting msgs and actually doing anything,
> i.e. iotop and top only list two processes as working.

If you're not using basic.qos then the messages will get pushed out to 
whichever consumers will take them - it's not guaranteed to be fair.

> Anybody has any clues?

tl;dr: ack your messages, or use autoack mode.

Cheers, Simon

-- 
Simon MacMullen
RabbitMQ, VMware