[rabbitmq-discuss] Queue size of the mirrored queue

[Matthew Sackman]

Hi Melvin,

On Fri, Oct 28, 2011 at 04:53:09AM -0700, ||M|| wrote:
> At this moment I'm testing the HA active/active setup of rabbitMQ to
> see if we can replace or current messagging solution with it. I
> created the clustered on 2virutal machines and I must say I like what
> I see so far.

That's great to hear!

> I just have 1 question. Is it correct that I can't see the number of
> messages that are in my durable queue when is processing. When I sent
> (for example) 1.000.000 messages to the queue (using the fanout
> exchange with 1 queue bind) with my publisher it will take some time,
> but when it's done the subscribers are still receiving, because this
> is not so fast as the publishing it will take much more time. But the
> queue size is 3 messages at most.

Ahh, what's going on is that the messages will be being buffered
internally before they get to the queue. Messages within Rabbit pass
between a number of different erlang processes. The process that spends
the most CPU per message will have the biggest backlog of these
messages. In the case of mirrored queues, it's definitely the queue
itself that'll be spending the most CPU per message.

Thus the queue will be receiving each message, processing it (including
all the HA gubbins), and then sending it out to the consumers as fast as
it can, but "as fast as it can" will still be slower than any other
process dealing with those messages within Rabbit. Thus there'll be a
backlog of messages building up immediately before the queue.

If you disable the consumers then you'll see the queue grow in length,
but again, your publisher may well report that it's published all 1e6
msgs before 1e6 msgs make it into the queue proper. This is the same
thing.

> It's not very imported because it just the interface, but I'm a little
> bit concurred about where this message are if the management interface
> (and the command line tool) can see them anywhere.

The numbers in the mgmt and command line are the numbers each actual
queue reports, thus precludes any internal backlog.

Hope that helps,

Matthew