[rabbitmq-discuss] message stats from a given node

[Brian Sullivan]

Yes, that's the information I am looking for.

I'll take a look at the proxy stuff, I hadn't considered that before.
Thanks, Matthias.

Cheers,
Brian


On Mon, Jan 18, 2010 at 11:57 PM, Matthias Radestock <matthias at lshift.net>wrote:

> Brian,
>
>
> Brian Sullivan wrote:
>
>> I was wondering if there are any good ideas out there for tracking message
>> rates on a given cluster (in our use case, it's mostly topic-routed
>> messages).  We have been using RabbitMQ for a while now, and have been
>> limited to tracking stats via a consumer that effectively subscribes to '#'
>> and simply breaks down these stats by topic.
>>
>
> So the information you are interested in is "average number of messages per
> second published, per routing key and time period", correct?
>
>
>  This works ok until the message volumes get pretty heavy - the node that
>> the client connects to has to be routed all messages on the cluster, and
>> this breaks down the scalability model for RabbitMQ - eventually that node
>> will be a bottleneck.
>>
>
> One thing you could do at the AMQP level is introduce stat-collecting
> proxies between the publishers and the brokers. Take a look at the RabbitMQ
> tracer in src/com/rabbitmq/tools/Tracer.java in the rabbitmq-java-client
> repo for an example of a proxy.
>
> Another route is via the "pluggable exchange types" we have under
> development - see the bug22169 branch in the rabbitmq-server hg repo. That
> would allow you to define a custom exchange type that works just like an
> ordinary topic (or other) exchange but collects stats on publishes on a
> per-node basis. In the simplest case this could just dump a timestamp and
> routing key to a file, leaving the actual statistics calculations to some
> other program. Or, at the other extreme, the custom exchange could compute
> per-node stats itself and, furthermore, participate in the collation of
> stats from multiple nodes, using the distributed computing features Erlang
> puts at your disposal.
>
>
> Regards,
>
> Matthias.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20100123/7e5d8d13/attachment.htm