[rabbitmq-discuss] message stats from a given node
Matthias Radestock
matthias at lshift.net
Tue Jan 19 07:57:01 GMT 2010
Brian,
Brian Sullivan wrote:
> I was wondering if there are any good ideas out there for tracking
> message rates on a given cluster (in our use case, it's mostly
> topic-routed messages). We have been using RabbitMQ for a while now,
> and have been limited to tracking stats via a consumer that effectively
> subscribes to '#' and simply breaks down these stats by topic.
So the information you are interested in is "average number of messages
per second published, per routing key and time period", correct?
> This works ok until the message volumes get pretty heavy - the node that
> the client connects to has to be routed all messages on the cluster, and
> this breaks down the scalability model for RabbitMQ - eventually that
> node will be a bottleneck.
One thing you could do at the AMQP level is introduce stat-collecting
proxies between the publishers and the brokers. Take a look at the
RabbitMQ tracer in src/com/rabbitmq/tools/Tracer.java in the
rabbitmq-java-client repo for an example of a proxy.
Another route is via the "pluggable exchange types" we have under
development - see the bug22169 branch in the rabbitmq-server hg repo.
That would allow you to define a custom exchange type that works just
like an ordinary topic (or other) exchange but collects stats on
publishes on a per-node basis. In the simplest case this could just dump
a timestamp and routing key to a file, leaving the actual statistics
calculations to some other program. Or, at the other extreme, the custom
exchange could compute per-node stats itself and, furthermore,
participate in the collation of stats from multiple nodes, using the
distributed computing features Erlang puts at your disposal.
Regards,
Matthias.
More information about the rabbitmq-discuss
mailing list