[rabbitmq-discuss] RabbitMQ and Splunk

[Michael Vierling]

>First of all, it's nice to know someone's using rabbitmqadmin

I think it is great!  Cli tools are especially important to us.  It would be doable but painful to scrape the html output.

>I'd like to not call it kvp since it's really timestamp - key/value, which seems quite Splunk-specific.

Timestamp, followed by key/value pairs seems rather generic to me :-)  But feel free to call it anything you wish, including "splunk"

>I hadn't thought of using rabbitmqadmin to generate log files. I assume the idea
>is to provide historic data for trends etc. Does Splunk always work like this?

Yes, linux tools such as ps, iostat, memstat, etc are used by Splunk in a similar manner.  I plan on polling via rabbitmqadmin every 30 seconds and then using the data for dashboard graphs.  It can be extremely helpful when troubleshooting to visually look for changes in behavior.

>How do quotes (and non-ASCII characters) need to be escaped? The link does not explain.

Splunk attempts to apply UTF-8 encoding by default.  See - http://www.splunk.com/base/Documentation/latest/Admin/Configurecharactersetencoding

> I assume you're not proposing to use any of the "standard fields"?

No, I'm not proposing this.  Also, we can add mapping tables to convert RabbitMQ keys to generic Splunk keys.

Michael

-----Original Message-----
From: rabbitmq-discuss-bounces at lists.rabbitmq.com [mailto:rabbitmq-discuss-bounces at lists.rabbitmq.com] On Behalf Of Simon MacMullen
Sent: Monday, November 01, 2010 3:56 AM
To: rabbitmq-discuss at lists.rabbitmq.com
Subject: Re: [rabbitmq-discuss] RabbitMQ and Splunk

On 30/10/10 01:13, Michael Vierling wrote:
> We're developing a Splunk plugin for RabbitMQ.  Splunk is an excellent
> log search engine and we highly recommend it. Anyway, while Splunk can
> ingest almost any log files, it prefers a key-value pair format. You can
> see this page for more details:
>
> http://www.splunk.com/wiki/Apps:Common_Information_Model
>
> So in that spirit, I'd like to propose the following patch to the
> rabbitmqadmin script, which ships with your management 2.1.1 plugin.
> This patch adds a key-value pair Splunk compatible option (kvp) to the
> script. It would be very helpful if this could be incorporated into the
> official Management plugin.

<snip>

Hi Michael.

First of all, it's nice to know someone's using rabbitmqadmin - I think 
you're the first to ask about it...

In order to accept your patch, I'd need to get you to sign our 
contributor agreement (yes, even for something this small). Or I could 
reimplement it; that might be easier.

But before that, can we clear up a few issues:

* I'd like to not call it kvp since it's really timestamp - key/value, 
which seems quite Splunk-specific.
* I hadn't thought of using rabbitmqadmin to generate log files. I 
assume the idea is to provide historic data for trends etc. Does Splunk 
always work like this?
* How do quotes (and non-ASCII characters) need to be escaped? The link 
does not explain.
* I assume you're not proposing to use any of the "standard fields"?

Cheers, Simon

-- 
Simon MacMullen
Staff Engineer, RabbitMQ
SpringSource, a division of VMware

_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss at lists.rabbitmq.com
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss