[rabbitmq-discuss] lager vs sasl error log

[Tim Watson]

Hi Gavin,

On 21 Mar 2013, at 21:02, "Gavin M. Roy" <gmr at meetme.com> wrote:
> Fwiw in some advocacy discussions, the logging output was a reason some people have discounted or not moved forward with Rabbit.
> From conversations I have had recently with people, logging and config file syntax are the bigger approachachability issues with Rabbit.
> 

That's a pity - the logging is something we could change. The config file syntax, not so much. I suppose we could bootstrap the prelaunch phase to parse and XML file (or whatever) and spew out the native format...

> My goal is to be able to do log aggregation for our largish count of RabbitMQ nodes, which afaik is not possible with Rabbit out of the box. This makes it nearly impossible to stay on top of logs for anything but troubleshooting individual nodes when problems occur.

Yeah with the current implementation you'd need to install your own log handler to make that work. In terms of changing the output format though, allowe to play devil's advocate for a moment...

The info/progress/error reports have a lot of data in them, process stats, stack traces, etc. are you wanting to see all that plonked on one single long line? Because I can see how that'd make parsing/aggregating easier, but not reading them. Does the lager output fit exactly with what you want (I've not looks at it or a couple of years)?

> 
> On Thursday, March 21, 2013, Ben Hood <0x6e6562 at gmail.com> wrote:
> Hey Tim,
> 
> Thanks for the heads up.
> 
> Cheers,
> 
> Ben
> On Thursday, 21 March 2013 at 12:36, Tim Watson wrote:
> 
>> Hi Ben,
>> 
>> On 21 Mar 2013, at 12:13, Ben Hood wrote:
>>> I think a good start would be to implement Gavin's suggestion about having a more UNIXy output, since you could do this without having to change the transport.
>> 
>> That's an understand-able requirement, however...
>> 
>>> If you are aggregating lots of Rabbit logs, this results in lots of double entries such as
>>> 
>>> Mar 21 12:05:30 a logger: : =INFO REPORT==== 21-Mar-2013::12:05:29 ===
>>> Mar 21 12:05:30 a logger: : started TCP Listener on [::]:5672
>> 
>> Sure - that multi-line output is what Erlang's built in error_logger generates.
>> 
>>> If I understand Gavin correctly, you could improve the SNR by putting this on one line.
>> 
>> To do that, we'd have to change from the built-in error_logger to something else, such as lager. That idea's been up for discussion before and as I said, I'll raise it again.
>> 
>> Cheers,
>> Tim
>> 
>>> Cheers,
>>> 
>>> Ben
>>> 
>>> On Thursday, 21 March 2013 at 12:03, Tim Watson wrote:
>>> 
>>>> Hi all,
>>>> 
>>>> We've discussed moving to a different (non-standard) logging provider before, and I'll raise the issue again. Afaik we've not run into the issues with error_logger very often (i.e., the reasons why lager was invented, viz massive memory consumption due to serialisation of large terms in logging output). The 'filtering possibilities' with lager don't necessarily require the parse_transforms to be used iirc - you can submit a logging request direct to lager and specify the log level at the call site. Again, I'll raise the discussion with the rest of the team and let's see what comes of that.
>>>> 
>>>> Cheers,
>>>> Tim
>>>> 
>>>> On 19 Mar 2013, at 08:48, Gotthard, Petr wrote:
>>>> 
>>>>> Correct me if I’m wrong, but I don’t think that moving to lager requires any changes inside rabbitmq. Since rabbitmq (as far as I know) uses standard Erlang’s logging and since standard logs are redirected to lager, all rabbitmq related logs are processed by lager. Wihout many filtering possibilities though.
>>>>> Petr
>>>>> From: rabbitmq-discuss-bounces at lists.rabbitmq.com [mailto:rabbitmq-discuss-bounces at lists.rabbitmq.com] On Behalf Of Gavin M. Roy
>>>>> Sent: 18. března 2013 17:36
>>>>> To: Discussions about RabbitMQ
>>>>> Subject: [rabbitmq-discuss] lager vs sasl error log
>>>>> Any plans to move rabbitmq.log to lager (and allow for lager-syslog) while keeping rabbit-sasl.log for crash dumps?
>>>>> Would be nice to be able to get the plugable loggers and improvements to logging performance lager purports to provide, as well as a more modern, *NIXy logging output.
>>>>> Cheers,
>>>>> 
>>>>> Gavin
>>>>> _______________________________________________
>>>>> rabbitmq-discuss mailing list
>>>>> rabbitmq-discuss at lists.rabbitmq.com
>>>>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>>> 
>>>> _______________________________________________
>>>> rabbitmq-discuss mailing list
>>>> rabbitmq-discuss at lists.rabbitmq.com
>>>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>> 
>>> _______________________________________________
>>> rabbitmq-discuss mailing list
>>> rabbitmq-discuss at lists.rabbitmq.com
> 
> 
> -- 
> Gavin M. Roy
> Chief Technology Officer
> 
> 100 Union Square Drive
> New Hope, PA 18938
> p. +1.215.862.1162 x263
> f. +1.215.862.0465
>       
> 
> The public market leader in social discovery. (NYSE MKT: MEET) 
> 
> 
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130322/bf3364f0/attachment.htm>