[logback-user] User-filtered logging - can be done?
Y M
ym119162 at hotmail.com
Thu Dec 15 14:32:01 CET 2011
Ralph,
Many thanks for your reply.
> Your configuration is not a great way to do it. Every event that passes the log level will get passed to the appender for filtering by user. This is more expensive than having the filter be a turbo filter.
> The way I would handle this is to use the DynamicThresholdFilter.
Yes, this turbo filter is precisely what I need to do the user level filtering. It was so briefly mentioned in the manual that I barely took notice of it, and never inspected its capabilities. It would further allow user filtered events in loggers other than 'com.sample.project', but not a problem, maybe even better.
>If you want each user to log to a separate file then use the SiftingAppender to route the log events to a log file for that user regardless of the logger.
Not a requirement right now, but I may try it once I get everything else right. But I a sifting would make all events from 'com.sample.project' log to separate user files (i.e. I'd get a file for user 999999 containing its WARN events, despite not targetting it for finer logging through the turbo filter), am I right? That's something I'd like to avoid, as the application currently handles a few hundred users at any time, and a dozen new user specific log files every day would not be desirable.
> The configuration below accepts the event if it is from a specific user regardless of its level. All others flow through unimpeded. Unfortunately, if you don't want debug events for the user logged to LEGACY or STDOUT then you will need to add a ThresholdFilter to those two appenders. However, this isn't as bad since they will only be for that single user.
If I understood correctly, to mimic this appending behavior I want: the ThresholdFilter on STANDARD (example) will need to "double filter" on WARN, removing the DEBUG+ events allowed by the turbo filter, is that it? I may think this is a bit inefficient, but more out-of-the-box than other solutions, like creating a custom logger (if that could do it somehow).
> Notice I specified the log level on all the loggers. IMO this is better in practice because a) the hierarchy doesn't have to be searched all the way to the parent on each event so it will perform better and b) it is less confusing to people looking a the configuration, especially when you specify additivity.
a) I'm afraid this is not correct, the 'Performance' section in Architecture Chapter of the manual says every logger always knows its effective level and it is notified when it changes; if some efficiency would be achieved, it would be during start up or during level changes while running (or so it seems :-)b) you may be right on this... I usually go for a 'less code, more convention' rule, it also makes level changes simpler (also because I use conditional processing for test environment, and changing only the ROOT level is simpler)
Thanks for the great answer, I'll definitely try it and post a follow-up.If anybody knows a different approach, I'd be glad to try it.
Thanks again!
> From: rgoers at apache.org
> Date: Wed, 14 Dec 2011 23:52:29 -0800
> To: logback-user at qos.ch
> Subject: Re: [logback-user] User-filtered logging - can be done?
>
> Your configuration is not a great way to do it. Every event that passes the log level will get passed to the appender for filtering by user. This is more expensive than having the filter be a turbo filter.
>
> The way I would handle this is to use the DynamicThresholdFilter. If you want each user to log to a separate file then use the SiftingAppender to route the log events to a log file for that user regardless of the logger.
>
> The configuration below accepts the event if it is from a specific user regardless of its level. All others flow through unimpeded. Unfortunately, if you don't want debug events for the user logged to LEGACY or STDOUT then you will need to add a ThresholdFilter to those two appenders. However, this isn't as bad since they will only be for that single user.
>
> Notice I specified the log level on all the loggers. IMO this is better in practice because a) the hierarchy doesn't have to be searched all the way to the parent on each event so it will perform better and b) it is less confusing to people looking a the configuration, especially when you specify additivity.
>
> <configuration>
> <turboFilter class="ch.qos.logback.classic.turbo.DynamicThresholdFilter">
> <Key>userId</Key>
> <DefaultThreshold>WARN</DefaultThreshold>
> <onLower>NEUTRAL</onLower>
> <onHigherOrEqual>ACCEPT</onHigherOrEqual>
> <MDCValueLevelPair>
> <value>123456</value>
> <level>DEBUG</level>
> </MDCValueLevelPair>
> </turboFilter>
>
> <appender name="FILTERED" class="ch.qos.logback.core.rolling.RollingFileAppender">
> ...
> </appender>
>
> <appender name="STDOUT"
> class="ch.qos.logback.core.ConsoleAppender">
> <layout class="ch.qos.logback.classic.PatternLayout">
> <Pattern>TEST %d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</Pattern>
> </layout>
> </appender>
>
> <logger name="com.sample.legacy" additivity="false" level="warn">
> <appender-ref ref="LEGACY"/>
> </logger>
>
> <logger name="com.sample.project" level="warn"> <!-- Add additivity="false" if you don't want these also logged to STDOUT -->
> <appender-ref ref="FILTERED"./>
> </logger>
>
> <root level="warn" >
> <appender-ref ref="STDOUT" />
> </root>
> </configuration>
>
> Ralph
>
> On Dec 14, 2011, at 3:50 PM, Chris Pratt wrote:
>
> > You might look at using the User Name as a Marker. Then you could set up a filter that will filter on that User's Marker.
> > (*Chris*)
> >
> > On Wed, Dec 14, 2011 at 3:27 PM, Y M <ym119162 at hotmail.com> wrote:
> > Hello,
> >
> > I'm trying to obtain a certain logging configuration, but so far I'm unsuccessful. I've read through the Logback manual, but I don't know if I can do what I want.
> >
> > It is a web application, and my objective is to enable a finer logging level for specific users, getting written to a specific log file, allowing better debugging directly in production environment. When needed, someone will set a tuned XML config using JMX to enable logging for the duration of the tests/data gathering (JMX is working fine). This is the relevant part of my logback.xml file:
> >
> > <configuration debug="true">
> >
> > ...
> >
> > <appender name="FILTERED" class="ch.qos.logback.core.rolling.RollingFileAppender">
> > <filter class="com.sample.project.UserFilter">
> > <!--
> > //filter code
> > if (level != null && !event.getLevel().isGreaterOrEqual(level)) {
> > return FilterReply.DENY;
> > }
> > if (user.equals(MDC.get("userId"))) {
> > return FilterReply.ACCEPT;
> > } else {
> > return FilterReply.DENY;
> > }
> >
> > -->
> > <level>DEBUG</level>
> > <user>123456</user>
> > </filter>
> >
> > ...
> > </appender>
> >
> > <logger name="com.sample.project">
> > <appender-ref ref="FILTERED" />
> > </logger>
> > <logger name="com.sample.legacy" additivity="false">
> > <appender-ref ref="LEGACY" />
> > </logger>
> > <root level="WARN">
> > <appender-ref ref="STANDARD" />
> > </root>
> >
> > </configuration>
> >
> > With this config, I intend to have:
> > - WARN+ level at root;
> > - events from legacy project tree only on 'LEGACY' (working fine);
> > - 'FILTERED' logging only DEBUG+ events from user '123456' under 'com.sample.project', while its level ramains at WARN (not OK).
> >
> > Sample:
> > getLogger("com.sample.abc").warn("..."); //logged to STANDARD
> > getLogger("com.sample.abc").debug("..."); //not logged
> > getLogger("com.sample.legacy").warn("..."); //logged to LEGACY
> > getLogger("com.sample.project").warn("..."); //logged to STANDARD; also logged to FILTERED if user is 123456
> > getLogger("com.sample.project").debug("..."); //logged to FILTERED if user is 123456
> >
> > As I tested, a 'logger.debug()' from the desired user does not get logged, the filter is not even called, and I suspect that the effective WARN level denies the event before reaching my filter (I hoped for the other way around). If I set DEBUG on 'com.sample.project', 'FILTERED' will get the correct logging, but 'STANDARD' will be flooded with DEBUG+ logging from 'com.sample.project'. And disabling additivity on it removes all logging belonging to this hierarchy from 'STANDARD'.
> >
> > So, I see Logback is working properly, my desired logging seems to be against the standard architecture. But is there a way to archive the described effect?
> > I was thinking of a TurboFilter, allowing lower level events from selected users (not tested yet), but I'm not sure the effective level would kick in first, also it wouldn't do the desired logging separation between 'STANDARD' and 'FILTERED' anyway. As another approach, maybe this would require a custom Logger class, handling specific appenders in a special way. Unfortunately, I have no idea if I can plug in custom Loggers, and also not sure how to code it (Logger class source is quite complex in a quick look).
> >
> > Sorry if it wasn't clear, I'm not sure how to present the situation. I can provide further details, if needed.
> > Any help is welcome. If someone could show me which way to follow, maybe classes or methods to extend, I'd be grateful.
> >
> > Many thanks!
> >
> > _______________________________________________
> > Logback-user mailing list
> > Logback-user at qos.ch
> > http://mailman.qos.ch/mailman/listinfo/logback-user
> >
> > _______________________________________________
> > Logback-user mailing list
> > Logback-user at qos.ch
> > http://mailman.qos.ch/mailman/listinfo/logback-user
>
> _______________________________________________
> Logback-user mailing list
> Logback-user at qos.ch
> http://mailman.qos.ch/mailman/listinfo/logback-user
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.qos.ch/pipermail/logback-user/attachments/20111215/eacab895/attachment.html>
More information about the Logback-user
mailing list