[logback-user] Can the SiftingAppender discard 'unknown' log events?

[Ceki Gulcu]

Hello Colm,

Thanks for your feedback. Adding an OnConsoleStatusListener can help in 
identifying such errors. Just add

<statusListener
    class="ch.qos.logback.core.status.OnConsoleStatusListener" />

at the top of your config file. See also 
http://logback.qos.ch/manual/configuration.html#statusListener

Cheers,

http://logback.qos.ch/manual/configuration.html#statusListener

On 24.11.2010 10:41, Colm.McDonnell at rbs.com wrote:
> Thanks Ceki, I have implemented that and tested it successfully.
>
> There's a typo in the Janino evaluator's expression, noting it here in case someone reuses this solution: s/mdg/mdc
>
> Thanks for the prompt response, much appreciated.
>
>
> -----Original Message-----
> From: logback-user-bounces at qos.ch [mailto:logback-user-bounces at qos.ch] On Behalf Of Ceki Gülcü
> Sent: 23 November 2010 22:51
> To: logback users list
> Subject: Re: [logback-user] Can the SiftingAppender discard 'unknown' log events?
>
> Hello Colm,
>
> You could add a filter into the nested appender so that when the value for the MDC key isolatedLogName is null, the event is denied. For example,
>
> <appender name="SIFTER"
>             class="ch.qos.logback.classic.sift.SiftingAppender">
>     <discriminator>
>       <key>isolatedLogName</key>
>       <defaultValue>unknown</defaultValue>
>     </discriminator>
>     <sift>
>       <appender name="${isolatedLogName}"
>                 class="ch.qos.logback.core.FileAppender">
>         <filter class="ch.qos.logback.core.filter.EvaluatorFilter">
>           <!-- GEventEvaluator requires Groovy -->
>           <evaluator
>              class="ch.qos.logback.classic.boolex.GEventEvaluator">
>             <expression>
>               (e.mdc == null) ||
>                  (e.mdc.get("isolatedLogName") == null)
>             </expression>
>           </evaluator>
>           <OnMismatch>NEUTRAL</OnMismatch>
>           <OnMatch>DENY</OnMatch>
>       </filter>
>       <file>${log.path:-target/log}/${isolatedLogName}.log</file>
>       <encoder>
>         <pattern>%d | %-5p | %logger{5} | %m%n</pattern>
>          </encoder>
>       </appender>
>     </sift>
> </appender>
>
>
> or alternatively using Janino
>
>
> <appender name="SIFTER"
>             class="ch.qos.logback.classic.sift.SiftingAppender">
>     <discriminator>
>       <key>isolatedLogName</key>
>       <defaultValue>unknown</defaultValue>
>     </discriminator>
>     <sift>
>       <appender name="${isolatedLogName}"
>                 class="ch.qos.logback.core.FileAppender">
>         <filter class="ch.qos.logback.core.filter.EvaluatorFilter">
>           <!-- JaninoEventEvaluator requires Janino -->
>           <evaluator
>              class="ch.qos.logback.classic.boolex.JaninoEventEvaluator">
>             <expression>
>               mdc == null || mdg.get("isolatedLogName") == null
>             </expression>
>           </evaluator>
>           <OnMismatch>NEUTRAL</OnMismatch>
>           <OnMatch>DENY</OnMatch>
>       </filter>
>       <file>${log.path:-target/log}/${isolatedLogName}.log</file>
>       <encoder>
>         <pattern>%d | %-5p | %logger{5} | %m%n</pattern>
>          </encoder>
>       </appender>
>     </sift>
> </appender>
>
> In both cases, the contents of ${log.path:-target/log}/unknown.log will be empty.
>
> In case of very intensive usage, you could also write your own custom evaluator in Java inserting the above logic into your evaluator in order to improve performance.
>
> HTH,
>
> On 23/11/2010 11:49 AM, Colm.McDonnell at rbs.com wrote:
>> I am using the SiftingAppender to isolate log events with a specific
>> MDC key. This works really well, giving me high signal:noise ratio log
>> files for specific invocations on particular process etc etc.
>> However the SiftingAppender (or rather the MDCBasedDiscriminator)
>> insists on the provision of a 'default value'. If I supply this
>> default value (e.g. "unknown") then all unmatched events will be
>> directed to unknown.log. But the 'other' log events are already logged
>> separately so the entries in 'unknown.log' are unwanted duplicates
>> (for me). Ideally I would use the sifting appender for a particular
>> use case and let everything else be logged the way it is currently
>> logged i.e. I would lke to provide no default value and have the
>> appender understand this to mean 'ignore any events which do not match
>> the given key' but that seems not to be supported so I'm trying to
>> figure out how to either discard the unwanted events or to incorporate
>> my existing logging into the sifting appender.
>> I'd be grateful if someone could suggest how I might do either of the
>> following:
>> * somehow discard the 'unknown' events discriminated by the
>> SiftingAppender i.e. to ignore them completely as they are already
>> logged elsewhere or
>> * figure out how to control the 'unknown' log events so that Ii can
>> direct them to an 'everything else' appender which is distinct from
>> the 'isolated events' appender I've included an example of the config
>> below. The STDOUT appender is the main appender, there's a wrapper
>> process running the java app and this wrapper process redirects STDOUT
>> to its own log file (with its own rolling rules etc) therefore the
>> 'everything else' events are all logged to STDOUT. I'd like the SIFTER
>> to only log events for which it gets a match otherwise we end up with
>> the same events in STDOUT and in "unknown.log". Alternatively, if
>> there is a way to configure SIFTER such that it wraps the STDOUT
>> appender and the unmatched events are only directed to that appender
>> then that would work too. I have tried several configurations but I
>> cannot associate the unmatched events with a specific appender.
>> <configuration>
>> <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
>> <encoder>
>> <pattern>[%-5p] [%t] %logger{5}: %m%n</pattern>  </encoder>  </appender>
>> <appender name="SIFTER"
>> class="ch.qos.logback.classic.sift.SiftingAppender">
>> <discriminator>
>> <Key>isolatedLogName</Key>
>> <defaultValue>unknown</defaultValue>
>> </discriminator>
>> <sift>
>> <appender name="${isolatedLogName}"
>> class="ch.qos.logback.core.FileAppender">
>> <File>${log.path:-target/log}/${isolatedLogName}.log</File>
>> <encoder>
>> <Pattern>%d | %-5p | %logger{5} | %m%n</Pattern>  </encoder>
>> </appender>  </sift>  </appender>
>> <!-- ... loggers excluded for brevity ... -->  <root level="INFO">
>> <appender-ref ref="STDOUT" />  <appender-ref ref="SIFTER"/>  </root>
>> </configuration>  For example: I tried this:
>> <appender name="SIFTER"
>> class="ch.qos.logback.classic.sift.SiftingAppender">
>> <discriminator>
>> <Key>isolatedLogName</Key>
>> <defaultValue>everythingElse</defaultValue>
>> </discriminator>
>> <sift>
>> <appender name="isolatedLogName"
>> class="ch.qos.logback.core.FileAppender">
>> <File>${log.path:-target/log}/${isolatedLogName}.log</File>
>> <encoder>
>> <Pattern>%d | %-5p | %logger{5} | %m%n</Pattern>  </encoder>
>> </appender>  <appender name="everythingElse"
>> class="ch.qos.logback.core.ConsoleAppender">
>> <encoder>
>> <pattern>[%-5p] [%t] %logger{5}: %m%n</pattern>  </encoder>  </appender>
>> </sift>  </appender>  And logback created two files - everythingElse.log
>> and<the real value associated with the isolatedLogName key>.log -
>> then wrote all output to the "everythingElse" appender. I was trying
>> to tell it to do this: "if the key's value is the default value then
>> ignore it or if that is not possible then direct the output to the
>> appender named after that value otherwise direct to the other
>> appender".
>> Any help appreciated, thanks.
>
> _______________________________________________
> Logback-user mailing list
> Logback-user at qos.ch
> http://qos.ch/mailman/listinfo/logback-user
>