[logback-user] Can the SiftingAppender discard 'unknown' log events?

[Colm.McDonnell at rbs.com]

Thanks Ceki, I have implemented that and tested it successfully.

There's a typo in the Janino evaluator's expression, noting it here in case someone reuses this solution: s/mdg/mdc

Thanks for the prompt response, much appreciated.


-----Original Message-----
From: logback-user-bounces at qos.ch [mailto:logback-user-bounces at qos.ch] On Behalf Of Ceki Gülcü
Sent: 23 November 2010 22:51
To: logback users list
Subject: Re: [logback-user] Can the SiftingAppender discard 'unknown' log events?

Hello Colm,

You could add a filter into the nested appender so that when the value for the MDC key isolatedLogName is null, the event is denied. For example,

<appender name="SIFTER"
           class="ch.qos.logback.classic.sift.SiftingAppender">
   <discriminator>
     <key>isolatedLogName</key>
     <defaultValue>unknown</defaultValue>
   </discriminator>
   <sift>
     <appender name="${isolatedLogName}"
               class="ch.qos.logback.core.FileAppender">
       <filter class="ch.qos.logback.core.filter.EvaluatorFilter">
         <!-- GEventEvaluator requires Groovy -->
         <evaluator
            class="ch.qos.logback.classic.boolex.GEventEvaluator">
           <expression>
             (e.mdc == null) ||
                (e.mdc.get("isolatedLogName") == null)
           </expression>
         </evaluator>
         <OnMismatch>NEUTRAL</OnMismatch>
         <OnMatch>DENY</OnMatch>
     </filter>
     <file>${log.path:-target/log}/${isolatedLogName}.log</file>
     <encoder>
       <pattern>%d | %-5p | %logger{5} | %m%n</pattern>
        </encoder>
     </appender>
   </sift>
</appender>


or alternatively using Janino


<appender name="SIFTER"
           class="ch.qos.logback.classic.sift.SiftingAppender">
   <discriminator>
     <key>isolatedLogName</key>
     <defaultValue>unknown</defaultValue>
   </discriminator>
   <sift>
     <appender name="${isolatedLogName}"
               class="ch.qos.logback.core.FileAppender">
       <filter class="ch.qos.logback.core.filter.EvaluatorFilter">
         <!-- JaninoEventEvaluator requires Janino -->
         <evaluator
            class="ch.qos.logback.classic.boolex.JaninoEventEvaluator">
           <expression>
             mdc == null || mdg.get("isolatedLogName") == null
           </expression>
         </evaluator>
         <OnMismatch>NEUTRAL</OnMismatch>
         <OnMatch>DENY</OnMatch>
     </filter>
     <file>${log.path:-target/log}/${isolatedLogName}.log</file>
     <encoder>
       <pattern>%d | %-5p | %logger{5} | %m%n</pattern>
        </encoder>
     </appender>
   </sift>
</appender>

In both cases, the contents of ${log.path:-target/log}/unknown.log will be empty.

In case of very intensive usage, you could also write your own custom evaluator in Java inserting the above logic into your evaluator in order to improve performance.

HTH,

On 23/11/2010 11:49 AM, Colm.McDonnell at rbs.com wrote:
> I am using the SiftingAppender to isolate log events with a specific 
> MDC key. This works really well, giving me high signal:noise ratio log 
> files for specific invocations on particular process etc etc.
> However the SiftingAppender (or rather the MDCBasedDiscriminator) 
> insists on the provision of a 'default value'. If I supply this 
> default value (e.g. "unknown") then all unmatched events will be 
> directed to unknown.log. But the 'other' log events are already logged 
> separately so the entries in 'unknown.log' are unwanted duplicates 
> (for me). Ideally I would use the sifting appender for a particular 
> use case and let everything else be logged the way it is currently 
> logged i.e. I would lke to provide no default value and have the 
> appender understand this to mean 'ignore any events which do not match 
> the given key' but that seems not to be supported so I'm trying to 
> figure out how to either discard the unwanted events or to incorporate 
> my existing logging into the sifting appender.
> I'd be grateful if someone could suggest how I might do either of the
> following:
> * somehow discard the 'unknown' events discriminated by the 
> SiftingAppender i.e. to ignore them completely as they are already 
> logged elsewhere or
> * figure out how to control the 'unknown' log events so that Ii can 
> direct them to an 'everything else' appender which is distinct from 
> the 'isolated events' appender I've included an example of the config 
> below. The STDOUT appender is the main appender, there's a wrapper 
> process running the java app and this wrapper process redirects STDOUT 
> to its own log file (with its own rolling rules etc) therefore the 
> 'everything else' events are all logged to STDOUT. I'd like the SIFTER 
> to only log events for which it gets a match otherwise we end up with 
> the same events in STDOUT and in "unknown.log". Alternatively, if 
> there is a way to configure SIFTER such that it wraps the STDOUT 
> appender and the unmatched events are only directed to that appender 
> then that would work too. I have tried several configurations but I 
> cannot associate the unmatched events with a specific appender.
> <configuration>
> <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
> <encoder>
> <pattern>[%-5p] [%t] %logger{5}: %m%n</pattern> </encoder> </appender> 
> <appender name="SIFTER" 
> class="ch.qos.logback.classic.sift.SiftingAppender">
> <discriminator>
> <Key>isolatedLogName</Key>
> <defaultValue>unknown</defaultValue>
> </discriminator>
> <sift>
> <appender name="${isolatedLogName}"
> class="ch.qos.logback.core.FileAppender">
> <File>${log.path:-target/log}/${isolatedLogName}.log</File>
> <encoder>
> <Pattern>%d | %-5p | %logger{5} | %m%n</Pattern> </encoder> 
> </appender> </sift> </appender>
> <!-- ... loggers excluded for brevity ... --> <root level="INFO"> 
> <appender-ref ref="STDOUT" /> <appender-ref ref="SIFTER"/> </root> 
> </configuration> For example: I tried this:
> <appender name="SIFTER" 
> class="ch.qos.logback.classic.sift.SiftingAppender">
> <discriminator>
> <Key>isolatedLogName</Key>
> <defaultValue>everythingElse</defaultValue>
> </discriminator>
> <sift>
> <appender name="isolatedLogName" 
> class="ch.qos.logback.core.FileAppender">
> <File>${log.path:-target/log}/${isolatedLogName}.log</File>
> <encoder>
> <Pattern>%d | %-5p | %logger{5} | %m%n</Pattern> </encoder> 
> </appender> <appender name="everythingElse" 
> class="ch.qos.logback.core.ConsoleAppender">
> <encoder>
> <pattern>[%-5p] [%t] %logger{5}: %m%n</pattern> </encoder> </appender> 
> </sift> </appender> And logback created two files - everythingElse.log 
> and <the real value associated with the isolatedLogName key>.log - 
> then wrote all output to the "everythingElse" appender. I was trying 
> to tell it to do this: "if the key's value is the default value then 
> ignore it or if that is not possible then direct the output to the 
> appender named after that value otherwise direct to the other 
> appender".
> Any help appreciated, thanks.

_______________________________________________
Logback-user mailing list
Logback-user at qos.ch
http://qos.ch/mailman/listinfo/logback-user

***********************************************************************************
The Royal Bank of Scotland plc. Registered in Scotland No 90312. 
Registered Office: 36 St Andrew Square, Edinburgh EH2 2YB. 
Authorised and regulated by the Financial Services Authority. The
Royal Bank of Scotland N.V. is authorised and regulated by the 
De Nederlandsche Bank and has its seat at Amsterdam, the 
Netherlands, and is registered in the Commercial Register under 
number 33002587. Registered Office: Gustav Mahlerlaan 10, 
Amsterdam, The Netherlands. The Royal Bank of Scotland N.V. and 
The Royal Bank of Scotland plc are authorised to act as agent for each 
other in certain jurisdictions.
 
This e-mail message is confidential and for use by the addressee only.
If the message is received by anyone other than the addressee, please 
return the message to the sender by replying to it and then delete the 
message from your computer. Internet e-mails are not necessarily 
secure. The Royal Bank of Scotland plc and The Royal Bank of Scotland 
N.V. including its affiliates ("RBS group") does not accept responsibility 
for changes made to this message after it was sent. 

Whilst all reasonable care has been taken to avoid the transmission of 
viruses, it is the responsibility of the recipient to ensure that the onward
transmission, opening or use of this message and any attachments will 
not adversely affect its systems or data. No responsibility is accepted 
by the RBS group in this regard and the recipient should carry out such 
virus and other checks as it considers appropriate. 

Visit our website at www.rbs.com

***********************************************************************************