[logback-user] ENC: RES: Implementing MultiFileAppender
Leonardo de Moura Rocha Lima
llima at v2telecom.com.br
Tue May 29 21:14:08 CEST 2007
Hello, all.
First let me introduce myself. I'm Leonardo Lima, and I've been using log4j
for about 2 years.
In the past, I needed to generate one log file per 'context', context being
anything that I put in my MDC. I went to the log4j user list, discussed a
little with other users and I used an implementation that I found at log4j
bug tracking system. I've been using it since October last year and it's all
ok.
Sebastien contacted me and the other guys that were talking about (how to)
implementing what became the MultiFileAppender so that we could contribute
it to logback; but at the time I couldn't switch to nor learn logback.
Now I want to use logback and reproduce the functionality of
MultiFileAppender (that I went and improved myself over). So, the result's
in the emails that follows.
So, please, any input/feedback is much welcome. I improved the code below
using EHCache as caching solution to handle the timeout/max open files
problems that would occour, but maybe something simpler and built-in is
better. And, as an improvement, I thought about applying the whole
PatternLayout parsing to filenames, so we could use thread names and much
more information on file names. That and the option to auto-create dirs,
would be awesome (at least for me). Think about self-organizing log!
Thanks a lot for your time.
Leonardo de Moura Rocha Lima
-----Mensagem original-----
De: Ceki Gulcu [mailto:ceki at qos.ch]
Enviada em: terça-feira, 29 de maio de 2007 15:25
Para: Leonardo de Moura Rocha Lima
Cc: 'Sebastien Pennec'
Assunto: Re: RES: Implementing MultiFileAppender
Hello Leonardo,
Since this conversation might interest other users, could you please post
your
message on the logback-user mailing list?
Many thanks,
Leonardo de Moura Rocha Lima wrote:
> Hello, Sebastien.
>
> I'm sorry that it took such a long time to reply your e-mail, but only now
I
> got 'green light' to go into logback. We now have a new project that we'd
> like to use logback and keep the MultiFileAppender behavior (developed
> in-house) that we had in log4j.
> For what I gathered, the concept of using MDC properties in filenames
isn't
> yet implement in logback. So, I am on my way to implement it.
>
> I tried for a couple of hours to understand and use your recommendation
> about using Joran replay functionality to achieve what we need, but
failed.
> I did run and (I think I) understood the FruitConfigurationTest. I read
> Chapter 3 of the logback Manual that has an example of using Joran to
> re/load configuration. But I failed to understand how I replay the
> configuration once the first run is running.
>
> So I went and did something that's not that beautiful, but works. I
> implemented an Appender has a parser that substitutes the %X{} and %d{}
tags
> (not recursively, but more than one tag per filename. Example:
> client.%X{ip}.%X{port}.%d{yyyy}.%d{MM}.log) and create a new filename from
a
> base filename, plus a map of FileAppenders to delegate the logging events
> to. It's a naïve approach, but it's suiting my needs (use MDC to generate
> context, roll logger using Time). Only thing left is handling appenders
> 'timeout' and 'max open files', so I don't get too many file handles nor
> keep file handles open for no reason. I thought about using some cache
> solution to handle that, but it would add more dependency to the
project...
>
> I'd like to request your guidance on the matter. Should I go on with what
I
> wrote (with or without the cache), or please teach me how the 'replay'
> functionality would be used in this matter. And, if I wanted to contribute
> the code, what should I provide. I've never contributed anything before...
>
> I embed the code of my class here because I don't know how you handle
> attachments...
>
> Thanks in advance, and keep up the good work on logback.
>
> Leonardo de Moura Rocha Lima
> 55 11 3094.3939
>
>
> import java.text.SimpleDateFormat;
> import java.util.Date;
> import java.util.HashMap;
> import java.util.Map;
>
> import ch.qos.logback.classic.spi.LoggingEvent;
> import ch.qos.logback.core.AppenderBase;
> import ch.qos.logback.core.FileAppender;
> import ch.qos.logback.core.Layout;
>
> public class LAppender extends AppenderBase<LoggingEvent> {
>
> /**
> * Base Filename. Will be parsed for %X{} and %d{} tags.
> */
> private String file;
>
> /**
> * FileAppender option; will propagate.
> */
> private Layout<LoggingEvent> layout;
>
> /**
> * FileAppender option; will propagate.
> */
> private boolean immediateFlush = true;
>
> /**
> * FileAppender option; will propagate.
> */
> private boolean bufferedIO;
>
> /**
> * FileAppender option; will propagate.
> */
> private String encoding;
>
> /**
> * FileAppender option; will propagate.
> */
> private boolean append;
>
> /**
> * FileAppender option; will propagate.
> */
> private int bufferSize;
>
> /**
> * Map of current active FileAppenders. Needs improvement: max open
> files
> * and timeout, to keep resources
> */
> private Map<String, FileAppender<LoggingEvent>> appenders = new
> HashMap<String, FileAppender<LoggingEvent>>();
>
> /**
> * Return the FileAppender for the given filename. Create a new one
> and add
> * to cache, if needed.
> */
> private FileAppender<LoggingEvent> getFA(String filename) {
> if (appenders.get(filename) != null) {
> return appenders.get(filename);
> }
>
> FileAppender<LoggingEvent> resp = new
> FileAppender<LoggingEvent>();
> resp.setAppend(append);
> resp.setBufferedIO(bufferedIO);
> resp.setBufferSize(bufferSize);
> resp.setContext(this.getContext());
> resp.setEncoding(encoding);
> resp.setFile(filename);
> resp.setImmediateFlush(immediateFlush);
> resp.setLayout(layout);
> resp.setName(this.getName());
> resp.start();
> appenders.put(filename, resp);
> return resp;
> }
>
> @Override
> /**
> * Delegate the LoggingEvent to the right FileAppender
> */
> protected void append(LoggingEvent eventObject) {
> getFA(getFile(eventObject)).doAppend(eventObject);
> }
>
> /**
> * Return the filename for the given LoggingEvent, based on the base
> * filename.
> */
> private String getFile(LoggingEvent eventObject) {
> return substituteDate(substituteContext(file, eventObject
> .getMDCPropertyMap()), new
> Date(eventObject.getTimeStamp()));
> }
>
> /**
> * Parse and replace %X{} tags.
> */
> private String substituteContext(String base, Map<String, String>
> mdc) {
> if (base == null)
> return null;
> if (mdc == null)
> return base;
>
> int idx = base.indexOf("%X{");
> if (idx < 0)
> return base;
> int lidx = base.indexOf('}', idx);
> String str = base.substring(idx + 3, lidx);
> String val = mdc.get(str);
> if (val == null)
> val = "null";
> return substituteContext(base.replaceAll("\\%X\\{" + str +
> '}', val),
> mdc);
> }
>
> /**
> * Parse and replace %d{} tags.
> */
> private String substituteDate(String base, Date dt) {
> if (base == null)
> return null;
> if (dt == null)
> dt = new Date();
>
> int idx = base.indexOf("%d{");
> if (idx < 0)
> return base;
> int lidx = base.indexOf('}', idx);
> String str = base.substring(idx + 3, lidx);
> String val = new SimpleDateFormat(str).format(dt);
> if (val == null)
> val = "null";
> return substituteDate(base.replaceAll("\\%d\\{" + str + '}',
> val), dt);
> }
>
> @Override
> /**
> * Propagate the stop action to everyone
> */
> public void stop() {
> for (FileAppender<LoggingEvent> fa :
> this.appenders.values()) {
> fa.stop();
> }
> }
>
> public void setFile(String file) {
> this.file = file;
> }
>
> public void setLayout(Layout<LoggingEvent> layout) {
> this.layout = layout;
> }
>
> public void setAppend(boolean append) {
> this.append = append;
> }
>
> public void setBufferedIO(boolean bufferedIO) {
> this.bufferedIO = bufferedIO;
> }
>
> public void setEncoding(String encoding) {
> this.encoding = encoding;
> }
>
> public void setImmediateFlush(boolean immediateFlush) {
> this.immediateFlush = immediateFlush;
> }
>
> public void setBufferSize(int bufferSize) {
> this.bufferSize = bufferSize;
> }
>
> }
>
>
> -----Mensagem original-----
> De: Sebastien Pennec [mailto:sebastien at qos.ch]
> Enviada em: segunda-feira, 23 de outubro de 2006 14:53
> Para: HBender at Ergonomics.ch; Patrick.Wyss at mobilesolutions.ch;
online at l3o.net
> Cc: Ceki Gülcü
> Assunto: Implementing MultiFileAppender
>
> Hello Heri and Leo and Patrick,
>
> You might have heard of logback, a rewrite of log4j by Ceki Gülcü, log4j's
> founder.
> Logback has a smaller memory footprint and is faster by a wide margin. It
> has much
> better error reporting capability and is generally more robust. Logback
has
> support
> for markers which enables capabilities not found elsewhere. We will be
> documenting
> these capabilities in the near future.
>
> We've read your mails on the log4j mailing list about a MultiFileAppender,
> and found
> your idea very interesting.
>
> In logback, there is mostly all the bricks one might need to create such a
> component.
>
> Our new version of the Joran configuration framework allows replay of
> configuration
> elements. This is exactly what's needed to create new FileAppenders based
on
> specific
> situations. When replaying configuration sequences, one can replace some
> properties,
> such as file name, to fine tune the new component.
>
> For documentation about Joran, you might want to check the following
> resources
>
> - http://logback.qos.ch/joran.html
> - logback-core/examples/src/joran/ (available from Subversion repo)
> - ch.qos.logback.core.joran.replay.FruitConfigurationTest (Subversion
repo)
>
> Replay functionality in Joran is only available from the Subversion repo.
It
> will be
> included in logback version 0.5.
>
> Regarding FileAppender and derived appenders, logback has similar support
> compared to
> log4j 1.3, in addition to several niceties.
>
> Would you like to contribute MultiFileAppender to logback? If you are
> interested, we
> will gladly provide support for this endeavor.
>
> Kind regards,
>
> Sébastien
>
--
Ceki Gülcü
Logback: The reliable, generic, fast and flexible logging framework for
Java.
http://logback.qos.ch
More information about the Logback-user
mailing list