[logback-user] Trouble with MaxHistory when using TimeBasedRollingPolicy

[Jesse Bexten]

Hello group!

Our company has been a long time user of log4j and JCL and recently I have discovered Logback and SLF4J. I am extremely pleased with the quality of this product thus far. SLF4j immediately resolves the class loader trouble we had been having with our applications using log4j under WAS. Moving forward with this project I had hoped to switch our logging implementation to logback using the RollingFileAppender and TimeBasedRollingPolicy. It exactly meets a new requirement I've been given regarding automatic compression and deletion of archived logs.

 Here's the trouble. The archive remover is a little to specific about which files it deletes at rollover. It computes the name of the file to be deleted based on the current time regardless of the age of the oldest file. So for example my log file pattern is 

${AppLogPath}\app.%d{yyyy-MM-dd-HH-mm}.zip

max history is 5. 

my log directory contains

app.2010-06-02-10-29.zip
app.2010-06-02-10-30.zip
app.2010-06-02-10-31.zip
app.2010-06-02-10-32.zip
app.2010-06-02-10-33.zip

If there is no logging activity until 10:37 then DefaultArchiveRemover subtracts the 6 minutes from the current time and computes the name of the file to be deleted as forms.2010-06-02-10-31.zip. The older files app.2010-06-02-10-29.zip, app.2010-06-02-10-30.zip will never be deleted because the rollover time that would have deleted them has already past. 

I was testing with a 1 minute rollover period which I know is unrealistic,however we still split our logs hourly for our applications. During business hours this problem will not affect us, but we usually have several hours of inactivity later in the evening. Thus we would have hours in the day when rollover doesn't occur and some of our archived logs will not be cleaned up. I would like to see a solution that deletes the oldest entry at rollover or all files older than a certain threshold, or at least some solution or option that won't leave some logs stranded.

One final note we have been achieving the automatic deletion of logs using a custom log4j appender that reads the last modified date of the file and deletes all files older than a specified age that meet the pattern of our log file. I was hoping to be able to replace this with a more efficient method using logback.

Thanks for any help. Sorry my first post is such a long one, but I have see this issue documented in other posts and a few JIRA logs but they do not contain the amount detail I have given which is perhaps why they have been addressed. I feel this a significant problem which makes the maxhistory portion of TimeBasedRollingPolicy unusable for most purposes.

Jesse