<style>
/* Changing the layout to use less space for mobiles */
@media screen and (max-device-width: 480px), screen and (-webkit-min-device-pixel-ratio: 2) {
#email-body { min-width: 30em !important; }
#email-page { padding: 8px !important; }
#email-banner { padding: 8px 8px 0 8px !important; }
#email-avatar { margin: 1px 8px 8px 0 !important; padding: 0 !important; }
#email-fields { padding: 0 8px 8px 8px !important; }
#email-gutter { width: 0 !important; }
}
</style>
<div id="email-body">
<table id="email-wrap" align="center" border="0" cellpadding="0" cellspacing="0" style="background-color:#f0f0f0;color:#000000;width:100%;">
<tr valign="top">
<td id="email-page" style="padding:16px !important;">
<table align="center" border="0" cellpadding="0" cellspacing="0" style="background-color:#ffffff;border:1px solid #bbbbbb;color:#000000;width:100%;">
<tr valign="top">
<td bgcolor="#003366" style="background-color:#003366;color:#ffffff;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;line-height:1;"><img src="http://jira.qos.ch/s/en_USb9v8he-418945332/850/25/_/jira-logo-scaled.png" alt="" style="vertical-align:top;" /></td>
</tr><tr valign="top">
<td id="email-banner" style="padding:32px 32px 0 32px;">
<table align="left" border="0" cellpadding="0" cellspacing="0" width="100%" style="width:100%;">
<tr valign="top">
<td style="color:#505050;font-family:Arial,FreeSans,Helvetica,sans-serif;padding:0;">
<img id="email-avatar" src="http://jira.qos.ch/secure/useravatar?avatarId=10122" alt="" height="48" width="48" border="0" align="left" style="padding:0;margin: 0 16px 16px 0;" />
<div id="email-action" style="padding: 0 0 8px 0;font-size:12px;line-height:18px;">
<a class="user-hover" rel="mikej1688@gmail.com" id="email_mikej1688@gmail.com" href="http://jira.qos.ch/secure/ViewProfile.jspa?name=mikej1688%40gmail.com" style="color:#326ca6;">Mike J</a>
created <img src="http://jira.qos.ch/images/icons/issuetypes/bug.png" height="16" width="16" border="0" align="absmiddle" alt="Bug"> <a style='color:#326ca6;text-decoration:none;' href='http://jira.qos.ch/browse/LOGBACK-879'>LOGBACK-879</a>
</div>
<div id="email-summary" style="font-size:16px;line-height:20px;padding:2px 0 16px 0;">
<a style='color:#326ca6;text-decoration:none;' href='http://jira.qos.ch/browse/LOGBACK-879'><strong>Logback zip compression cannot delete the original files after the zip file is created in a case of multiple jvms</strong></a>
</div>
</td>
</tr>
</table>
</td>
</tr>
<tr valign="top">
<td id="email-fields" style="padding:0 32px 32px 32px;">
<table border="0" cellpadding="0" cellspacing="0" style="padding:0;text-align:left;width:100%;" width="100%">
<tr valign="top">
<td id="email-gutter" style="width:64px;white-space:nowrap;"></td>
<td>
<table border="0" cellpadding="0" cellspacing="0" width="100%">
<tr valign="top">
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 10px 10px 0;white-space:nowrap;">
<strong style="font-weight:normal;color:#505050;">Issue Type:</strong>
</td>
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 0 10px 0;width:100%;">
<img src="http://jira.qos.ch/images/icons/issuetypes/bug.png" height="16" width="16" border="0" align="absmiddle" alt="Bug"> Bug
</td>
</tr> <tr valign="top">
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 10px 10px 0;white-space:nowrap;">
<strong style="font-weight:normal;color:#505050;">Affects Versions:</strong>
</td>
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 0 10px 0;width:100%;">
1.0.13 </td>
</tr>
<tr valign="top">
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 10px 10px 0;white-space:nowrap;">
<strong style="font-weight:normal;color:#505050;">Assignee:</strong>
</td>
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 0 10px 0;width:100%;">
<a class="user-hover" rel="logback-dev@qos.ch" id="email_logback-dev@qos.ch" href="http://jira.qos.ch/secure/ViewProfile.jspa?name=logback-dev%40qos.ch" style="color:#326ca6;">Logback dev list</a>
</td>
</tr> <tr valign="top">
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 10px 10px 0;white-space:nowrap;">
<strong style="font-weight:normal;color:#505050;">Components:</strong>
</td>
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 0 10px 0;width:100%;">
logback-core </td>
</tr>
<tr valign="top">
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 10px 10px 0;white-space:nowrap;">
<strong style="font-weight:normal;color:#505050;">Created:</strong>
</td>
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 0 10px 0;width:100%;">
02/Jul/13 11:49 PM
</td>
</tr> <tr valign="top">
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 10px 10px 0;white-space:nowrap;">
<strong style="font-weight:normal;color:#505050;">Description:</strong>
</td>
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 0 10px 0;width:100%;">
<p style='margin-top:0;margin-bottom:10px;'>Our app is a java app with multiple jvms involved. That is, the main java app starts other java apps as sub-processes by launching them via java's ProcessBuilder. All java apps here use the same logback.xml and log info to the same log file, $</p>
{THE_LOG_DIR}/hhss-%d{yyyy-MM-dd-HH-mm}.%i.log.zip.<br/>
<br/>
Here is the logback.xml used:<br/>
<br/>
<configuration debug="false" scan="true"><br/>
<property name="THE_LOG_DIR" value="C:/ProgramData/HH/SS/LOG"/><br/>
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender"><br/>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"><br/>
<fileNamePattern>${THE_LOG_DIR}
<p style='margin-top:0;margin-bottom:10px;'>/hhss-%d</p>
{yyyy-MM-dd-HH-mm}
<p style='margin-top:0;margin-bottom:10px;'>.%i.log.zip</fileNamePattern> <br/>
<maxHistory>30</maxHistory><br/>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP"> <br/>
<maxFileSize>5MB</maxFileSize><br/>
</timeBasedFileNamingAndTriggeringPolicy> <br/>
</rollingPolicy><br/>
<encoder><br/>
<pattern>%d</p>
{dd MMM yyyy HH:mm:ss.SSS}
<p style='margin-top:0;margin-bottom:10px;'> %X</p>
{process}
<p style='margin-top:0;margin-bottom:10px;'><span class="error">[%thread]</span> %-5level %logger</p>
{36}
<p style='margin-top:0;margin-bottom:10px;'> - %msg%n</pattern><br/>
</encoder><br/>
</appender><br/>
<root level="debug"><br/>
<appender-ref ref="FILE"/><br/>
</root></p>
<p style='margin-top:0;margin-bottom:10px;'> </configuration></p>
<p style='margin-top:0;margin-bottom:10px;'>The test case is divided into 3 steps:</p>
<p style='margin-top:0;margin-bottom:10px;'>1). First tested the simplest case where the main app had no any java sub-process. In this(single jvm) case, the zip file was created with no problem and the original to-be-zipped (or unzipped)files were deleted. that's what is expected;</p>
<p style='margin-top:0;margin-bottom:10px;'>2). Let the main app start just one java sub-process. In this (2-jvm) case, both apps kept logging info to the same log file. It was found that the logging data was not heavy, and almost all unzipped files were removed after the compression but there were some left behind;</p>
<p style='margin-top:0;margin-bottom:10px;'>3). The main java app started all java sub-processes (totally 6) and some of them logged info heavily to the same log file. As you can see from the logback.xml above, the rotation and zip compression occurred every minute. In this (multi-jvm)case, the zip files were created by logback with no problem, but almost all the unzipped files were left behind.</p>
<p style='margin-top:0;margin-bottom:10px;'>So the item 2). and 3). are not what we expect. why did it happen? I checked the relevant class, ch.qos.logback.core.rolling.helper.Compressor and found the method zipCompress(...) indeed has a deletion operation over the old unzipped files, </p>
<p style='margin-top:0;margin-bottom:10px;'>if (!file2zip.delete())</p>
{
addStatus(new WarnStatus("Could not delete [" + nameOfFile2zip + "].", this));
}
<p style='margin-top:0;margin-bottom:10px;'>but in a multi-jvm environment (maybe a multi-threading env has the same issue), it was found that the deletion cannot be happened. Why? Here is my explanation.</p>
<p style='margin-top:0;margin-bottom:10px;'>1). The compression process is asynchronous (via AsynchronousCompressor.java). The various jvms might hold the file's handle to log info to the same log file alternatively. If a jvm is not the owner of the file handle at the zip compression time, then it might not have privileges to delete the unzipped file because it is being held by another jvm. If no SecurityManager is set, then the File.delete() only returns a boolean false, nothing else;<br/>
2). Tn the method zipCompress(...), the above piece of code is called only after the zip compression is successfully done. It implies when the zip compression is only done by a jvm, then the deletion is executed by the same jvm. It didn't consider a special case: due to the compression's asynchronous feature, a jvm can do the zip compression, and another jvm can do the deletion operation immediately after the zip compression. I indeed found such case in our testing: before the main java app did the zip compression, it was found that the zip file was already created by another jvm. In this case, the current code didn't handle it right, I think(still in the method of zipCompress(...))</p>
<p style='margin-top:0;margin-bottom:10px;'>if (zippedFile.exists()) </p>
{
addStatus(new WarnStatus("The target compressed file named ["
+ nameOfZippedFile + "] exist already.", this));
return;
}
<p style='margin-top:0;margin-bottom:10px;'>It did not do any thing, and then quit. since the jvm which zipped the file didn't get the file handle at the time, the jvm can also not delete the unzipped file. the file was left behind. The more the jvm are, the more unzipped files are left behind.</p>
<p style='margin-top:0;margin-bottom:10px;'>A possible solution to fix it is simple - adding a few lines to the block,</p>
<p style='margin-top:0;margin-bottom:10px;'>if (zippedFile.exists())<br/>
{<br/>
addStatus(new WarnStatus("The target compressed file named <span class="error">[" + nameOfZippedFile + "]</span> exist already.", this));<br/>
try<br/>
{<br/>
Path path = FileSystems.getDefault().getPath(nameOfFile2zip);<br/>
while(!Files.deleteIfExists(path))</p>
{
addStatus(new WarnStatus("Could not delete [" + nameOfFile2zip + "].", this));
}
<p style='margin-top:0;margin-bottom:10px;'> }<br/>
catch (Exception e)</p>
{
addStatus(new WarnStatus("Could not delete [" + nameOfFile2zip + "].", this));
}<br/>
catch (Throwable t)<br/>
{
addStatus(new WarnStatus("Could not delete [" + nameOfFile2zip + "].", this));
}
<p style='margin-top:0;margin-bottom:10px;'> return;<br/>
}</p>
<p style='margin-top:0;margin-bottom:10px;'>That is, even the zip file is found existing (it might be created by another jvm), the current jvm should still delete the unzipped file since it own the file handle at the time.</p>
<p style='margin-top:0;margin-bottom:10px;'>I have tested the change in the case 3). where all java jvms (6 jvms)share the same logback.xml and log to the same log file. it worked as expected.</p>
</td>
</tr>
<tr valign="top">
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 10px 10px 0;white-space:nowrap;">
<strong style="font-weight:normal;color:#505050;">Environment:</strong>
</td>
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 0 10px 0;width:100%;">
<p style='margin-top:0;margin-bottom:10px;'>OS: windows 2008 server R2 (64 bits)<br/>
java version: jse 1.7.0_21</p>
</td>
</tr>
<tr valign="top">
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 10px 10px 0;white-space:nowrap;">
<strong style="font-weight:normal;color:#505050;">Project:</strong>
</td>
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 0 10px 0;width:100%;">
<a style="color:#326ca6;" href="http://jira.qos.ch/browse/LOGBACK">logback</a>
</td>
</tr> <tr valign="top">
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 10px 10px 0;white-space:nowrap;">
<strong style="font-weight:normal;color:#505050;">Labels:</strong>
</td>
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 0 10px 0;width:100%;">
none
</td>
</tr>
<tr valign="top">
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 10px 10px 0;white-space:nowrap;">
<strong style="font-weight:normal;color:#505050;">Priority:</strong>
</td>
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 0 10px 0;width:100%;">
<img src="http://jira.qos.ch/images/icons/priorities/major.png" height="16" width="16" border="0" align="absmiddle" alt="Major"> Major
</td>
</tr>
<tr valign="top">
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 10px 10px 0;white-space:nowrap;">
<strong style="font-weight:normal;color:#505050;">Reporter:</strong>
</td>
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 0 10px 0;width:100%;">
<a class="user-hover" rel="mikej1688@gmail.com" id="email_mikej1688@gmail.com" href="http://jira.qos.ch/secure/ViewProfile.jspa?name=mikej1688%40gmail.com" style="color:#326ca6;">Mike J</a>
</td>
</tr> </table>
</td>
</tr>
</table>
</td>
</tr>
</table>
</td><!-- End #email-page -->
</tr>
<tr valign="top">
<td style="color:#505050;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:10px;line-height:14px;padding: 0 16px 16px 16px;text-align:center;">
This message is automatically generated by JIRA.<br />
If you think it was sent incorrectly, please contact your JIRA administrators<br />
For more information on JIRA, see: <a style='color:#326ca6;' href='http://www.atlassian.com/software/jira'>http://www.atlassian.com/software/jira</a>
</td>
</tr>
</table><!-- End #email-wrap -->
</div><!-- End #email-body -->