[logback-user] logback performance issue

[Jason Tang]

Hi

We use logback for debug log, and find it has performance problem.
Here is the environment and traffic information:
Config:

   - RollingFileAppender
   - Default encoder

Traffic

   - 60~100 concurrent threads write debug log parallel
   - each time, the logback will record a short message (eg, trace
   information, method in/out)
   - several hundreds record per thread per seconds, which means several
   ten thousand records per second total in one JVM.
   - Compress log every 20M

After running our application, we found the performance is poor, and IO,
CPU is not the bottle neck, but check the stack, all threads is waiting on
the *OutputStreamAppender.*subAppend().

*Linux 2.6.32.27-0.2.2.2501.1.PTF-default 03/14/12 _x86_64_
*
*avg-cpu: %user %nice %system %iowait %steal %idle 12.82 0.00 2.20 0.43
0.00 84.55 Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz
await svctm %util sda 0.00 415.50 0.00 9.50 0.00 3400.00 357.89 0.27 28.21
7.37 7.00
*

I searched, and find some previous discuss (
http://mailman.qos.ch/pipermail/logback-user/2011-March/002211.html) and it
is same situation we have here, and the suggestion is to "vote back"
the "ImmediateFlush"
and "BufferSize".

So I wonder if it is a performance problem? or Configuration fault?

When I check the IO stat, I notice the io package is small, and it is same
as the behavior in OutputStreamAppender.

So if it is the existed problem or limitation, can we have the flush
control and buffer size configuration back?

Although we try to have the socketAppender and have our owner buffer
solution when writer to file, but we need to find out what's the real
problem.

BRs
//Jason
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.qos.ch/pipermail/logback-user/attachments/20120313/9d404629/attachment.html>