[logback-user] Release of logback version 1.3.0-alpha10

[Ralph Goers]

> On Aug 30, 2021, at 10:57 AM, Ceki Gülcü <ceki at qos.ch <mailto:ceki at qos.ch>> wrote:
> 
> 
> Responses inline.

> Personally, I was quite surprised by:
> 
> 1) the disappointing performance of the LMAX disruptor.
> 2) the non uniform impact of running the benchmark under a hypervisor/virtual CPU.


Well, here we will have to just disagree. From my analysis you aren’t measuring the performance of the 
disruptor at all. When I ran it under a profiler the disruptor doesn’t even show up. It is all overhead in Log4j’s 
PatternLayout coupled with some of the complexities in the way I/O was done to be garbage free. We are 
working on correcting that and have seen some promising results. But as long as file I/O is the limiting factor 
you really aren’t measuring the performance of the disruptor. 

On item 2, I am not sure what you are referring to. I’ve just been running my tests on my MacBook Pro. While 
I do see that Logback’s FileAppender has been faster my results for Log4j2 look nothing like yours. FYI, here 
are results that I just ran on my MacBook Pro using Log4j 2.15.0-SNAPSHOT. My MacBook Pro says it has 8 
cores.

8 Threads
Benchmark                                                                   Mode  Cnt     Score     Error   Units
AsyncWithFileAppenderBenchmark.log4j1File            thrpt   10  1409.525 ±  57.006  ops/ms
AsyncWithFileAppenderBenchmark.log4j2AsyncFile  thrpt   10  1743.430 ±  51.178  ops/ms
AsyncWithFileAppenderBenchmark.logbackFile         thrpt   10  2157.374 ±  49.966  ops/ms
FileAppenderBenchmark.log4j1File                             thrpt   10   764.812 ±  13.624  ops/ms
FileAppenderBenchmark.log4j2File                             thrpt   10  2281.328 ± 148.979  ops/ms
FileAppenderBenchmark.logbackFile                          thrpt   10  1946.389 ±  62.855  ops/ms


16 Threads
Benchmark                                                                  Mode  Cnt     Score     Error   Units
AsyncWithFileAppenderBenchmark.log4j1File            thrpt   10  1228.149 ±  76.286  ops/ms
AsyncWithFileAppenderBenchmark.log4j2AsyncFile  thrpt   10  1590.441 ±  29.938  ops/ms
AsyncWithFileAppenderBenchmark.logbackFile         thrpt   10  1994.770 ± 105.827  ops/ms
FileAppenderBenchmark.log4j1File                             thrpt   10   825.138 ±  18.012  ops/ms
FileAppenderBenchmark.log4j2File                             thrpt   10  2231.246 ± 111.451  ops/ms
FileAppenderBenchmark.logbackFile                          thrpt   10  1884.429 ±  48.463  ops/ms

My Mac apparently doesn’t do hyper threading so if that is what you mean I don’t have something handy to 
test it on.

I will admit that both the results above for both Log4j 1 and Log4j 2 are a bit surprising. Log4j 1’s synchronous 
test should be much faster. And Log4j 2’s asynchronous test shouldn’t be much different than the synchronous 
test.  You can be sure that we are continuing to look at these until we can explain the results better.

> 
> While I will not be ordered around, I remain open to suggestions
> including alternative ways of benchmarking.
> 

I apologize if you felt like I was commanding you to do something. I certainly know better than that.

What I am suggesting is that the whole section above "Comparing log4j, log4j2 and logback” is fine IMO. 
My issue is that when readers get to the section below there is no mention that what is being benchmarked 
is asynchronous logging with the queues full. For example, "The above results show that throughput in 
synchronous logging is actually higher than that of asynchronous logging” would be more clear if you simply 
added “when the number of incoming events exceeds the speed of the appender.” to the end of the sentence.
Otherwise the casual reader will believe it is truly measuring just the overhead of asynchronous logging.


>> Also, I noticed that you have configured Logback’s FileAppender with a 256KB buffer but left Log4j2’s
>> appender at its default of 8KB.
> 
> This is a fair point. I have modified the configuration files [3] and will run the benchmarks again.
> 
> [3] https://github.com/ceki/logback-perf/commit/9736a37f76492b <https://github.com/ceki/logback-perf/commit/9736a37f76492b>

I looked at the commit. I still don’t see bufferSize=“262144” added to the configuration for Log4j2.

Ralph
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.qos.ch/pipermail/logback-user/attachments/20210830/e9d04e02/attachment.html>