[logback-user] FileAppender Performance without IOBuffering

[Ceki Gulcu]

Even without "BufferedIO" logback can log about 100'000 events per 
second. Yes, one hundred thousand events per second. The "BufferedIO" 
terminology as applicable to FileOutputStream is a misnomer because at 
the disk level FileOutputStream will already buffers disk access 
regardless of whether "BufferedIO" is enabled or not. "BufferedIO" 
actually pertains to whether FileOutputStream will buffer (java) char to 
byte conversions. It has nothing to do with disk access.

Given the above, the "BufferedIO" was removed in more recent versions of 
logback.

HTH,

On 23.10.2010 02:06, Hontvári József Levente wrote:
>   I measured a similar setup quite a few years ago. Logback did not even
> exist then, we used our own simple logging library. The measured
> software was a server which received messages on several TCP
> connections, did some calculation, most of the time it cached the
> result, but eventually it saved results to a MySQL database. There was
> one log entry per message (similarly to an HTTP access log)., about
> 50-100 log entries per second. I found that unbuffered logging reduced
> the throughput of the server drastically, by 30%. On the other hand, if
> buffering was switched on then the performance impact was negligible. I
> measured it on Windows XP with a standard desktop hard disk, likely some
> WD PATA 7200 RPM drive.
>
> I believe that logging 1000 or more entries is practically impossible if
> flushing is switched off. A single hard disk may do a few hundred IO
> operation / s. Even if repeatedly appending to the same file is a quick
> operation, this is simply too much. Maybe if you have a battery backed
> SAS controller, that could help, in that case the controller itself
> could buffer writes, but I am just thinking, I have absolutely no
> experience with this. It is clear that you need some kind of buffering.
> I am not sure, however, if all kind of buffering was eliminated from
> logback or just one type. I am sure Ceki will give us the answer.
>
>
>
> On 2010.10.22. 22:59, Kenneth.Lam at barclayscapital.com wrote:
>>
>> Hi,
>>
>> We’re looking to use Logback with one of our services to aid in
>> metrics recording. I’ve noticed that the ability to use any form of
>> buffered IO with the FileAppender class was removed. The reason cited
>> was that significant performance testing showed little benefit in
>> doing so. I was wondering if anybody could provide more detail
>> concerning the environment and conditions that this testing was done.
>> We will generally be logging entries on the order of 1000’s to
>> 10,000’s entries per second. Does anybody know if Logback can handle
>> this type of loading? Thanks.
>>
>> _______________________________________________
>>
>> This e-mail may contain information that is confidential, privileged
>> or otherwise protected from disclosure. If you are not an intended
>> recipient of this e-mail, do not duplicate or redistribute it by any
>> means. Please delete it and any attachments and notify the sender that
>> you have received it in error. Unless specifically indicated, this
>> e-mail is not an offer to buy or sell or a solicitation to buy or sell
>> any securities, investment products or other financial product or
>> service, an official confirmation of any transaction, or an official
>> statement of Barclays. Any views or opinions presented are solely
>> those of the author and do not necessarily represent those of
>> Barclays. This e-mail is subject to terms available at the following
>> link: www.barcap.com/emaildisclaimer
>> <http://www.barcap.com/emaildisclaimer>. By messaging with Barclays
>> you consent to the foregoing. Barclays Capital is the investment
>> banking division of Barclays Bank PLC, a company registered in England
>> (number 1026167) with its registered office at 1 Churchill Place,
>> London, E14 5HP. This email may relate to or be sent from other
>> members of the Barclays Group.//
>>