[logback-user] FileAppender Performance without IOBuffering

[Hontvári József Levente]

I measured a similar setup quite a few years ago. Logback did not even 
exist then, we used our own simple logging library. The measured 
software was a server which received messages on several TCP 
connections, did some calculation, most of the time it cached the 
result, but eventually it saved results to a MySQL database. There was 
one log entry per message (similarly to an HTTP access log)., about 
50-100 log entries per second. I found that unbuffered logging reduced 
the throughput of the server drastically, by 30%. On the other hand, if 
buffering was switched on then the performance impact was negligible. I 
measured it on Windows XP with a standard desktop hard disk, likely some 
WD PATA 7200 RPM drive.

I believe that logging 1000 or more entries is practically impossible if 
flushing is switched off. A single hard disk may do a few hundred IO 
operation / s. Even if repeatedly appending to the same file is a quick 
operation, this is simply too much. Maybe if you have a battery backed 
SAS controller, that could help, in that case the controller itself 
could buffer writes, but I am just thinking, I have absolutely no 
experience with this. It is clear that you need some kind of buffering. 
I am not sure, however, if all kind of buffering was eliminated from 
logback or just one type. I am sure Ceki will give us the answer.



On 2010.10.22. 22:59, Kenneth.Lam at barclayscapital.com wrote:
>
> Hi,
>
> We're looking to use Logback with one of our services to aid in 
> metrics recording.  I've noticed that the ability to use any form of 
> buffered IO with the FileAppender class was removed.  The reason cited 
> was that significant performance testing showed little benefit in 
> doing so.  I was wondering if anybody could provide more detail 
> concerning the environment and conditions that this testing was done. 
>  We will generally be logging entries on the order of 1000's to 
> 10,000's entries per second.  Does anybody know if Logback can handle 
> this type of loading?  Thanks.
>
> _______________________________________________
>
> This e-mail may contain information that is confidential, privileged 
> or otherwise protected from disclosure. If you are not an intended 
> recipient of this e-mail, do not duplicate or redistribute it by any 
> means. Please delete it and any attachments and notify the sender that 
> you have received it in error. Unless specifically indicated, this 
> e-mail is not an offer to buy or sell or a solicitation to buy or sell 
> any securities, investment products or other financial product or 
> service, an official confirmation of any transaction, or an official 
> statement of Barclays. Any views or opinions presented are solely 
> those of the author and do not necessarily represent those of 
> Barclays. This e-mail is subject to terms available at the following 
> link: www.barcap.com/emaildisclaimer 
> <http://www.barcap.com/emaildisclaimer>. By messaging with Barclays 
> you consent to the foregoing.Barclays Capital is the investment 
> banking division of Barclays Bank PLC, a company registered in England 
> (number 1026167) with its registered office at 1 Churchill Place, 
> London, E14 5HP.This email may relate to or be sent from other members 
> of the Barclays Group.//
>
> _______________________________________________
>
>
> _______________________________________________
> Logback-user mailing list
> Logback-user at qos.ch
> http://qos.ch/mailman/listinfo/logback-user

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://qos.ch/pipermail/logback-user/attachments/20101023/69dd2f6a/attachment.html>