From e.vanoosten at grons.nl Sun Feb 2 07:47:43 2025 From: e.vanoosten at grons.nl (Erik van Oosten) Date: Sun, 2 Feb 2025 08:47:43 +0100 Subject: [slf4j-user] When more parameters than placeholders, extra parameters are not logged In-Reply-To: <1250089793.3587854.1737769097501@mail.yahoo.com> References: <1250089793.3587854.1737769097501.ref@mail.yahoo.com> <1250089793.3587854.1737769097501@mail.yahoo.com> Message-ID: <078b595c-e43d-4ac8-9268-2ea742927e0c@grons.nl> Hi Russ, Mailing lists are not so popular anymore nowadays, chances that a sl4fj developer will respond here are not so good. Probably the best thing to do is write a feature request on https://github.com/qos-ch/slf4j/issues. A pull request is probably be appreciated as well! Kind regards,     Erik. Op 25-01-2025 om 02:38 schreef Russ Block via slf4j-user: > With this method or any similar method, >     public void info(String format, Object... arguments) { >         if (isInfoEnabled()) { >             handleArgArrayCall(Level.INFO, null, format, arguments); >         } >     } > If the number of objects in 'arguments' is more than the number of > placeholders, {}, the extra argument objects are not logged.  Is there > a reason for that? > > For example >   public void setTemperature(Integer temperature) { >     oldT = t; >     t = temperature; >   //Sent three parameters log with two placeholders >   //Output "Temperature set to 90. Old value was null." The word extra > is not printed >     logger.info("Temperature set to {}. Old value was {}.", t, oldT, > "extra"); >     if(temperature.intValue() > 50) { >       logger.info("Temperature has risen above 50 degrees."); >     } >   } > > > If I get into a situation where I have sent more argument objects than > placeholders that it should append those un-placed arguments to the > end of the logged string.  The intent of the developer is to have the > parameter logged. Finding this out from production is not good.  What > is sent to be logged is important information, depending on the log level. > > My use case is that I want to push some legacy code to start looking > like SLF4J so we can migrate away from the older pattern filled with > isLogTraceEnabled calls.  I learned of the above when I was writing > JUnit tests to see all the things the code would do and not do. > > I feel that https://www.slf4j.org/faq.html#logging_performance is > telling me that, in some way, the placeholder string substitution is > faster than appending multiple strings together.  I understood the > main point of the argument as converting items to string before I > needed them was where the overhead was.  Keeping them as type Object > reduces that overhead. > > I understand there are Sonarqube rules and PMD to prevent me from not > having the correct number of argument objects.  This would help, but I > feel like there are still going to be misses.  Since my code is > custom, Sonarqube and PMD do not feel like options. > > _______________________________________________ > slf4j-user mailing list > slf4j-user at qos.ch > https://mailman.qos.ch/cgi-bin/mailman/listinfo/slf4j-user -- Erik van Oosten e.vanoosten at grons.nl https://day-to-day-stuff.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From jo at durchholz.org Sun Feb 2 09:30:09 2025 From: jo at durchholz.org (jo at durchholz.org) Date: Sun, 2 Feb 2025 10:30:09 +0100 Subject: [slf4j-user] When more parameters than placeholders, extra parameters are not logged In-Reply-To: <1250089793.3587854.1737769097501@mail.yahoo.com> References: <1250089793.3587854.1737769097501.ref@mail.yahoo.com> <1250089793.3587854.1737769097501@mail.yahoo.com> Message-ID: On 25.01.25 02:38, Russ Block via slf4j-user wrote: > With this method or any similar method, >     public void info(String format, Object... arguments) { >         if (isInfoEnabled()) { >             handleArgArrayCall(Level.INFO, null, format, arguments); >         } >     } > If the number of objects in 'arguments' is more than the number of > placeholders, {}, the extra argument objects are not logged.  Is there a > reason for that? I can't answer authoritatively, but I guess it is outside the scope of the specification as-is. For people using IntelliJ it is not a problem as it knows about SLF4J and flags log lines with extra or missing arguments. Even better, IntelliJ will give you a refactoring option to replace logger.log("foo is: " + foo) with logger.log("foo is: {}", foo) It also correctly handles the case with multiple parameters, and there's even a hotkey for making it happen that my brain does not remember anymore but my fingers do. I hope other IDEs have started offering similar functionality, but I don't know specifics. IntelliJ does have some deficits, in particular its cache management if pretty flaky compared to Eclipse's, but I found IntelliJ's refactoring support such a time-save that I embraced it and never looked back, though other IDEs might (should, actually!) have caught up since. > If I get into a situation where I have sent more argument objects than > placeholders that it should append those un-placed arguments to the end > of the logged string.  The intent of the developer is to have the > parameter logged.  Finding this out from production is not good.  What > is sent to be logged is important information, depending on the log level. The knee-jerk answer to that would be "you have a bug in your code, just fix it". (I believe that you want something sensible but your arguments are not strong enough. I.e. I'm going to strike down the arguments and suggesting a solution, so don't get disappointed before the end.) > My use case is that I want to push some legacy code to start looking > like SLF4J so we can migrate away from the older pattern filled with > isLogTraceEnabled calls. I'd assume that you had extra parameters in the legacy code as well, so your situation did not get worse, did it? > I learned of the above when I was writing > JUnit tests to see all the things the code would do and not do. > > I feel that https://www.slf4j.org/faq.html#logging_performance is > telling me that, in some way, the placeholder string substitution is > faster than appending multiple strings together.  I understood the main > point of the argument as converting items to string before I needed them > was where the overhead was.  Keeping them as type Object reduces that > overhead. Actually, while the placeholder substitution _is_ faster than a string append, the real advantage is this: With logger.debug("foo is: " + foo), the + operator will be called before logger.debug starts, and foo.toString() and string concatenation happen whether logger.debug does anything with it or not. With logger.debug("foo is: {}", foo), logger.debug will FIRST check if it needs to do anything, and ONLY THEN will it do string substitution. The string substitution can actually be slightly slower than the concatenation, because there's the extra step of extracting the substitution markers and copying the text to the left and right of them. However, the substitution code is heavily optimized and should still be faster anyway in the vast majority of the cases. This is, however, just for the do-log case. The main speedup comes from the no-log case doing less work. > I understand there are Sonarqube rules and PMD to prevent me from not > having the correct number of argument objects.  This would help, but I > feel like there are still going to be misses.  Since my code is custom, > Sonarqube and PMD do not feel like options. You can configure these things to report only those bugs that you are interested in. The main deterrent is that the setup takes time. Now to the promised solution, should you still need it: You can do your own Formatter that appends any the extraneous parameters. Caveat: Formatters do not live in SLF4J, they live in the logging backend - typically Logback. (SLF4J has a different and somewhat confusing terminology for "backend" that I don't remember.) SLF4J merely manages all the information needed to decide whether a log message should actually be output, and leaves the actual outputting to the backend. Anyway, you'll have to dive into the code of the existing standard formatter, find out how it's working and how to make it append the extra parameters. For Logback, you do not have to write your own backend, you can configure your Formatter, and even inherit from Logback's standard Formatter so you do not have to do a lot; however, you'll want to make 150% sure that you do not introduce any bugs, so plan a lot of review (if possible) and testing. Fortunately it is more time reading docs and code than actual coding, so the extra effort for quality control will not be very noticeable. Still, I believe the refactoring support of an IDE, if possible, will be easier :-) HTH! Jo From ceki at qos.ch Sun Feb 2 10:26:18 2025 From: ceki at qos.ch (Ceki Gulcu) Date: Sun, 2 Feb 2025 11:26:18 +0100 Subject: [slf4j-user] When more parameters than placeholders, extra parameters are not logged In-Reply-To: <078b595c-e43d-4ac8-9268-2ea742927e0c@grons.nl> References: <1250089793.3587854.1737769097501.ref@mail.yahoo.com> <1250089793.3587854.1737769097501@mail.yahoo.com> <078b595c-e43d-4ac8-9268-2ea742927e0c@grons.nl> Message-ID: <81950b7a-cdba-4a4b-ac05-c0d9b82b4427@qos.ch> Hi Erik, Thank you for your message. In this case, Ross cross posted on slf4j-dev and I has already provided an answer there. I repeat it here for the convenience of the reader. Catering for the eventuality of extra parameters would be relatively complicated. More importantly, I think that it is not SLF4J's responsibility to fix every possible error that could be made by the programmer. In addition, the fluent API allows adding key value pairs which should avoid the pitfall described by Ross. Best regards, -- Ceki Gülcü Sponsoring SLF4J/logback/reload4j at https://github.com/sponsors/qos-ch On 02/02/2025 08:47, Erik van Oosten via slf4j-user wrote: > Hi Russ, > > Mailing lists are not so popular anymore nowadays, chances that a sl4fj > developer will respond here are not so good. Probably the best thing to > do is write a feature request on https://github.com/qos-ch/slf4j/issues. > A pull request is probably be appreciated as well! > > Kind regards, >     Erik. > > > Op 25-01-2025 om 02:38 schreef Russ Block via slf4j-user: >> With this method or any similar method, >>     public void info(String format, Object... arguments) { >>         if (isInfoEnabled()) { >>             handleArgArrayCall(Level.INFO, null, format, arguments); >>         } >>     } >> If the number of objects in 'arguments' is more than the number of >> placeholders, {}, the extra argument objects are not logged.  Is there >> a reason for that? >> >> For example >>   public void setTemperature(Integer temperature) { >>     oldT = t; >>     t = temperature; >>   //Sent three parameters log with two placeholders >>   //Output "Temperature set to 90. Old value was null." The word extra >> is not printed >>     logger.info("Temperature set to {}. Old value was {}.", t, oldT, >> "extra"); >>     if(temperature.intValue() > 50) { >>       logger.info("Temperature has risen above 50 degrees."); >>     } >>   } >> >> >> If I get into a situation where I have sent more argument objects than >> placeholders that it should append those un-placed arguments to the >> end of the logged string.  The intent of the developer is to have the >> parameter logged. Finding this out from production is not good.  What >> is sent to be logged is important information, depending on the log level. >> >> My use case is that I want to push some legacy code to start looking >> like SLF4J so we can migrate away from the older pattern filled with >> isLogTraceEnabled calls.  I learned of the above when I was writing >> JUnit tests to see all the things the code would do and not do. >> >> I feel that https://www.slf4j.org/faq.html#logging_performance is >> telling me that, in some way, the placeholder string substitution is >> faster than appending multiple strings together.  I understood the >> main point of the argument as converting items to string before I >> needed them was where the overhead was.  Keeping them as type Object >> reduces that overhead. >> >> I understand there are Sonarqube rules and PMD to prevent me from not >> having the correct number of argument objects.  This would help, but I >> feel like there are still going to be misses.  Since my code is >> custom, Sonarqube and PMD do not feel like options. >> >> _______________________________________________ >> slf4j-user mailing list >> slf4j-user at qos.ch >> https://mailman.qos.ch/cgi-bin/mailman/listinfo/slf4j-user > > -- > Erik van Oosten > e.vanoosten at grons.nl > https://day-to-day-stuff.blogspot.com > > > _______________________________________________ > slf4j-user mailing list > slf4j-user at qos.ch > https://mailman.qos.ch/cgi-bin/mailman/listinfo/slf4j-user From ceki at qos.ch Sun Feb 2 10:34:41 2025 From: ceki at qos.ch (Ceki Gulcu) Date: Sun, 2 Feb 2025 11:34:41 +0100 Subject: [slf4j-user] When more parameters than placeholders, extra parameters are not logged In-Reply-To: References: <1250089793.3587854.1737769097501.ref@mail.yahoo.com> <1250089793.3587854.1737769097501@mail.yahoo.com> Message-ID: <47122ea1-2a6b-457e-9e56-375d8b15e692@qos.ch> Hi Joachim, Thank you for your thoughtful comments. As a side note, I would like to insist that using parameters messages should be preferred to message and parameter concatenation for the performance argument of the no-log case you mentioned. For the log-enabled case, for parameterized message the back-end can perform optimizations that it cannot perform if the message already incorporates the parameters via string concatenation. Best regards, On 02/02/2025 10:30, Joachim Durchholz via slf4j-user wrote: > On 25.01.25 02:38, Russ Block via slf4j-user wrote: >> With this method or any similar method, >>      public void info(String format, Object... arguments) { >>          if (isInfoEnabled()) { >>              handleArgArrayCall(Level.INFO, null, format, arguments); >>          } >>      } >> If the number of objects in 'arguments' is more than the number of >> placeholders, {}, the extra argument objects are not logged.  Is there >> a reason for that? > > I can't answer authoritatively, but I guess it is outside the scope of > the specification as-is. > > For people using IntelliJ it is not a problem as it knows about SLF4J > and flags log lines with extra or missing arguments. > Even better, IntelliJ will give you a refactoring option to replace >   logger.log("foo is: " + foo) > with >   logger.log("foo is: {}", foo) > It also correctly handles the case with multiple parameters, and there's > even a hotkey for making it happen that my brain does not remember > anymore but my fingers do. > > I hope other IDEs have started offering similar functionality, but I > don't know specifics. > IntelliJ does have some deficits, in particular its cache management if > pretty flaky compared to Eclipse's, but I found IntelliJ's refactoring > support such a time-save that I embraced it and never looked back, > though other IDEs might (should, actually!) have caught up since. > >> If I get into a situation where I have sent more argument objects than >> placeholders that it should append those un-placed arguments to the >> end of the logged string.  The intent of the developer is to have the >> parameter logged.  Finding this out from production is not good.  What >> is sent to be logged is important information, depending on the log >> level. > > The knee-jerk answer to that would be "you have a bug in your code, just > fix it". > (I believe that you want something sensible but your arguments are not > strong enough. I.e. I'm going to strike down the arguments and > suggesting a solution, so don't get disappointed before the end.) > >> My use case is that I want to push some legacy code to start looking >> like SLF4J so we can migrate away from the older pattern filled with >> isLogTraceEnabled calls. > > I'd assume that you had extra parameters in the legacy code as well, so > your situation did not get worse, did it? > > > I learned of the above when I was writing > JUnit tests to see all > the things the code would do and not do. > > >> I feel that https://www.slf4j.org/faq.html#logging_performance is >> telling me that, in some way, the placeholder string substitution is >> faster than appending multiple strings together.  I understood the >> main point of the argument as converting items to string before I >> needed them was where the overhead was.  Keeping them as type Object >> reduces that overhead. > > Actually, while the placeholder substitution _is_ faster than a string > append, the real advantage is this: > With logger.debug("foo is: " + foo), the + operator will be called > before logger.debug starts, and foo.toString() and string concatenation > happen whether logger.debug does anything with it or not. > With logger.debug("foo is: {}", foo), logger.debug will FIRST check if > it needs to do anything, and ONLY THEN will it do string substitution. > > The string substitution can actually be slightly slower than the > concatenation, because there's the extra step of extracting the > substitution markers and copying the text to the left and right of them. > However, the substitution code is heavily optimized and should still be > faster anyway in the vast majority of the cases. > This is, however, just for the do-log case. The main speedup comes from > the no-log case doing less work. > > >> I understand there are Sonarqube rules and PMD to prevent me from not >> having the correct number of argument objects.  This would help, but I >> feel like there are still going to be misses.  Since my code is >> custom, Sonarqube and PMD do not feel like options. > You can configure these things to report only those bugs that you are > interested in. > The main deterrent is that the setup takes time. > > Now to the promised solution, should you still need it: > > You can do your own Formatter that appends any the extraneous parameters. > > Caveat: Formatters do not live in SLF4J, they live in the logging > backend - typically Logback. (SLF4J has a different and somewhat > confusing terminology for "backend" that I don't remember.) SLF4J merely > manages all the information needed to decide whether a log message > should actually be output, and leaves the actual outputting to the backend. > > Anyway, you'll have to dive into the code of the existing standard > formatter, find out how it's working and how to make it append the extra > parameters. > For Logback, you do not have to write your own backend, you can > configure your Formatter, and even inherit from Logback's standard > Formatter so you do not have to do a lot; however, you'll want to make > 150% sure that you do not introduce any bugs, so plan a lot of review > (if possible) and testing. Fortunately it is more time reading docs and > code than actual coding, so the extra effort for quality control will > not be very noticeable. > > Still, I believe the refactoring support of an IDE, if possible, will be > easier :-) > > HTH! > Jo > _______________________________________________ > slf4j-user mailing list > slf4j-user at qos.ch > https://mailman.qos.ch/cgi-bin/mailman/listinfo/slf4j-user -- Ceki Gülcü Sponsoring SLF4J/logback/reload4j at https://github.com/sponsors/qos-ch From jo at durchholz.org Sun Feb 2 10:35:59 2025 From: jo at durchholz.org (jo at durchholz.org) Date: Sun, 2 Feb 2025 11:35:59 +0100 Subject: [slf4j-user] When more parameters than placeholders, extra parameters are not logged In-Reply-To: <1250089793.3587854.1737769097501@mail.yahoo.com> References: <1250089793.3587854.1737769097501.ref@mail.yahoo.com> <1250089793.3587854.1737769097501@mail.yahoo.com> Message-ID: <57db32e1-a3e4-490a-8217-d37b5f92b7f1@durchholz.org> An additional thought: Modern JVMs are pretty efficient at building closures. Log4j2 (not the old log4j version 1) is an alternative where you can replace logger.debug("foo is: " + foo) with logger.debug(() -> "foo is: " + foo) This has the same avoid-constructing-strings logic as logger.debug("foo is: {}", foo) but you don't need to refactor anything. The downside is that forgetting to create the closure will make logging slow, so in the end, you'll likely fall back to the same make-sure-everything-is-mass-refactored approach as before, so the overall advantage isn't big. However, you'll get low-hanging fruits faster, at the expense of having to try out a different logging framework. Which means I won't recommend anything, I'm just presenting more choices in the hopes that one of them matches your situation better. Regards, Jo From ceki at qos.ch Sun Feb 2 15:46:56 2025 From: ceki at qos.ch (Ceki Gulcu) Date: Sun, 2 Feb 2025 16:46:56 +0100 Subject: [slf4j-user] When more parameters than placeholders, extra parameters are not logged In-Reply-To: <57db32e1-a3e4-490a-8217-d37b5f92b7f1@durchholz.org> References: <1250089793.3587854.1737769097501.ref@mail.yahoo.com> <1250089793.3587854.1737769097501@mail.yahoo.com> <57db32e1-a3e4-490a-8217-d37b5f92b7f1@durchholz.org> Message-ID: Hi all, For your information, in addition to being efficient and clean, SLF4J's fluent API also allows setting the message via a supplier (closure). See https://www.slf4j.org/manual.html#fluent and also https://www.slf4j.org/apidocs/org/slf4j/spi/LoggingEventBuilder.html On 02/02/2025 11:35, Joachim Durchholz via slf4j-user wrote: > An additional thought: > > Modern JVMs are pretty efficient at building closures. > Log4j2 (not the old log4j version 1) is an alternative where you can > replace > >   logger.debug("foo is: " + foo) > > with > >   logger.debug(() -> "foo is: " + foo) > > This has the same avoid-constructing-strings logic as > >   logger.debug("foo is: {}", foo) > > but you don't need to refactor anything. > > The downside is that forgetting to create the closure will make logging > slow, so in the end, you'll likely fall back to the same make-sure- > everything-is-mass-refactored approach as before, so the overall > advantage isn't big. > However, you'll get low-hanging fruits faster, at the expense of having > to try out a different logging framework. > > Which means I won't recommend anything, I'm just presenting more choices > in the hopes that one of them matches your situation better. > > Regards, > Jo > _______________________________________________ > slf4j-user mailing list > slf4j-user at qos.ch > https://mailman.qos.ch/cgi-bin/mailman/listinfo/slf4j-user -- Ceki Gülcü Sponsoring SLF4J/logback/reload4j at https://github.com/sponsors/qos-ch