[slf4j-user] hi and a basic question

[Hendrick, James]

Thanks for the quick response. I now see that this is an API layer, and that it requires a logging library "underneath"/

Just to be clear about what I am trying to accomplish, and give a few more details:

- We use a central logging service (currently splunk) and I need to simplify log collection to that service with minimal "handoff" from app to OS team.

- Operating system logs send syslog natively to a log-server "layer" which then routes to our splunk forwarders (and wherever else need a log)

- Applications currently mostly log to individual files on each server 
  This has several disadvantages, including:
     Operating system syslogs still need to be handled by a syslog daemon.
     Accessing local application and system logs pushes people to request local OS account access to develop / debug code.
     Using the central logging system to pick them up requires another agent on each server (e.g. local splunk forwarder)
     Using the system logger to pick them up (have syslog-ng use each file as a "source") also requires added work (changing the syslog config for every application)
     Apps writing local files requires additional pieces to do things like rotate / delete log files, etc.
     Performance suffers since writing and reading each log event has to wait for I/O *twice* before it can be forwarded on the network

- I want to have everything log to the kernel system logging facility so that the standard daemon (rsyslog, syslog-ng, etc.) simply picks up the event stream
  I see these as advantages:
      System images can be essentially "static" (whether in a data center or as a public cloud approved image) and independent of application configurations
      Application developers are free to modify what they log without having to engage system support teams to get the logs collected.
      One local process (syslog) can handle both system and application logs.
      No local "logrotate" is really necessary, or is at least greatly simplified.
      I/O is eliminated as a potential bottleneck since logs are passed from process to process and then onto the network.
      Choice of central logging system is abstracted away (e.g. switching from arcsight to splunk to sumo to elasticsearch does not change system or application configurations)

I think we can do what I am looking for within any of the logging libraries / frameworks (like logback, log4j, etc.) 
I was thinking (naively) that slog4j would directly make this easier.

Thanks for the help - If anyone is interested / has suggestions / wants to discuss further - I am happy to.

If not - I'll sit here and listen quietly for a while :-)

Thanks again!
Jim




>-----Original Message-----
>From: slf4j-user [mailto:slf4j-user-bounces at qos.ch] On Behalf Of Joachim Durchholz
>Sent: Tuesday, January 31, 2017 1:17 AM
>To: slf4j-user at qos.ch
>Subject: Re: [slf4j-user] hi and a basic question
>
>Am 30.01.2017 um 22:46 schrieb Hendrick, James:
>> Sorry if this is covered somewhere. I searched back a couple months and didn't see
>anything.
>>
>> How does the API support sending to a local system syslog service?
>
>slf4j does not, it is just a uniform API for various logging libraries.
>
>> Specifically I do *not* want this to send to a server over the network (e.g. udp/514)
>but to send to the local kernel logger.
>
>The easiest way would be to use syslog on "localhost". Network stacks
>are optimized to discover that case, the bytes never get sent out.
>
>It would be slightly better to directly use the kernel's logging API,
>but I guess somebody would have to write a Java interface to that using
>JNI or JNA.
>
>> With that, how do I as a developer have control over things in the syslog header and
>message body?
>>
>> It looks like: logger.debug("This is my application syslog message part")
>> Would generate a Severity debug message - but I do not easily see where to set the
>rest of the header fields.
>>
>> For example, if I want APP-NAME (in the syslog header) to use a specific string -
>where is that set?
>> Also for things like PROCID, MSGID, PRI etc.
>
>I tried something similar, a few years ago.
>
>The most important challenge is to make sure that this information gets
>logged even for log messages from 3rd-party libraries. I.e. you need
>some kind of global variable because you can't send the data through the
>API.
>The downside is that you have to be pretty careful whenever the software
>starts working on a new task (you usually have a sessionid or something
>similar that you want to log).
>In typical applications, you need a thread-local static variable to keep
>that information, with all the fun associated with passing the data on
>to child processes. You can put the data in the MDC, which has this
>specific ground covered, but insists that all data be stored as Strings.
>The other downside is that you are providing data that's specific to a
>particular appender, and you need to do extra configuration to make that
>data visible if you use a different appender.
>
>> I want to have different groups of developers separate their logs using the syslog
>standard fields, and then use the standard syslog daemon to forward these to our
>central logging service.
>
>The alternative would be to log to a local file, and have a background
>process watch the local log files and send newly appended lines to the
>central syslog. (I have seen this done using Splunk. I hear Splunk is
>pretty expensive, so you might want to investigate alternatives; more
>likely, you already have something installed and are supposed to use that.)
>The advantage is that you can infer information like app-name or
>developer from the name of the directory the file is located in, so you
>may be able to avoid using the MDC entirely.
>_______________________________________________
>slf4j-user mailing list
>slf4j-user at qos.ch
>http://mailman.qos.ch/mailman/listinfo/slf4j-user