[slf4j-dev] MDC Type functionality
John E. Conlon
jconlon at verticon.com
Tue Mar 27 23:51:32 CEST 2007
Gardiner, Paul wrote:
> -----Original Message-----
> From: dev-bounces at slf4j.org [mailto:dev-bounces at slf4j.org] On Behalf Of
> John E. Conlon
> Sent: Tuesday, March 27, 2007 2:39 PM
> To: slf4j developers list
> Subject: Re: [slf4j-dev] MDC Type functionality
>
> Paul,
>
> Gardiner, Paul wrote:
>
>> John,
>>
>> What you have in the repo looks like an implementation of an OSGi log
>> service that uses SLF4j to do the logging. What I have is a bit
>> different. I'll explain in a bit more detail.
>>
>> The OSGi log service I am using is the Equinox extended log service.
>> The SLF4J implementation I did (the "log user") is a separate bundle
>> that uses the OSGi log service; it doesn't provide any services
>>
> itself.
>
>> Instead, it writes out log entries to the OSGi log service. I use the
>> slf4j.api bundle, and then export org.slf4j.impl, and my logger
>>
> package
>
>> from my bundle, which the slf4j.api bundle then imports.
>>
>>
>
>
> So your 'log user' might also be called in our slf4j naming conventions
> a: slf4j-osgi.
> Bundle wiring (with the symbol --> meaning uses some package or service
> from another bundle) so far looks like:
> [PAG] By "-->", I was just intending to show the flow of the logged
> message, not a dependency.
>
> originatingBundle(s) --> slf4j-api --> your 'log user'(aka slf4j-osgi)
>
> Right?
> [PAG] Yes, assuming that the "slf4j-osgi" is not the one in the repo,
> because that one is an OSGi log service, not a user of the log service.
> My slf4j implementation is a user of the service.
>
That is correct.
>
>> Then I have a completely separate bundle (the "log reader user") that
>> registers a listener with the OSGi log service. Every time a log
>>
> entry
>
>> is made by the log user, the log reader user's filter method gets
>>
> called
>
>> (isLoggable), and if it returns true, its logged method gets called,
>> which is where the logs get written out to their final destination.
>>
> Paul, I have a rough idea of what your doing but I maybe hossed up
> somewhere - if from the following you see that I am still confused
> please send me a list of the package imports and exports from each of
> your bundles and also note which services are used by each of the
> bundles.
> [PAG] Would it be easier to send the whole workspace? It's not too big.
>
>
Yes you can just send directly to my email address jconlon at verticon.com
and not copy the list.
- thanks,
John
>> The
>> logged method receives an ExtendedLogEntry object as its argument,
>>
> which
>
>> in addition to the stuff inherited from LogEntry, contains the logger
>> name, thread id, a sequence number, and a context object, which can be
>> any object defined by the log user. Currently, I have the bundle and
>> FQCN included in this object. Note that the bundle is the bundle that
>> made the original log entry, not the log user bundle. My log reader
>> bundle is using log4j to log out the messages.
>>
>> So all together, there are three components - log user bundle (slf4j),
>> the OSGi extended log service (in Equinox incubator), and the log
>>
> reader
>
>> user (currently I'm using log4j). When making a log entry, the path
>>
> the
>
>> log message takes is ->originating bundle->log user->OSGi logger->log
>> reader user.
>>
>>
> 1. originatingBundle(s) --> 2. slf4j-api bundle--> 3. your 'log
> user'(aka slf4j-osgi) bundle --> 4. OSGi logger(LogService) bundle--> 5.
>
> 'log reader user' bundle--> 6. log4j
> [PAG] Yes, that looks right on.
>
> I think I got it so far. So how are you using the log4j(6)? Or in
> other words how does your 'log reader user' bundle (5) use the log4j (6)
>
> packages, does it import them from a log4j bundle or are they privately
> packaged by you into your 'log reader user' (5) bundle?
> [PAG] In the case of log4j, it has a Require-Bundle entry in the
> manifest. Because the "slf4j-osgi" bundle and the logback bundle both
> use slf4j, I had to place the logback and slf4j jars inside the logback
> reader bundle, otherwise I would go into an endless loop.
>
>
>> The advantage to this approach is that it is reasonably efficient,
>> allows users to use LoggerFactory.getLogger(getClass())... and still
>> get bundle and other context information without having to use a
>>
> service
>
>> reference or bundle as an argument, it allows for different listener
>> types, and it doesn't break existing OSGi log users/log reader users.
>>
>>
> Native LogService client bundles would then be wired like:
> 3*. LogService client bundles -->4. OSGi logger(LogService) bundle--> 5.
>
> 'log reader user' bundle--> 6. log4j
>
> Top end sounds good - both Equinox LogService clients (3*) and slf4j
> clients (1) are supported.
>
>> So to answer your question about which implementation, the answer is
>> it's an OSGi log user implementation.
>>
> OSGi logging (like many other OSGi services) can be split up (or
> combined) in many different ways. So from slf4j perspective it's impl
> is your OSGi 'log user'(3) . From the perspective of the actual
> outputing of log messages to the world the 'logging impl' would be
> log4j(6). Sorry for mixing up terms in previous email.
>
>> I implemented it using a
>> LocationAwareLogger, but nothing is being done with the marker data at
>> the moment, because I can't use logback as the log reader user.
>>
> So your 'log user' (3) is implemented with a LocationAwareLogger which
> means you have the information you want at (3) and you want to
> (obliviously) get it to (6).
>
> the path again...
> 3. 'log user'(aka slf4j-osgi) bundle --> 4. OSGi logger(LogService)
> bundle--> 5. 'log reader user' bundle--> 6. log4j
>
> So if you want your logging messages from originatingBundles that use
> slf4j apis to use this path
> 1. originatingBundle(s) --> 2. slf4j-api bundle--> 3. your 'log
> user'(aka slf4j-osgi) bundle --> 4. OSGi logger(LogService) bundle
>
> and you want to retain context information then you would have to map
> the Marker information you have at 'log user' (3) to something that the
> LogService (4) can use passing that through to (5). You would then have
>
> to pull out this information again at (5) and pass it to a logger(6) for
>
> output. Sounds messy.
> [PAG] Yes. I had to serialize the marker to a byte array and
> deserialize it on the other side. Yes, kind of messy.
>
> Why not implement the slf4j at the 'log reader user' bundle (5)?
> [PAG] A couple of reasons. First, the way it's implemented now, all a
> programmer has to do is import the slf4j-api bundle, and then use it.
> They don't have to worry about context, because its all taken care of.
> Second, the OSGi api without the extensions has no way of getting
> context through to the reader where no service reference is used. The
> objective here is to provide a better logging api while still using the
> OSGi log service, and provide implementation options on the reader side.
>
>> The
>> reason for this is that I already have a binding to OSGi, so if I then
>> bind to logback, I have a classloader issue. This should be ok if I
>> completely segregate the bundles, but unfortunately I can't do that,
>> because the Marker object comes from the log user bundle, which is
>> different than the Marker object from the log reader user bundle.
>>
>>
> The Marker interface and the BasicMarker are exported from the slf4j-api
>
> bundle.
>
>> Having said that, I'm not really asking for ideas on this, unless you
>> have ideas off the top of your head. I really haven't looked into
>>
> this
>
>> very much. I tried implementing logback, it got lots of class loader
>> errors, so I put log4j back.
>>
> Your use case is rather generic so I expect this will come up again in
> the future so trying to get a handle on it now is a good thing.
> (Actually had it on my todo list to address this.) ;-)
>
> Did you make a logback bundle?
> [PAG] Yes. It seems to be working very well, and the markers work fine
> also.
>
> Alternative suggestion -
> An OSGi bundle implementation of a logback based slf4j binding that I
> will call here 'slf4j&osgireader-logback' that also listens to the
> OSGi logService (5*). That replaces the 'log reader user' bundle (5)
> the log user (3) as well.
>
> slf4j clients now do:
> 1. originatingBundle(s) --> 2. slf4j-api bundle--> 5*.
> slf4j&osgireader-logback bundle
>
> and LogService clients now do:
>
> 3*. LogService client bundles -->4. OSGi logger(LogService) bundle-->
> 5*.slf4j&osgireader-logback bundle
>
> WDYT?
> John
>
>
>> Also, since both the log user and log
>> reader user use SLF4J (since logback uses it), I end up getting an
>> endless loop. Once I have looked a bit deeper, I'll pose some
>> questions, but for now, I just don't understand what's going on
>>
> enough.
>
>> I'm pretty sure the class loader issues are solvable though.
>>
>> As far as MDC goes, I think I'll just drop it. I likely can get the
>>
> job
>
>> done with Markers, but I'll have to learn a bit more about how they
>> work. I understand how to use them, but need to learn how to use them
>> properly and appropriately.
>>
>> Thanks,
>> Paul
>>
>> -----Original Message-----
>> From: dev-bounces at slf4j.org [mailto:dev-bounces at slf4j.org] On Behalf
>>
> Of
>
>> John E. Conlon
>> Sent: Monday, March 26, 2007 5:26 PM
>> To: slf4j developers list
>> Subject: Re: [slf4j-dev] MDC Type functionality
>>
>> Gardiner, Paul wrote:
>>
>>
>>> The extended log service in equinox is similar to the standard one,
>>>
>>>
>> but
>>
>>
>>> it adds a few extra methods for supporting named loggers, isLoggable,
>>> and logging context objects. Here's the basic idea:
>>> https://www.eclipse.org/bugs/show_bug.cgi?id=147824. It's just an
>>> incubator project right now, but I have a pressing need at the
>>>
> moment.
>
>>>
>>>
>>>
>> Interesting thread... I think our devs and users may find it most
>> interesting as slf4j is mentioned several times: :-)
>>
>> For example:
>>
>>
>>
>>> Message payload for the LogService consistes primarily of String
>>>
> based
>
>>>
>>>
>> messages
>>
>>
>>> and when relevant a Throwable. In some situations a generic
>>>
>>>
>> String-based
>>
>>
>>> logging API is insufficient and in these cases it's desirable to pass
>>>
>>>
>> additonal
>>
>>
>>> information with the understanding that it may be ignored. SLF4J uses
>>>
>>>
>> it's
>>
>>
>>> Marker API in this fashion to allow strong coupling between logger
>>>
> and
>
>>> recipient without impacting the generic log API.
>>>
>>>
>> ...
>>
>>
>>
>>> For this to work there are a couple of problems that need thought:
>>> 1) we need a binding of the various logging API to the log service
>>> - we should consider looking at SLF4J as its defined a few useful
>>>
>>>
>> static
>>
>>
>>> bindings already for JCL and log4j.
>>>
>>>
>> ...
>>
>>
>>
>>> I'm step by step migrating to SLF4J, because I really enjoy the
>>>
>>>
>> simplicity and
>>
>>
>>> effectiveness of the API. Problem is: I NEED to know which OSGi
>>>
> Plugin
>
>>>
>>>
>> is
>>
>>
>>> logging. This is a serious problem, as all implementations only
>>>
>>>
>> provide a
>>
>>
>>> thread based context.
>>>
>>>
>> Note: The osgi-over-slf4j binding in the repo already provides this
>> information as BundleSymbolicName and Version.
>>
>>
>>
>>
>>
>>
>>
>>> I'm new to SLF4J, but I have looked around at markers, and found
>>>
> there
>
>>> isn't a lot of info on how they are used.
>>>
>>>
>> Logger logger = LoggerFactory.getLogger("testMarker");
>> Marker blue = MarkerFactory.getMarker("BLUE");
>> logger.debug(blue, "hello");
>> logger.info(blue, "hello");
>> logger.warn(blue, "hello");
>> logger.error(blue, "hello");
>>
>> Take a look at the test cases in slf4j.
>>
>>
>>> It looks like a single piece
>>> of data, but with a hierarchy.
>>>
>>>
>> Yes.
>>
>>
>>> How would it work in this case? Would I
>>> (or the end user) create a marker named "session_id", and then create
>>>
>>>
>> a
>>
>>
>>> child with the actual session id as the name?
>>>
>>>
>>>
>>>
>> Well from what I gather about the Equinox extension is that there is a
>>
>
>
>> context associated with logging messages.
>> In your use case the user or client application is an Equinox Log
>> Service client, not a slf4j api user right? That would mean YOU would
>> have to create the Marker from the context in the log messages in your
>>
>
>
>> 'equinox-over-slf4j'.
>>
>> Now the critical question is which Logging API implementation are you
>> planning to have your 'equinox-over-slf4j' bind with? If you use
>>
> Markers
>
>> you will have to use an implementation that uses them too and that
>>
> would
>
>> be either one that you implement or the Logback implementation. Check
>> the logback project for more details on how it handles marker
>> hierarchies. http://logback.qos.ch/
>>
>> cheers,
>> John
>>
>>
>>
>>> -----Original Message-----
>>> From: dev-bounces at slf4j.org [mailto:dev-bounces at slf4j.org] On Behalf
>>>
>>>
>> Of
>>
>>
>>> John E. Conlon
>>> Sent: Monday, March 26, 2007 12:55 PM
>>> To: slf4j developers list
>>> Subject: Re: [slf4j-dev] MDC Type functionality
>>>
>>> Hello Paul,
>>>
>>> Have you considered using the
>>>
>>>
>>> org.slf4j.Marker
>>>
>>> to move this data to the readers? See the FAQ
>>> http://www.slf4j.org/faq.html#marker_interface
>>>
>>> BTW - You may have noticed in our slf4j source repository we have
>>> implemented a simple osgi log service for slf4j?
>>> http://svn.slf4j.org/viewvc/slf4j/trunk/osgi-over-slf4j/
>>>
>>> I have not worked with the Equinox log service but plan to do so
>>>
> soon.
>
>>>
>>>
>>
>>
>>> Paul would you be so kind to please provide a link to the
>>>
>>>
>> documentation
>>
>>
>>> that can describe the Equinox extensions? Is this log service the
>>>
> same
>
>>>
>>>
>>
>>
>>> used by Eclipse as well?
>>>
>>> thanks,
>>> John
>>>
>>>
>>> Gardiner, Paul wrote:
>>>
>>>
>>>
>>>> I have written an SLF4J binding to the Equinox extended log service,
>>>>
>
>
>>>> which is an extension of the OSGi logging service, that includes
>>>> contextual information, and passes "isLoggable" statements through
>>>>
> to
>
>>>>
>>>>
>>
>>
>>>> log listeners (readers). The problem I have is supporting extra data
>>>>
>
>
>>>> that is not part of the log message. SLF4J does not include MDC or
>>>>
>>>>
>> NDC
>>
>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>> functionality, so I am not sure of the best way to pass through this
>>>>
>
>
>>>> kind of data. A typical use case is session id, which is passed as
>>>> MDC. When the log entry is made, the session id is included in the
>>>> file appender, but omitted from an appender that sends pages. One
>>>> solution is to read all MDC data and pass it through to the OSGi log
>>>>
>
>
>>>> service, and then recreate it in the log reader. However, I was
>>>> wondering if there was a more elegant/less expensive solution.
>>>>
>>>> Thanks,
>>>>
>>>> Paul
>>>>
>>>>
>>>>
>>>>
>>>>
> ------------------------------------------------------------------------
>
>>
>>
>>>
>>>
>>>
>>>> _______________________________________________
>>>> dev mailing list
>>>> dev at slf4j.org
>>>> http://www.slf4j.org/mailman/listinfo/dev
>>>>
>>>>
>>>>
>>> _______________________________________________
>>> dev mailing list
>>> dev at slf4j.org
>>> http://www.slf4j.org/mailman/listinfo/dev
>>> _______________________________________________
>>> dev mailing list
>>> dev at slf4j.org
>>> http://www.slf4j.org/mailman/listinfo/dev
>>>
>>>
>>>
>>>
>>>
>> _______________________________________________
>> dev mailing list
>> dev at slf4j.org
>> http://www.slf4j.org/mailman/listinfo/dev
>> _______________________________________________
>> dev mailing list
>> dev at slf4j.org
>> http://www.slf4j.org/mailman/listinfo/dev
>>
>>
>>
>>
>
> _______________________________________________
> dev mailing list
> dev at slf4j.org
> http://www.slf4j.org/mailman/listinfo/dev
> _______________________________________________
> dev mailing list
> dev at slf4j.org
> http://www.slf4j.org/mailman/listinfo/dev
>
>
>
More information about the slf4j-dev
mailing list