[slf4j-dev] Structured data "was Plan for SLF4J 2.0"

[Ceki Gülcü]

On 11/03/2010 2:21 AM, Ralph Goers wrote:
>
> On Mar 10, 2010, at 12:46 AM, Ceki Gülcü wrote:
>
>> On 10/03/2010 2:41 AM, Ralph Goers wrote:
>>> I am extremely tired of this discussion.
>>
>> I am sorry to hear that. Do you feel that your arguments are not being
>> heard properly? Or do you think that the matter under discussion
>> admits such an obvious solution that it does not merit debate?

 > Yes. It is obvious if you've tried to do what you are proposing. It
 > sounds very similar to the mess I had to deal with with EventData,
 > except with EventData it can't be passed as a parameter since it is
 > passed through a LocationAwareLogger, which is a point you keep
 > ignoring.

Yes, unfortunately you can't pass parameters to LocationAwareLogger's
log method which is a major impediment to wrapping intended as
syntactic sugar.
>>
>>> On Mar 9, 2010, at 10:08 AM, Ceki Gülcü wrote:
>>
>>>> Let's say you have a parameter of type 'House' you would like to log
>>>> and you wrap it inside a new type called StructuredDataHouse and pass
>>>> it to a logger as the first parameter (the message being "{}").
>>>
>>> You can't do that on a LocationAwareLogger so this is impossible
>>> with any Logger implementation based on LoggerWrapper.
>>
>> It's not the logger which wraps House but the user. The location of
>> the logger call remains unchanged by the wrapping which is there for
>> encoding purposes only. Here is an example:
>>
>>   StructuredDataHouse  sdh = new StructuredDataHouse(house);
>>   logger.info("{}", sdh);
>>
>>
>> If this encoding thing catches on, we could dispense with the wrapping
>> thing altogether. You could register an RFC5424 "subencoder" for the
>> House type and the RFC5424Encoder would look it up at runtime.
>>
>> So you could just write:
>>
>>   logger.info("{}", house);
>
>
> There is a big difference between an interface of
>
> info(String msg, Object param);
>
> and
>
> info(Message msg)
>
> although the code is similar:
>
> logger.info(new HouseMessage(house));
>
> the way it is processed in Logback is much, much different and a lot simpler.
>
>>
>> The RFC5424 encoder and the transformer for House would just output the correct information.
>>
>>>> The
>>>> RFC5424Encoder detects that this parameter supports RFC5424 encoding
>>>> and asks StructuredDataHouse for its RFC5424 encoded
>>>> data. RFC5424Encoder only neeeds to deals with objects supporting
>>>> RFC5424 encoding, there is no need to anticipate other encoding types.
>>
>>> 1. You can have a bunch of parameters. The encoder has to check
>>> every one of them.
>>
>> Yes, but that's just iterating over the parameters.
>
> And having to have special logic to interpret them. What will it do with Objects that it doesn't understand? Probably just call toString() which may result in garbage from a lot of objects. With a Message you are guaranteed that it will generate something meaningful because that is the contract. The contract with Object is less than helpful.
>
>>
>>> 2. I guess you'd also have to be able to configure a whole list of
>>> encoders and run through all of them to make sure each of your
>>> parameters was formatted correctly, even if the message doesn't
>>> contain data matching any of them.


 > > Well, the encoder is unique per appender. However, as discussed above
 > > there might be subencoder specific for each type you care about. For
 > > types without sub-encoders, some default heuristic would be applied which
 > > I already mentioned.

 > Exactly. This is where it turns into a complicated nightmare. Encoders
 > referencing sub-encoders all having to be managed in the
 > configuration, and each sub-encoder would have to be called to see if
 > it understands each Object. No configuration at all is required for a
 > Message to render itself. The contract could probably be enhanced a
 > little bit to pass getFormattedMessage a bit of information about the
 > Layout or Appender so that it can have a bit of variety in rendering
 > itself, but it should always render something meaningful.

We already have heuristics for encoding an arbitrary object. During
serialization message parameters are transformed into
string. Similarly, Log4jXMLLayout transforms message parameters to
string as well.

There are several low-level encoding schemes worth mentioning, namely
text, XML, object serialization, protobuf and RFC5424. When encoding
an event in XML the natural inclination is to encode parameters in XML
as well. During serialization of an event using serialization for
message parameters is equally natural. The same goes for RFC 5424 and
protobuf.

I am speculating but are you assuming that encoding message parameters
in XML is always appropriate (at least as a nice fallback)? If true,
you can live with a single transformation method provided by the
Message interface. However, the problem gets really interesting if you
add the requirement to be able to retrieve the original message
parameter which opens up a whole bunch of new possibilities.

>>>> The end result is very similar to asking your StructuredDataMessage in
>>>> the org.slf4j.message package for its formatted message, except that
>>>> the question is asked by a RFC5424Encoder. A different encoder would
>>>> ask a different question.
>>
>>> This is not similar at all. The layout/encoder/whatever calls
>>> getFormattedMessage and gets an appropriate response. Since everything
>>> is a Message the method is always there.
>>
>> How is this different than toString()? Everything is an object and the
>> toString() method is always there.

 > Yes. But it often doesn't do what you want.  See above for the
 > rest. Encoders may be useful doing things like compressing the data
 > stream, but trying to get fancy like this is just a horrible idea.

You may be right but I think the idea is worth trying/experimenting with.

 > By the way, did you update the doc on the site? I don't see any
 > reference to Encoders and had to look at the code to see that they are
 > currently only used in classes extending OutputStreamAppender. So I
 > guess a SocketAppender or SyslogAppender currently can't have an
 > encoder.

Encoders are new in 0.9.19, the docs on the site will be updated when
0.9.19 comes out. They have been updated in the git repo.

--
Ceki