[logback-dev] What is the most efficient way - preferrably platform agnostic - to submit events from "the outside"?

[Joern Huxhorn]

On 01.03.2009, at 22:18, Thorbjoern Ravn Andersen wrote:

> Joern Huxhorn skrev:
>>
>>> Why use StAX - you need a SAX parser or something more  
>>> sophisticated?  I am unfamiliar with the project.
>>
>> Well, I just like the API.
>> See http://www.developer.com/xml/article.php/3397691 for a  
>> comparison. While I don't use the skip-feature right now, I like to  
>> have that option. I dislike everything that's using a DOM because  
>> has to read the complete XML and keep it in memory... all in all,  
>> it's just a matter of personal preference... ;)
>
> FIne with me.  DOM's don't scale well, and one of the tricks to  
> speed up XSLT's is to figure out when stuff can be done (before  
> readign all the data in) which is a science in itself.
>

It's also impossible to read an invalid XML if you are using DOM. I'd  
like to be able to read an XML file which is only missing the last  
closing tag, for example.

>
>>> Since the serialization mechanism in java relies on the exact  
>>> ordering and type of each field in order to generate and interpret  
>>> the byte stream without any additional information in the byte  
>>> stream but the raw values, it makes sense to me that it will break  
>>> if the class signature change.  It would not be hard to use an  
>>> interface internally and let multiple versions of the class to  
>>> serialize implement it.    It is not an evil, it is just a trade  
>>> off :)
>>
>> Yes, I understand the reasons for the situation but it doesn't make  
>> it easy to read serialized data of previous versions of the class.
> Then you need customized serializer and deserializer code, and a  
> version tag in the stream.
>
>
>>
>> That way, the whole logic of transforming a given object to an  
>> arbitrary byte array is entirely detached from the class itself.
>> It would also remove the need that LoggingEvent is aware of  
>> persisting at all. The responsibility is entirely in the Serializer/ 
>> Deserializer implementation, i.e. there's no need for  
>> getLoggerContextVO or similar methods.
>>
> It is still a painpoint.  But the version number in the data stream  
> is no worse than the serialID in the stream anyway :)

Well, a stream could contain both LoggingEvent0916 and  
LoggingEvent0918 and the Deserializer would produce the correct output  
using instanceof and some defined conversion. The point is that the  
old class would not be lost as it is the case if just the  
serialVersionUID is changed. It could therefore still be deserialized.

>
>>>> Have you had the time to check my xml format? While it's not  
>>>> exactly terse ;) it's definitely human-readable.
>>> I think I have understood what the problem with my use of the  
>>> "humanly readable" term is.  It is not the actual XML with names  
>>> and tags, but the data carried I am talking about.  Timestamps are  
>>> fine but is not really readable to a human.
>>>
>>> I can easily live with one character tag and attribute names if  
>>> the data in the fields carry meaning to the human eye :)
>>
>> Well, I'm using yyyy-MM-dd'T'HH:mm:ss.SSSZ as the primary time  
>> stamp format and then apply some magic to change it into a valid  
>> xml timestamp, i.e. I add a colon between hh and mm of the timezone.
>> It mad my sigh quite a bit when I realized that SimpleDateFormat  
>> did not support this...
>>
> A valid xml timestamp?  According to what dialect?  SOAP?   Frankly  
> I think that these date objects are expensive to reconstruct -  
> perhaps you should look into this if performance is an issue to  
> you?  I believe the date object just serializes  the long datestamp  
> which is much cheaper to deserialize than to deconstruct a string  
> into pieces and build a date/calendar object from it.  Note you also  
> need timestamp information for this so it most likely goes through a  
> Calendar.
>

Point taken, I wasn't really thinking about that when I was  
implementing it, I primarily wanted to do it *right* ;)

It's an xs:dateTime (http://www.w3.org/TR/xmlschema-2/#dateTime) which  
seemed reasonable to me.
xs:dateTime expects a colon between timezone hh and mm while  
SimpleDateFormat does only support hhmm :p

I implemented your suggestion and it's - unsurprisingly -  
deserializing quite a bit faster, now.
By default, I write both timeStamp (formatted) and timeStampMillis but  
I only evaluate timeStampMillis if it's available and a long,  
otherwise I evaluate timeStamp as before.

Now I just need to update the schema... and I really need to implement  
an EventWrapper Serializer that's using my xml so I can use it as the  
Lilith file format...! Performance and size is absolutely acceptable.

I moved the benchmarks to http://apps.sourceforge.net/trac/lilith/wiki/SerializationPerformance 
  so I don't have to change the closed bug all the time :p

>>>
>>>>> I see a good need for a production strength "xml receiver ->  
>>>>> sfl4j events" added to the slf4j-ext package (or so) - would it  
>>>>> licensewise be possible to adapt your work into this?
>>>>
>>>> My code is licensed under LGPL v3 so this shouldn't be a problem.  
>>>> Since I'm the only developer I could grant any license, anyway.
>>>> I'm not sure what you mean with "slf4j events", though. You mean  
>>>> "logback" instead of "slf4j", right?
>>> No, in the above I actually mean slf4j events.    A receiver which  
>>> accepts incoming events and throw them straight into a slf4j  
>>> log.debug(....) statement (or whatever level was used) - which to  
>>> me would be a generic way to glue two separate platforms together.
>>
>> But slf4j doesn't really define the events, does it?
> You have a point there.  It doesn't.  It is the logging backend that  
> does :)
>
> Sigh.  I just hoped for a low entry solution here.  Back to the  
> brainstorming tank.

While such a bridge would certainly have it's use case, quite some  
information would be lost while tunneling the events if they are  
tunneled using the logger interface. Thread, code location etc...

Joern.