[logback-dev] What is the most efficient way - preferrably platform agnostic - to submit events from "the outside"?

[Thorbjoern Ravn Andersen]

Joern Huxhorn skrev:
>
> On 28.02.2009, at 21:44, Thorbjoern Ravn Andersen wrote:
>>>> I am still pondering on a language agnostic receiver.   The reason 
>>>> for the XML being uninteresting was because it was much more 
>>>> verbose than the plain serialised byte object?
>>>
>>> I wouldn't call it uninteresting ;)
>>> It's just more expensive to create such events so I'd only use it if 
>>> I have to.
>> Are they?  How come?     Perhaps if using XMLEncoder instead of 
>> rolling your own :)
>
> I've done some more benchmarking concerning purely 
> serialization/deserialization without any disk I/O, i.e. using just 
> byte[].
>
> http://apps.sourceforge.net/trac/lilith/ticket/28
> the last table at the bottom.
>
> That's actually quite interesting... I didn't expect that using my own 
> XML Serializer + compression would actually result in smaller data 
> size than using java serialization + compression.
> Even uncompressed, my Serializer doesn't produce much more bytes than 
> java serialization.... I didn't expect that...
>
> However, creation and handling of XML is still much slower than pure 
> java serialization - which doesn't surprise me at all.
> My own implementation (using StAX) is a lot faster than the generic 
> java.beans.XML one, though.

I have seen your data and it is interesting.  If you want compression 
you need something lighter than the default setting of the gzip 
compressor, as even the lightest setting give reasonable results for 
repetitive data without using much time.

Why use StAX - you need a SAX parser or something more sophisticated?  I 
am unfamiliar with the project.

>>>> Would a sufficiently terse xml-dialect be interesting?  I was 
>>>> thinking of having one-character node names and attribute names? 
>>>> (and moving the namespace outside the fragments).
>>>
>>> I'm not sure about that. Thinking about a binary format would 
>>> probably be more worthwhile...
>> Binary formats are rather painful to extend at a later date.  Just 
>> see how hard it is even with help from the serialization modules.
>
> The main problem with java serialization is IMHO that it's not 
> possible to *somehow* load older versions of a class. If you change a 
> class and change it's serialVersionUID then you have no chance to load 
> any previously serialized objects. No chance at all. That's quite evil...
> To do something like this one would need to reimplement the class, 
> with a different name, and implement conversion from old to new.
>
Since the serialization mechanism in java relies on the exact ordering 
and type of each field in order to generate and interpret the byte 
stream without any additional information in the byte stream but the raw 
values, it makes sense to me that it will break if the class signature 
change.  It would not be hard to use an interface internally and let 
multiple versions of the class to serialize implement it.    It is not 
an evil, it is just a trade off :)


>>
>> You may remember that I am in a situation where our production 
>> servers are inaccessible and where I want our logs to be both humanly 
>> readable as well as reprocessable.    I would be very interested in 
>> defining a very terse XML dialect for this purpose, as Ceki has 
>> demonstrated earlier that my needs cannot be fullfilled by the log4j 
>> dtd.
>
> Have you had the time to check my xml format? While it's not exactly 
> terse ;) it's definitely human-readable.
I think I have understood what the problem with my use of the "humanly 
readable" term is.  It is not the actual XML with names and tags, but 
the data carried I am talking about.  Timestamps are fine but is not 
really readable to a human.

I can easily live with one character tag and attribute names if the data 
in the fields carry meaning to the human eye :)

>> I see a good need for a production strength "xml receiver -> sfl4j 
>> events" added to the slf4j-ext package (or so) - would it licensewise 
>> be possible to adapt your work into this?
>
> My code is licensed under LGPL v3 so this shouldn't be a problem. 
> Since I'm the only developer I could grant any license, anyway.
> I'm not sure what you mean with "slf4j events", though. You mean 
> "logback" instead of "slf4j", right?
No, in the above I actually mean slf4j events.    A receiver which 
accepts incoming events and throw them straight into a slf4j 
log.debug(....) statement (or whatever level was used) - which to me 
would be a generic way to glue two separate platforms together.


-- 
  Thorbjørn Ravn Andersen  "...plus... Tubular Bells!"