[logback-dev] What is the most efficient way - preferrably platform agnostic - to submit events from "the outside"?

[Ceki Gulcu]

Hello all,

Several fairly broad topics have been mentioned in this thread. One is
communication strategies which can deal with arbitrary network or host
failures. Another is the RPC mechanism, yet another is the question of
data encoding which prefarably would be easy-to-use,
language-agnostic, resistant to changes in the data (long term version
stability), fast, small and human-readable. Those who still beleive in
Santa would like to have all these properties at the same time.

Each of these topics is very broad on its own. I think that focusing
on a single topic at a time will improve the chances of a getting a
concrete result from the discussion.

Regarding plain old java serialization, I think it is more robust than
what many people think. As long as the serialVersionUID of the writer
(let's call it W), and the reader (call it R) are the same, java
serialization can handle the addition of field in W missing in R, as
well as missing fields in W added in R. In many respects java
serialization is more robust than XML, especially if you turn on DTD
validation in XML.

I found the object serialization examples in [1] and especially [2]
very useful.

[1] http://java.sun.com/j2se/1.4.2/docs/guide/serialization/examples/index.html
[2] 
http://java.sun.com/j2se/1.4.2/docs/guide/serialization/examples/evolveserial/index3.html

Logback takes advantage of the optimized way java serialization deals
with object references. Many references contained in a LoggingEvent (now
LoggingEventVO) are transferred by referece instead of by value. I was
under the impression that serialization of LoggingEvents was already
very fast. However, if we can significantly improve on the existing,
so much the better.

As there are certainly pros and cons for each serialization approach,
instead of the debate eventually degenerating into a religious
argument, we are likely to be better served by basing comparisons on
the same logging event data collection, which in the compression world
is called a "corpus". At present time, we do not have a logging event
corpus. Just as importantly, logback currently lacks a format for
storing the said corpus. Given that this corpus will serve as a
yardstick for a long time, and performance is not an issue, a human
readable text format such as XML seems like a reasonable choice.

Is anyone interested in providing a corpus?

Is anyone interested in writing a LoggingEventVO to XML and XML to
LoggingEvent converter(s)?

To answer some of Joern's questions, I would like to note that
logback-classic, and in particular ILoggingEvent, does not contain any
java serialization-specific methods. Serialization is now an appender
issue not handled by ILoggingEvent.

Regarding old events, I would very much like to come up with a
strategy for supporting "old" versions of serialized logging
events. For example, by having versioned serializer/deserializers,
e.g. LoggingEvent0916 and LoggingEvent0918.  It's a highly technical
issue which would need to be decided by actual implementation.

Having said that, defining a corpus seems to me as being the most
pressing issue at this time.

Once we settle on a corpus, we can more objectively debate the merits
of such and such serialization strategy.

More below.

Maarten Bosteels wrote:


> Sure, will send it to list tonight (haven't got access to it right now).
> 
> As I wrote earlier, I think protobuf (and Thrift) have some very cool 
> features:
> * you can update messages without breaking old code [a]
> * faster than XML
> * smaller than XML
> * binary, but with a convenient human-readable representation for 
> debugging and editing
> * language-neutral, platform-neutral

It looks to good to be true. Does Santa exist after all?

> [a] http://code.google.com/apis/protocolbuffers/docs/proto.html#updating
> 
> Maarten


-- 
Ceki Gülcü
Logback: The reliable, generic, fast and flexible logging framework for Java.
http://logback.qos.ch