[logback-user] Log events delivery guarantee.

[Ralph Goers]

I created the FlumeAppender for Log4j with just this purpose in mind. The FlumeAppender will write the log event to local disk and then returns control to the application. At this point eventual delivery is guaranteed. A background thread reads the events that have been written to disk and forwards them on to another Apache Flume node. When that node confirms it has accepted them the event is deleted from local disk. The FlumeAppender has the ability to fail over to alternate Flume nodes. If none are available the events will simply stay on disk until it is full.

Ralph

> On Jun 2, 2017, at 10:01 AM, Oleksandr Gavenko <gavenkoa at gmail.com> wrote:
> 
> I made search in a list and only found one relevant message:
> 
> http://markmail.org/message/tdkr745eqf3vxqme
> 
> From Michael Reinhold, Mar 19, 2014 5:45:23 pm:
>> The method described for a disk-cached AsyncAppender of course
>> makes perfect sense. I think it would make a good alternative to
>> the current AsyncAppender for some scenarios and would be a solid
>> addition to the current Logback appenders.
> 
> I am going to implement log collection into ElasticSearch and I would
> like to achieve high level of delivery reliability. Not in sense of
> latency but in sense of completeness.
> 
> Business processes operate on collected data and nothing should be
> lost if remote log collector is down or my application
> crashed/restarted.
> 
> Only local disk space the most safest place for write due to low
> latency, persistence and availability (except rare cases of FS
> corruption or disk is full error).
> 
> Network services definitely have downtime.
> 
> =========================
> 
> Is there persistence wrapper for log events to guaranty delivery at
> later time if remote service is not available? Like AsyncAppender for
> Appender?
> 
> Docs https://logback.qos.ch/manual/appenders.html doesn't show
> possibility for error:
> 
> public interface Appender<E> extends LifeCycle, ContextAware, FilterAttachable {
>  public String getName();
>  public void setName(String name);
>  void doAppend(E event);
> }
> 
> but in sources I see:
> 
>    void doAppend(E event) throws LogbackException;
> 
> Potentially it is possible to say that service unavailable via
> LogbackException subclass and start collect data into memory buffer
> until some limit, then fallback to disk media.
> 
> And try to send events to server with some defined pattern until
> RecipientDownException is gone.
> 
> On application startup PersistentAppender should check for serialized
> unpublished events and try to send them again.
> 
> As I see in https://logback.qos.ch/apidocs/ch/qos/logback/classic/spi/ILoggingEvent.html
> persister should store:
> 
> * timestamp
> * level
> * msg
> * marker (not sure is full hierarchy is needed as market is just a string)
> * MCD
> * StackTraceElement[] (this information can be useful but require a lot of CPU)
> 
> Some get* method are strange, like getArgumentArray(), I am not sure
> if this information is useful.
> 
> ======================
> 
> Alternative approach is to write log and later parse it. Like this
> done with Filebeat https://www.elastic.co/products/beats/filebeat that
> complement to ElasticSearch.
> 
> The problem with such solution is necessity to invent or follow file
> format. Formatting log with MDC and Markers that can be tricky.
> 
> ======================
> 
> Direct writer to ElasticSearch
> https://github.com/internetitem/logback-elasticsearch-appender buffer
> messages only in memory if ElasticSearch isn't available and
> application should be restarted everything is lost.
> _______________________________________________
> logback-user mailing list
> logback-user at qos.ch
> http://mailman.qos.ch/mailman/listinfo/logback-user