[logback-user] Using Logback with RabbitMQ
Ciprian Dorin Craciun
ciprian.craciun at gmail.com
Sat Oct 20 20:19:43 CEST 2012
On Fri, Oct 19, 2012 at 12:32 AM, Smith, Larry (ECS - Enterprise Cloud
Service) <larry.smith5 at hp.com> wrote:
> Hi all,
> Tried looking this up on the net, but there is nothing recent. I would like
> to use an appender (standard or custom) that could interface with RabbitMQ.
> I have a need to see logging data in real time, but cannot use the console
> nor a database.
> I would assume that a custom appender can be written to communicate with the
> RabbitMQ exchange, and I would also like to be able to disable the appender
> when it is not required.
> Any ideas on how to begin this?
Glad to hear someone is interested in such a thing, that is to use
RabbitMQ as a Logback transport system.
There is such a solution, open sourced under Apache 2 license,
composed of both a Logback appender, and then a client that allows you
to fetch the events. (Disclaimer: I'm the code's author.) :)
(and my own repository: https://github.com/cipriancraciun/logging-java )
The architecture is very simple:
* for the publisher side, you just add in your dependencies the
AMQP publisher, plus some custom configuration inside `logback.xml`;
(see the below link for a `logback.xml` example;)
* for the client side I've borrowed the idea from other networked
publishers from the logback code, that is I consume the events from
the RabbitMQ server, and I push them back into the logback system,
thus you can configure their routing as normal towards files,
database, etc. on the collector side; again all you need to do is add
something inside the `logback.xml` file like in the example below:
to run the consumer just `mvn package` the `loggin-connapp` and
`java -jar logging-connapp.jar`;
* it buffers events in memory if the connection to the broker is
down; (if needed capacity limits are trivial to add in the code;)
* it reconnects to the broker in case of connectivity failures;
* it publishes the messages on a separate thread than where the
message was created; (thus the incurred overhead is minimal;)
As potential issues:
* it uses native Java serialization to encode the messages, thus
you can't just look at the messages with any RabbitMQ consumer; (of
course the code is quite modular and you could replace that with
* you can't enable and disable the appender, but you can make the
logger itself suspend the publishing;
As for it's stability it has been used in production for over a
year with only minor incidents (due to serialization exceptions).
Unfortunately the documentation is almost non-existing in the
repository, except those example files, but if there is someone
interested in using it, I could easily add it.
Of course I can also answers questions directly on this mailing
list or through a private email.
P.S.: Inside the same repository there is an "experimental"
BerkeleyDB + Lucene support to store and query the logs, but these are
P.P.S: If you are **really** interested in using the appender or
the consumer, but you require other features and they are easy to
implement I could try to implement them.
More information about the Logback-user