[logback-user] logback autoscan is not working intermittently in k8s pod
Raymond Augé
raymond.auge at liferay.com
Fri Aug 20 16:01:53 CEST 2021
Hi, a long time lurker here 🙂.
So, I suspect the issue is with k8s and it's volumes for config
maps/secrets symlinks are a pain point in Java. Even inotify based
WachService cannot handle them.
The issue is that logback is only using a timestamp to detect configuration
file changes, which is fine as we know in 99% of Java usages.
However, this doesn't seem to work with the config map volumes supplied in
a k8s env because they use symlinks which don't cause the timestamp to
change, I agree it's weird and unexpected.
However, supplying configuration files via K8s config maps is going to
start being a much more common use case as time passes.
Logback should probably start including a checksum to detect if the
contents of the file have changed and use a logical OR with the timestamp
check so that either changes causes a reload.
FYI, I had to patch a similar issue in Apache Felix Fileinstall due to the
same k8s config maps/secrets issue [1]
My 2 cents,
Ray
[1] https://issues.apache.org/jira/browse/FELIX-6229
On Fri., Aug. 20, 2021, 5:35 a.m. Ceki Gülcü, <ceki at qos.ch> wrote:
> Hello Kedar,
>
> Have you tried to reproduce the problem on a stand alone machine without
> k8?
>
>
>
> On 20/08/2021 00:41, Kedar Sirshikar wrote:
> > Hello Ceki,
> > I tried debugging by reproducing the issue and observed that after 7th
> > attempt of log change, autoScan stops working.
> >
> > Have you came across similar issue in 1.2.3 version where autoScan does
> > not pick changes from config xml if it gets changed after a certain
> count ?
> > In the meantime, I am trying to see if same issue occurs on 1.1.3
> > version because setMainWatchURL method inConfigurationWatchListUtil is
> > different on both versions.
> >
> >
> > Thanks,
> > Kedar.
> >
> >
> > On Sat, Aug 14, 2021 at 3:00 AM <logback-user-request at qos.ch
> > <mailto:logback-user-request at qos.ch>> wrote:
> >
> > Send logback-user mailing list submissions to
> > logback-user at qos.ch <mailto:logback-user at qos.ch>
> >
> > To subscribe or unsubscribe via the World Wide Web, visit
> > http://mailman.qos.ch/mailman/listinfo/logback-user
> > <http://mailman.qos.ch/mailman/listinfo/logback-user>
> > or, via email, send a message with subject or body 'help' to
> > logback-user-request at qos.ch <mailto:logback-user-request at qos.ch>
> >
> > You can reach the person managing the list at
> > logback-user-owner at qos.ch <mailto:logback-user-owner at qos.ch>
> >
> > When replying, please edit your Subject line so it is more specific
> > than "Re: Contents of logback-user digest..."
> >
> >
> > Today's Topics:
> >
> > 1. Re: logback autoscan is not working intermittently in k8s pod
> > (Ceki Gülcü)
> >
> >
> >
> ----------------------------------------------------------------------
> >
> > Message: 1
> > Date: Fri, 13 Aug 2021 14:51:09 +0200
> > From: Ceki Gülcü <ceki at qos.ch <mailto:ceki at qos.ch>>
> > To: logback-user at qos.ch <mailto:logback-user at qos.ch>
> > Subject: Re: [logback-user] logback autoscan is not working
> > intermittently in k8s pod
> > Message-ID: <f39cbede-6e95-7e60-d01e-ca27599bf453 at qos.ch
> > <mailto:f39cbede-6e95-7e60-d01e-ca27599bf453 at qos.ch>>
> > Content-Type: text/plain; charset=utf-8; format=flowed
> >
> > Hello Kedar,
> >
> >
> > I see that changes to the logback config file are picked up. There
> are
> > some errors reported when processing the configuration file but these
> > are unrelated to the issue at hand, i.e. auto scan not working.
> >
> > I don't have further suggestions.
> >
> > --Ceki
> >
> >
> > On 13/08/2021 09:59, Kedar Sirshikar wrote:
> > > Hello team,
> > > Just trying to check if anyone got a chance to look at this case?
> > > Appreciate if any clue/hint is provided to resolve this issue.
> > >
> > > Thanks,
> > > Kedar.
> > >
> > >
> > > On Wed, Aug 11, 2021 at 12:57 PM Kedar Sirshikar
> > > <kedargsirshikar at gmail.com <mailto:kedargsirshikar at gmail.com>
> > <mailto:kedargsirshikar at gmail.com
> > <mailto:kedargsirshikar at gmail.com>>> wrote:
> > >
> > > Resending the attachment by updating the subject. Please let
> > me know
> > > if any additional details are required from my side.
> > >
> > > Thanks,
> > > Kedar.
> > >
> > >
> > > ---------- Forwarded message ---------
> > > From: *Kedar Sirshikar* <kedargsirshikar at gmail.com
> > <mailto:kedargsirshikar at gmail.com>
> > > <mailto:kedargsirshikar at gmail.com
> > <mailto:kedargsirshikar at gmail.com>>>
> > > Date: Wed, Aug 11, 2021 at 7:58 AM
> > > Subject: Re: logback-user Digest, Vol 115, Issue 2
> > > To: <logback-user at qos.ch <mailto:logback-user at qos.ch>
> > <mailto:logback-user at qos.ch <mailto:logback-user at qos.ch>>>
> > >
> > >
> > > Hi Ceki, Thank you for the reply. Forgot to send the logs
> > earlier so
> > > please refer attachment.
> > > I have also added Notes at lines 1 and 91 to describe the
> test.
> > >
> > >
> > > Thanks,
> > > Kedar.
> > >
> > >
> > > On Wed, Aug 11, 2021 at 3:00 AM <logback-user-request at qos.ch
> > <mailto:logback-user-request at qos.ch>
> > > <mailto:logback-user-request at qos.ch
> > <mailto:logback-user-request at qos.ch>>> wrote:
> > >
> > > Send logback-user mailing list submissions to
> > > logback-user at qos.ch <mailto:logback-user at qos.ch>
> > <mailto:logback-user at qos.ch <mailto:logback-user at qos.ch>>
> > >
> > > To subscribe or unsubscribe via the World Wide Web, visit
> > > http://mailman.qos.ch/mailman/listinfo/logback-user
> > <http://mailman.qos.ch/mailman/listinfo/logback-user>
> > > <http://mailman.qos.ch/mailman/listinfo/logback-user
> > <http://mailman.qos.ch/mailman/listinfo/logback-user>>
> > > or, via email, send a message with subject or body 'help'
> to
> > > logback-user-request at qos.ch <mailto:logback-user-request at qos.ch>
> > <mailto:logback-user-request at qos.ch
> > <mailto:logback-user-request at qos.ch>>
> > >
> > > You can reach the person managing the list at
> > > logback-user-owner at qos.ch <mailto:logback-user-owner at qos.ch>
> > <mailto:logback-user-owner at qos.ch <mailto:logback-user-owner at qos.ch
> >>
> > >
> > > When replying, please edit your Subject line so it is
> > more specific
> > > than "Re: Contents of logback-user digest..."
> > >
> > >
> > > Today's Topics:
> > >
> > > 1. logback autoscan is not working intermittently in
> > k8s pod
> > > (Kedar Sirshikar)
> > > 2. Re: logback autoscan is not working intermittently
> > in k8s pod
> > > (Ceki Gülcü)
> > >
> > >
> > >
> >
> ----------------------------------------------------------------------
> > >
> > > Message: 1
> > > Date: Tue, 10 Aug 2021 23:07:51 -0700
> > > From: Kedar Sirshikar <kedargsirshikar at gmail.com
> > <mailto:kedargsirshikar at gmail.com>
> > > <mailto:kedargsirshikar at gmail.com
> > <mailto:kedargsirshikar at gmail.com>>>
> > > To: logback-user at qos.ch <mailto:logback-user at qos.ch>
> > <mailto:logback-user at qos.ch <mailto:logback-user at qos.ch>>
> > > Subject: [logback-user] logback autoscan is not working
> > > intermittently
> > > in k8s pod
> > > Message-ID:
> > >
> > >
> > <CAO=
> cJL+EMFCTe7En+7nqUmzaxQoYVEYN1GXCYwdwfkt0KbjiDQ at mail.gmail.com <mailto:
> cJL%2BEMFCTe7En%2B7nqUmzaxQoYVEYN1GXCYwdwfkt0KbjiDQ at mail.gmail.com>
> > >
> > <mailto:
> cJL%2BEMFCTe7En%2B7nqUmzaxQoYVEYN1GXCYwdwfkt0KbjiDQ at mail.gmail.com
> <mailto:
> cJL%252BEMFCTe7En%252B7nqUmzaxQoYVEYN1GXCYwdwfkt0KbjiDQ at mail.gmail.com>>>
> > > Content-Type: text/plain; charset="utf-8"
> > >
> > > Hi team,
> > > I have raised a detailed query here
> > >
> > <
> https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod
> <
> https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod
> >
> > >
> > <
> https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod
> <
> https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod
> >>>
> > > about
> > > the issue I am facing.
> > > Can someone please share inputs or any hint for solving
> the
> > > above problem?
> > > Let me know if any details are required from my side.
> > >
> > > Thanks,
> > > Kedar.
> > > -------------- next part --------------
> > > An HTML attachment was scrubbed...
> > > URL:
> > >
> > <
> http://mailman.qos.ch/pipermail/logback-user/attachments/20210810/7ee3b6b7/attachment-0001.html
> <
> http://mailman.qos.ch/pipermail/logback-user/attachments/20210810/7ee3b6b7/attachment-0001.html
> >
> > >
> > <
> http://mailman.qos.ch/pipermail/logback-user/attachments/20210810/7ee3b6b7/attachment-0001.html
> <
> http://mailman.qos.ch/pipermail/logback-user/attachments/20210810/7ee3b6b7/attachment-0001.html
> >>>
> > >
> > > ------------------------------
> > >
> > > Message: 2
> > > Date: Wed, 11 Aug 2021 08:39:46 +0200
> > > From: Ceki Gülcü <ceki at qos.ch <mailto:ceki at qos.ch>
> > <mailto:ceki at qos.ch <mailto:ceki at qos.ch>>>
> > > To: logback-user at qos.ch <mailto:logback-user at qos.ch>
> > <mailto:logback-user at qos.ch <mailto:logback-user at qos.ch>>
> > > Subject: Re: [logback-user] logback autoscan is not
> working
> > > intermittently in k8s pod
> > > Message-ID: <2a210996-a364-e4f0-fe85-508c2af0b733 at qos.ch
> > <mailto:2a210996-a364-e4f0-fe85-508c2af0b733 at qos.ch>
> > > <mailto:2a210996-a364-e4f0-fe85-508c2af0b733 at qos.ch
> > <mailto:2a210996-a364-e4f0-fe85-508c2af0b733 at qos.ch>>>
> > > Content-Type: text/plain; charset=utf-8; format=flowed
> > >
> > >
> > > Hello Kedar,
> > >
> > > Can you also provide the internal status messages
> > generated by
> > > logback?
> > > --
> > > Ceki
> > >
> > > On 11/08/2021 08:07, Kedar Sirshikar wrote:
> > > > Hi team,
> > > > I have raised a detailed query here
> > > >
> > >
> > <
> https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod
> <
> https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod
> >
> > >
> > <
> https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod
> <
> https://stackoverflow.com/questions/68721761/logback-autoscan-is-not-working-intermittently-in-k8s-pod
> >>> about
> > >
> > > > the issue I am facing.
> > > > Can someone please share inputs or any hint for
> > solving the
> > > above problem?
> > > > Let me know if any details are required from my side.
> > > >
> > > > Thanks,
> > > > Kedar.
> > > >
> > > > _______________________________________________
> > > > logback-user mailing list
> > > > logback-user at qos.ch <mailto:logback-user at qos.ch>
> > <mailto:logback-user at qos.ch <mailto:logback-user at qos.ch>>
> > > > http://mailman.qos.ch/mailman/listinfo/logback-user
> > <http://mailman.qos.ch/mailman/listinfo/logback-user>
> > > <http://mailman.qos.ch/mailman/listinfo/logback-user
> > <http://mailman.qos.ch/mailman/listinfo/logback-user>>
> > > >
> > >
> > >
> > > ------------------------------
> > >
> >
> >
> > ------------------------------
> >
> > Subject: Digest Footer
> >
> > _______________________________________________
> > logback-user mailing list
> > logback-user at qos.ch <mailto:logback-user at qos.ch>
> > http://mailman.qos.ch/mailman/listinfo/logback-user
> > <http://mailman.qos.ch/mailman/listinfo/logback-user>
> >
> > ------------------------------
> >
> > End of logback-user Digest, Vol 115, Issue 6
> > ********************************************
> >
> >
> > _______________________________________________
> > logback-user mailing list
> > logback-user at qos.ch
> > http://mailman.qos.ch/mailman/listinfo/logback-user
> >
> _______________________________________________
> logback-user mailing list
> logback-user at qos.ch
> http://mailman.qos.ch/mailman/listinfo/logback-user
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.qos.ch/pipermail/logback-user/attachments/20210820/01f53442/attachment-0001.html>
More information about the logback-user
mailing list