supervision - discussion about system services, daemon supervision, init, runlevel management, and tools such as s6 and runit
 help / color / mirror / Atom feed
* Following log directories
@ 2020-06-25  9:27 Carl Winbäck
  2020-06-25 13:20 ` Guillermo
  0 siblings, 1 reply; 14+ messages in thread
From: Carl Winbäck @ 2020-06-25  9:27 UTC (permalink / raw)
  To: supervision

Hi folks!

Does anyone know if the follow-log-directories tool[1] from the nosh
suite is compatible with logs produced by s6-log?


Best regards,
Carl Winbäck a.k.a. cpach



[1] https://jdebp.eu/Softwares/nosh/guide/commands/follow-log-directories.xml

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Following log directories
  2020-06-25  9:27 Following log directories Carl Winbäck
@ 2020-06-25 13:20 ` Guillermo
  2020-06-25 14:52   ` Guillermo
  0 siblings, 1 reply; 14+ messages in thread
From: Guillermo @ 2020-06-25 13:20 UTC (permalink / raw)
  To: Supervision

Hi,

> Does anyone know if the follow-log-directories tool[1] from the nosh
> suite is compatible with logs produced by s6-log?

Let's find out. I'll test that.

G.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Following log directories
  2020-06-25 13:20 ` Guillermo
@ 2020-06-25 14:52   ` Guillermo
  2020-06-25 22:08     ` Carl Winbäck
  0 siblings, 1 reply; 14+ messages in thread
From: Guillermo @ 2020-06-25 14:52 UTC (permalink / raw)
  To: Supervision

El jue., 25 jun. 2020 a las 10:20, Guillermo escribió:
>
> El jue., 25 jun. 2020 a las 6:27, Carl Winbäck escribió:
>
> > Does anyone know if the follow-log-directories tool[1] from the nosh
> > suite is compatible with logs produced by s6-log?
>
> Let's find out. I'll test that.

And it appears to be compatible, indeed, as long as s6-log is invoked
with the 't' directive (prepend timestamp in TAI64N format), to match
the logging file format expected by follow-log-directories, as stated
in its documentation.

Also, note that follow-log-directories is a daemon, so it will keep
running and outputting log lines until killed, once it catches up with
the logging directory. TAI64N timestamps are not decoded, so if you
want them in human-readable format, pipe its standard output to
s6-tai64nlocal.

You might also want to separate follow-log-directories's standard
output stream from its standard error stream, since the program is
somewhat chatty during catch-ups and during logdir rotations.

G.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Following log directories
  2020-06-25 14:52   ` Guillermo
@ 2020-06-25 22:08     ` Carl Winbäck
  2020-06-25 22:32       ` John W Higgins
  0 siblings, 1 reply; 14+ messages in thread
From: Carl Winbäck @ 2020-06-25 22:08 UTC (permalink / raw)
  To: Guillermo; +Cc: Supervision

On Thu, 25 Jun 2020 at 17:00, Guillermo wrote:
> And it appears to be compatible, indeed, as long as s6-log is invoked
> with the 't' directive (prepend timestamp in TAI64N format), to match
> the logging file format expected by follow-log-directories, as stated
> in its documentation.

Neat! Thank you Guillermo for this information.

Best regards,
Carl

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Following log directories
  2020-06-25 22:08     ` Carl Winbäck
@ 2020-06-25 22:32       ` John W Higgins
  2020-06-26  0:09         ` Laurent Bercot
  2020-06-27  7:33         ` Jonathan de Boyne Pollard
  0 siblings, 2 replies; 14+ messages in thread
From: John W Higgins @ 2020-06-25 22:32 UTC (permalink / raw)
  To: Supervision

[-- Attachment #1: Type: text/plain, Size: 953 bytes --]

One thing to remember about tools like this is that s6 is very precise with
it's implemenation of TAI64N - it uses leap seconds.

I use lnav [1] for a similar concept and it parses TAI64N but it does not
deal with leap seconds so my timestamps were off by like 20 seconds (or
something similar). I had to modify the code to handle the leap seconds.

So just make sure you check that the timestamps are what you expect them to
be with non s6 tools.

John

[1] - http://lnav.org/

On Thu, Jun 25, 2020 at 3:08 PM Carl Winbäck <c@tunnel53.net> wrote:

> On Thu, 25 Jun 2020 at 17:00, Guillermo wrote:
> > And it appears to be compatible, indeed, as long as s6-log is invoked
> > with the 't' directive (prepend timestamp in TAI64N format), to match
> > the logging file format expected by follow-log-directories, as stated
> > in its documentation.
>
> Neat! Thank you Guillermo for this information.
>
> Best regards,
> Carl
>

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Following log directories
  2020-06-25 22:32       ` John W Higgins
@ 2020-06-26  0:09         ` Laurent Bercot
  2020-06-26  0:49           ` John W Higgins
  2020-06-27  7:33         ` Jonathan de Boyne Pollard
  1 sibling, 1 reply; 14+ messages in thread
From: Laurent Bercot @ 2020-06-26  0:09 UTC (permalink / raw)
  To: John W Higgins, Supervision

>I use lnav [1] for a similar concept and it parses TAI64N but it does not
>deal with leap seconds so my timestamps were off by like 20 seconds (or
>something similar). I had to modify the code to handle the leap seconds.

  TAI64N labels represent TAI time, which is linear; when you need a
leap second table is when you want to convert TAI labels to UTC or
local time.

  My guess is that lnav just prints the TAI time as if it were UTC,
which indeed will print a time that, in 2020, is too high by 27 seconds.
If that is the case, it's a bug that would be worth reporting to lnav.

--
  Laurent


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Following log directories
  2020-06-26  0:09         ` Laurent Bercot
@ 2020-06-26  0:49           ` John W Higgins
  2020-06-26  9:24             ` Jan Braun
                               ` (2 more replies)
  0 siblings, 3 replies; 14+ messages in thread
From: John W Higgins @ 2020-06-26  0:49 UTC (permalink / raw)
  To: Supervision

[-- Attachment #1: Type: text/plain, Size: 974 bytes --]

On Thu, Jun 25, 2020 at 5:09 PM Laurent Bercot <ska-supervision@skarnet.org>
wrote:

>
>   My guess is that lnav just prints the TAI time as if it were UTC,
> which indeed will print a time that, in 2020, is too high by 27 seconds.
> If that is the case, it's a bug that would be worth reporting to lnav.
>
>
I meant nothing towards s6 - but daemontools does not deal with leap
seconds (or at least it cetainly looks that way from my foolish viewpoint).

So the problem for something like lnav is that it gets stuck between
daemontools and other tools like s6 which implement something "differently"
- be that right or wrong or anything else under the Sun,

It looks like lnav took the concept from daemontools and ran with it - far
worse decisions have been made by a tool trying to accomodate users.

I was just pointing out that there is a second check which may need to be
applied to make sure the viewer is providing the information one believes
they are seeing.

John

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Following log directories
  2020-06-26  0:49           ` John W Higgins
@ 2020-06-26  9:24             ` Jan Braun
  2020-06-26  9:27             ` Laurent Bercot
  2020-06-27  7:33             ` Jonathan de Boyne Pollard
  2 siblings, 0 replies; 14+ messages in thread
From: Jan Braun @ 2020-06-26  9:24 UTC (permalink / raw)
  To: John W Higgins; +Cc: Supervision

[-- Attachment #1: Type: text/plain, Size: 1656 bytes --]

John W Higgins schrob:
> I meant nothing towards s6 - but daemontools does not deal with leap
> seconds (or at least it cetainly looks that way from my foolish viewpoint).

Daemontools assumes that time(3) returns TAI-like time (seconds actually
elapsed since the epoch). That's a rather sane approach that allows to
do time calculations with complete disregard for leap seconds and time
zones. And for communication with the user, you consult tables listing
the effects of the erratic movements of the earth and politicians,
respectively. These tables are usually shipped in the "right/" tzdata
directory, used by setting e.g. TZ=right/Europe/Lisbon .

Unfortunately, POSIX mandates that time(3) returns UTC-like time instead
(arithmetic difference between the epoch and now, calculated as if leap
seconds didn't exist). That's what most systems do, also because the NTP
protocol runs on UTC, and NTP clients by default set computer's clocks
to UTC-like timestamps. The obvious disadvantage is the need to keep
leap seconds in mind when doing any calculation, or to accept being
wrong by up to 27 (as of now) seconds. And then consult tables for
politician effects only, e.g. TZ=Europe/Lisbon or equivalently
TZ=posix/Europe/Lisbon .

So yes, daemontools does not deal with leap seconds, because that's the
smart thing to do. ;)
At least if you can configure your NTP client to support that scenario.


> I was just pointing out that there is a second check which may need to be
> applied to make sure the viewer is providing the information one believes
> they are seeing.

Yes. I hope I could provice some useful/interesting background.
Cheers,
    Jan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Following log directories
  2020-06-26  0:49           ` John W Higgins
  2020-06-26  9:24             ` Jan Braun
@ 2020-06-26  9:27             ` Laurent Bercot
  2020-06-26 19:52               ` John W Higgins
  2020-06-27  7:33             ` Jonathan de Boyne Pollard
  2 siblings, 1 reply; 14+ messages in thread
From: Laurent Bercot @ 2020-06-26  9:27 UTC (permalink / raw)
  To: John W Higgins, Supervision


>I meant nothing towards s6 - but daemontools does not deal with leap
>seconds (or at least it cetainly looks that way from my foolish viewpoint).

  Indeed, the tai64nlocal program from daemontools assumes that the
system clock follows TAI-10, which is a smart choice, but unfortunately
not the one that has been made by POSIX - which mandates the system
clock to follow UTC.

  Assuming TAI-10 was a reasonable choice in 1998 (when daemontools was
written) because POSIX was not as commonly followed a standard as it
became several years later.
  In 2020, however, TAI-10 is a niche setting that can only be done by
systems where all components are aware of it. Most software today
assumes UTC, and cannot be blamed for it because it just follows the
standard; daemontools' tai64nlocal is now the odd one out and does
not display accurate time on POSIX-compliant systems.

  In the case of the s6-tai64nlocal program, which is s6's version of
its venerable daemontools counterpart, this is configurable at skalibs
compilation time: the default assumption is UTC, but the
--enable-tai-clock configure option will tell skalibs that the system
clock follows TAI-10.

  Please bear in mind that this only impacts the time *display*, i.e.
when you want to print the current time to users, typically in a
broken-down fashion. The whole point of using TAI in the first place
is that time computations are kept simple and exact, and do not need
leap second awareness or clock torture techniques such as leap smear
(only Google is evil enough to have invented that). TAI is still the
correct way to represent time internally, and difficulties such as
"need to look at a leap second table" only arise when you want to
convert it to something that humans traditionally use, such as UTC or
local time, in order to display it in a form that is suitable for
human consumption.


>It looks like lnav took the concept from daemontools and ran with it - far
>worse decisions have been made by a tool trying to accomodate users.

  It is definitely not a critical bug, but it is a bug nonetheless, that
shows a lack of understanding of the historical context, the use cases
of TAI and UTC, and the relationship between the two. It would be worth
reporting it to lnav, so it can print accurate information.

--
  Laurent


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Following log directories
  2020-06-26  9:27             ` Laurent Bercot
@ 2020-06-26 19:52               ` John W Higgins
  2020-06-27  6:12                 ` Laurent Bercot
  0 siblings, 1 reply; 14+ messages in thread
From: John W Higgins @ 2020-06-26 19:52 UTC (permalink / raw)
  To: Supervision

[-- Attachment #1: Type: text/plain, Size: 2691 bytes --]

On Fri, Jun 26, 2020 at 2:27 AM Laurent Bercot <ska-supervision@skarnet.org>
wrote:

>
>
>   Please bear in mind that this only impacts the time *display*, i.e.
> when you want to print the current time to users, typically in a
> broken-down fashion. The whole point of using TAI in the first place
> is that time computations are kept simple and exact, and do not need
> leap second awareness or clock torture techniques such as leap smear
> (only Google is evil enough to have invented that). TAI is still the
> correct way to represent time internally, and difficulties such as
> "need to look at a leap second table" only arise when you want to
> convert it to something that humans traditionally use, such as UTC or
> local time, in order to display it in a form that is suitable for
> human consumption.
>
>
> >It looks like lnav took the concept from daemontools and ran with it - far
> >worse decisions have been made by a tool trying to accomodate users.
>
>   It is definitely not a critical bug, but it is a bug nonetheless, that
> shows a lack of understanding of the historical context, the use cases
> of TAI and UTC, and the relationship between the two. It would be worth
> reporting it to lnav, so it can print accurate information.
>
>
Except it's not quite that simple either.

As an example

echo "s6" | s6-log -t /tmp/a && echo "djb" | multilog t /tmp/a && echo "s6"
| s6-log -t /tmp/a

Produces the following /tmp/a/current file on my system, roughly as a write
this.

@400000005ef64dd2075013a2 s6
@400000005ef64db707675514 djb
@400000005ef64dd2077fa679 s6

There is a large problem there for someone building a tool to show
timestamps that begin as TAI. Either the middle one is 27 seconds in the
past or the first and last are 27 seconds in the future.

If one wants to support djb tooling - then leap seconds are off the table -
if one wishes to support s6 - leap seconds come on the table. What nosh
supports is unknown to me.

When lnav implemented the support - they did exactly what sounds reasonable
- they took the tool they wished to support (multilog I would presume as it
mentionds cr.yp.to) and implemented the code required to produce a view of
that tool's logs. To say that is a lack of understanding is rather harsh
here.

John

P.S. I totally appreciate s6 and all its beauty - I really just wanted to
point out to the original poster that they need to double check because the
worst thing one can do is miss something like this and then wonder why
their logs are off when they go and look at a production system. 27 seconds
seems obvious to catch - but in development one doesn't always focus enough
- it's not like 2 hours or something more obvious.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Following log directories
  2020-06-26 19:52               ` John W Higgins
@ 2020-06-27  6:12                 ` Laurent Bercot
  0 siblings, 0 replies; 14+ messages in thread
From: Laurent Bercot @ 2020-06-27  6:12 UTC (permalink / raw)
  To: John W Higgins, Supervision


>When lnav implemented the support - they did exactly what sounds reasonable
>- they took the tool they wished to support (multilog I would presume as it
>mentionds cr.yp.to) and implemented the code required to produce a view of
>that tool's logs. To say that is a lack of understanding is rather harsh
>here.

  Right. I was being charitable; it's indeed also possible that they
*intentionally* did the wrong thing, which is obviously much worse. :P

--
  Laurent


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Following log directories
  2020-06-25 22:32       ` John W Higgins
  2020-06-26  0:09         ` Laurent Bercot
@ 2020-06-27  7:33         ` Jonathan de Boyne Pollard
  1 sibling, 0 replies; 14+ messages in thread
From: Jonathan de Boyne Pollard @ 2020-06-27  7:33 UTC (permalink / raw)
  To: Supervision

John W Higgins:

> One thing to remember about tools like this is that s6 is very precise 
> with it's implemenation of TAI64N - it uses leap seconds.
>
For the case of follow-log-directories, note that it does not decode the 
timestamps into times.  As long as they follow the simple (and obvious) 
constraints in its manual, the tool is happy.


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Following log directories
  2020-06-26  0:49           ` John W Higgins
  2020-06-26  9:24             ` Jan Braun
  2020-06-26  9:27             ` Laurent Bercot
@ 2020-06-27  7:33             ` Jonathan de Boyne Pollard
  2020-06-28 19:38               ` Guillermo
  2 siblings, 1 reply; 14+ messages in thread
From: Jonathan de Boyne Pollard @ 2020-06-27  7:33 UTC (permalink / raw)
  To: Supervision

John W Higgins wrote:

> I meant nothing towards s6 - but daemontools does not deal with leap 
> seconds (or at least it cetainly looks that way from my foolish 
> viewpoint).
>
daemontools very much *does* deal in leap seconds.  The little-known 
requirement for daemontools, which *is* in its manual and has been all 
along albeit that many people have overlooked this, is that one use one 
of the Olson "right" timezones.

* http://jdebp.uk./Softwares/nosh/guide/timestamps.html

* http://cr.yp.to/daemontools/tai64nlocal.html


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Following log directories
  2020-06-27  7:33             ` Jonathan de Boyne Pollard
@ 2020-06-28 19:38               ` Guillermo
  0 siblings, 0 replies; 14+ messages in thread
From: Guillermo @ 2020-06-28 19:38 UTC (permalink / raw)
  To: Supervision

El sáb., 27 jun. 2020 a las 4:59, Jonathan de Boyne Pollard escribió:
>
> John W Higgins wrote:
>
> > I meant nothing towards s6 - but daemontools does not deal with leap
> > seconds (or at least it cetainly looks that way from my foolish
> > viewpoint).
> >
> daemontools very much *does* deal in leap seconds.  The little-known
> requirement for daemontools, which *is* in its manual and has been all
> along albeit that many people have overlooked this, is that one use one
> of the Olson "right" timezones.

If anyone is interested, decoding the TAI64N timestamps produced by
s6-log with nosh's version of tai64nlocal produces the same result as
doing so with s6-tai64nlocal, and decoding the TAI64N timestamps
produced by nosh's cyclog with s6-tai64nlocal produces the same result
as doing so with nosh's tai64nlocal. At least on my system without a
"right/" timezone.

On the other hand, I just realized that cyclog does not seem to
properly finalize the 'current' file (including the setting of the
execute-by-owner permission) when it exits normally. Perhaps the
destructor of class 'logger' should call logger::flush_and_close()
rather than logger::close()?

G.

^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2020-06-28 19:39 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-06-25  9:27 Following log directories Carl Winbäck
2020-06-25 13:20 ` Guillermo
2020-06-25 14:52   ` Guillermo
2020-06-25 22:08     ` Carl Winbäck
2020-06-25 22:32       ` John W Higgins
2020-06-26  0:09         ` Laurent Bercot
2020-06-26  0:49           ` John W Higgins
2020-06-26  9:24             ` Jan Braun
2020-06-26  9:27             ` Laurent Bercot
2020-06-26 19:52               ` John W Higgins
2020-06-27  6:12                 ` Laurent Bercot
2020-06-27  7:33             ` Jonathan de Boyne Pollard
2020-06-28 19:38               ` Guillermo
2020-06-27  7:33         ` Jonathan de Boyne Pollard

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).