[svlogd] / -ttt / why UTC?

[svlogd] / -ttt / why UTC?

[cpt.arsemerica.yahoo.com via supervision]

Hi!

My boxes use TAI (international atomic time) in order to have SI-seconds and 60sec minutes and 24hrs days... I do not like time jumps (exempli gratia: leap seconds as "implemented" on many UN*X boxes...)... and Google's "smeared time"... Details: https://www.reddit.com/r/voidlinux/comments/znlw6g/timezone_without_funny_minutesdays/

svlogd uses UTC timestamps, which look awkward on my boxes and possibly even dont get those 61sec minutes right (I did not crosscheck that...)...

Can i have a "-tttt" option for "localtime"?

Or a "-z <timezone specifier>" option?

And when we are at it: A "-d <strftime format>" option would be nice, too...?

Please... :-]

Thx.

Bye

Re: [svlogd] / -ttt / why UTC?

[Laurent Bercot]

>My boxes use TAI (international atomic time) in order to have SI-seconds and 60sec minutes and 24hrs days...

  If your system clock is set to TAI-10, then *all* the time-handling
software on your machine must be aware of it, in order to perform
time computations accurately. It is not sufficient to use the right/
time zones: a TAI-10 setup changes the interpretation of what
system calls such as time() and clock_gettime() return. It is only
possible to do this and have a consistent view of time if you control
any and every piece of software on your machine that needs to read
and display time.

  See https://skarnet.org/software/skalibs/flags.html#clockistai
for a description of the difference between "system clock set to TAI-10"
and "system clock set to UTC".

  If you're using a distribution - in your case, Void Linux - then
TAI-10 is not an option for you, unless the distribution explicitly
supports it and has built all its binaries to support it. Even if you
could make svlogd work with it, you would have trouble with other
software - this may be benign, like "date", or less so, like software
that checks the validity of X.509 certificates against certain things
like expiration dates.

  Believe me, I understand your plight, and I wish TAI-10 were the de
facto standard instead of UTC. I wish it were at least more widely
known and supported. I use it on my servers - but it comes at a heavy
price: not letting any third-party software, 99.9% of which is hardcoded
for UTC, interact with time, and only using my own tools, which support
the setup, to do that.

  I don't think you'll be able to keep running Void with a full TAI-10
setup and have everything work perfectly, at some point you'll have to
make a real choice. But right now, to superficially solve your current
issue, you could try running s6-log (from s6) in place of svlogd.
It does the same thing, but it supports TAI-10, if you build skalibs
with the --enable-tai-clock option, as described by the link above.
You probably have to build skalibs and s6 yourself: I don't think the
Void package for skalibs uses that option.

  Good luck,

--
  Laurent

Re: [svlogd] / -ttt / why UTC?

[cpt.arsemerica.yahoo.com via supervision]

Laurent Bercot <ska-supervision@skarnet.org> wrote:
>  If your system clock is set to TAI-10, [...]
>
I don't know exactly what u mean with "TAI-10".
I guess u are refering to those 10seconds
that were the initial difference between UTC and TAI at the start of 1972...
So it should be TAI-00:00:10...

> [...] then *all* the time-handling
> software on your machine must be aware of it,
>
here i disagree a little bit:
As long as the software uses glibc's time functions to break down
"seconds since the start of 1970* to year, month, mday, hours, minutes, seconds,
the software does not need any patches...
Ur link is a little bit misleading or outdated in that point...

exim, fetchmail, dma and s-nail need(ed) a change, because:
They omitted the non-zero seconds
instead of converting the timestamp to UTC and
printing the timezone as -00:00 (with a minus).

> I don't think you'll be able to keep running Void with a full TAI-10
>
What would be wrong about the "-tttt" (localtime time stamp) option?
Then I would not have to build/write my own *log daemon...
And svlogd just needs to use localtime_r(3)...

Re: [svlogd] / -ttt / why UTC?

[Laurent Bercot]

>I don't know exactly what u mean with "TAI-10".
>I guess u are refering to those 10seconds

  Yes. You cannot set your system clock to TAI, unless you want wildly
incorrect results from time() and similar system calls. Setting it 10
seconds earlier than TAI is the best you can do; and that's what the
right/ timezones expect.


>here i disagree a little bit:
>As long as the software uses glibc's time functions to break down
>"seconds since the start of 1970* to year, month, mday, hours, minutes, seconds,
>the software does not need any patches...
>Ur link is a little bit misleading or outdated in that point...

  Have you tried it?

  localtime() will work if your timezone is correctly set, yes.
  gmtime() *will not work*, because it assumes a UTC system clock.
  Programs making their own assumptions, and there are a lot of these,
will not work. Don't think everyone uses the libc; new compiled
language ecosystems such as for instance Go or Rust have their own time
libraries, and can you be certain they don't assume UTC?

  "As long as the software pays strict attention to the standards and
follows them to a T even for non-100%-cookie-cutter cases" is a very
strong hypothesis to make, even when you perform minor changes.
Changing the way the system clock counts time is NOT a minor change.


>What would be wrong about the "-tttt" (localtime time stamp) option?
>Then I would not have to build/write my own *log daemon...
>And svlogd just needs to use localtime_r(3)...

  In theory, nothing, except that it's a bad idea to timestamp logs
with local time (hello, we're in the 21st century, we manage computers
worldwide, we want logs to be shareable and mergeable across timezones).

  In practice, you're asking a maintainer to perform work that *you* 
need,
and that very few other people need. That's generally considered bad
form. The way to go about it would be to implement the functionality
yourself, and submit a patch; that said, since the last time we saw
svlogd's maintainer was four years ago in a flash, I probably wouldn't
bother if I were you.
  By recommending s6-log, I gave you the solution that requires from you
the least amount of work and the least amount of waiting.

--
  Laurent

Re: [svlogd] / -ttt / why UTC?

[cpt.arsemerica.yahoo.com via supervision]

> Yes. You cannot set your system clock to TAI, unless you want wildly
> incorrect results from time() [...]
>
In my world time() returns the SI seconds since the start of 1970.
Since TAI and UTC were off for fractions of a second
from 1970 to the end of 1972 might be true,
but my applications dont care for that time so much...
So I think my results are correct...
Just some little system clock frequency jitter (10msec) and
the round trip time of my VDSL internet connection (<40msec).
But that has nothing to do with /usr/share/zoneinfo/...


>>As long as the software uses glibc's time functions to break down
>>
>Have you tried it?
>
Sure... See:
% TZ=UTC date -d "2017-01-01 - 1 second"
Sat Dec 31 23:59:60 UTC 2016
% TZ=UTC date -d "2017-01-01 - 1 second" +%s
1483228826
% TZ=Etc/TAI date -d@1483228825
Sun Jan  1 00:00:35 TAI 2017
% TZ=UTC date -d@1483228825
Sat Dec 31 23:59:59 UTC 2016
% TZ=Etc/TAI date -d@1483228826
Sun Jan  1 00:00:36 TAI 2017
% TZ=UTC date -d@1483228826
Sat Dec 31 23:59:60 UTC 2016
% TZ=Etc/TAI date -d@1483228827
Sun Jan  1 00:00:37 TAI 2017
% TZ=UTC date -d@1483228827
Sun Jan  1 00:00:00 UTC 2017

I should also say,
that i do not only use a special timezone info file
( http://www0.wgboome.org/zoneinfo-TAI.tar.xz ),
but also a special NTP client,
that adds those 10 seconds,
which i could not do with that TAI file.

> gmtime() *will not work*, because it assumes a UTC system clock.
>
Yes. And that is what it gets...
I sym-linked that /usr/share/zoneinfo-leaps to /usr/share/zoneinfo

> Programs making their own assumptions, and there are a lot of these,
>
Yes, Everything that uses GLib... Javascript and balsa for example...
I patched balsa for myself (GLib devs refused to fix it properly)...
Somewhen I will find a patch for GLib's borkened interpretation of the zoneinfo files... Somewhen... :)

>>What would be wrong about the "-tttt" (localtime time stamp) option?
> In theory, nothing, except that it's a bad idea to timestamp logs
> with local time (hello, we're in the 21st century, we manage computers
> worldwide, we want logs to be shareable and mergeable across timezones).
>
In my case u wound land at TAI timestamps... :)
That's why i am asking...
I dont like to build packages myself...

> By recommending s6-log, I gave you the solution that requires from you
> the least amount of work and the least amount of waiting.
>
OK... s6-log uses localtime, when started with -T...
But that means another 1MiB on my litte root partition... giggle

I think, i will write socklog&svlogd myself...
Maybe i can even use less than 2MiB of main memory...

Or i continue to use svlogd without options,
but then the kernel log lines have a timestamp relative to boot time...
*face palm*

Re: [svlogd] / -ttt / why UTC?

[Alexis]

"Laurent Bercot" <ska-supervision@skarnet.org> writes:

> The way to go about it would be to implement the functionality
> yourself, and submit a patch; that said, since the last time we 
> saw
> svlogd's maintainer was four years ago in a flash

Well, Void maintains its own runit repo:

    https://github.com/void-linux/runit

So one could open an issue there and ask if a PR implementing this 
functionality would be considered.


Alexis.

Re: [svlogd] / -ttt / why UTC?

[Jonathan de Boyne Pollard]

>> Yes. You cannot set your system clock to TAI, unless you want wildly 
>> incorrect results from time() and similar system calls. Setting it 10 
>> seconds earlier than TAI is the best you can do; and that's what the 
>> right/ timezones expect.
>>
>
> In my world time() returns the SI seconds since the start of 1970. 
> Since TAI and UTC were off for fractions of a second from 1970 to the 
> end of 1972 might be true, but my applications dont care for that time 
> so much...
>

M. Bercot's point, which you edited out, is that what xe means by 
"TAI-10" is exactly that, because what you say here is untrue. TAI was 
different from UTC by over 10 seconds, not by merely fractions of 1 
second, and the "start of 1970" in one was not the "start of 1970" in 
the other.  People don't use TAI in Unices and Linux-based operating 
systems.  They use what is more accurately termed TAI-10; because there 
aren't versions of the timezone support data files (supplied "out of the 
box") that count SI seconds and use "start of 1970" TAI, only ones that 
count SI seconds and use "start of 1970" UTC, i.e. "start of 1970" TAI-10.

* https://unix.stackexchange.com/a/327403/5132

And whereas on an operating system like OpenBSD one should expect both 
"right" and "posix" timezones to be supported, and any inconsistencies 
in applications to be corrected with patches in ports, on Linux-based 
operating systems the hotch-potch nature of the various language runtime 
libraries and application repositories does not given one nearly as much 
confidence that it's all correct.

Well done for trying to push corrections back to the authors; but that 
makes you in a better position than most to agree with M. Bercot that 
the state of the applications and Linux-based operating systems as 
supplied leaves a little to be desired for TAI-10 operation.  But as M. 
Bercot and I both say, the major single offender here is {x,}ntpd.

> I think, i will write socklog&svlogd myself...
>

Personally, I just go with the Bernstein timestamps.  Then I can convert 
to whatever human-readable form that I like, without touching the raw 
log files or having to worry about parsing log files with 
locale-dependent formats in them; even reading the same log entries in 
multiple different timezones and locales if I really want to.