Re: Linuxisms in s6

[Jonathan de Boyne Pollard <J.deBoynePollard-newsgroups@NTLWorld.com>]

Adrian Chadd:

> [...] the uptime stuff really threw us.
>
It's unfair to lay such system time problems at s6's door. Systems whose 
system clock jumps 46 years during system bootstrap don't get to blame 
s6 for mad time gaps that appear in logs and service start time 
records.  There is a *lot* of the Unix and Linux worlds that depends 
from time being right.  It's not just s6 that is affected by such 
things.  You note crypto.  There are a lot of other things as well that 
have unstated, sometimes undocumented, and sometimes surprising 
dependencies upon system time being current.

Here's one such.

For quite a while, Linux distributions had rather an odd problem at 
bootstrap.  They'd repeatedly fsck volumes at every bootstrap when they 
need not have.  And this didn't affect U.S. or U.K. people, which is in 
part why it persisted for so long.

* https://bugs.launchpad.net/ubuntu/+source/e2fsprogs/+bug/63175

* https://bugs.archlinux.org/task/17438

* http://lwn.net/Articles/264498/

The problem was that people were erroneously running their real-time 
clocks in local time rather than UTC, and this triggered an odd hidden 
dependency upon having the right time in the system clock in countries 
where local time was in advance of UTC.  The Linux method for handling 
RTCs erroneously running in local time is for the system bootstrap to 
make a special settimeofday() call that effectively tells the kernel 
what the UTC offset is for the RTC hardware.  This could happen *after* 
the fsck of the root volume, however.  So whilst that fsck was 
happening, the kernel was assuming that UTC was the local time that it 
had taken from the RTC and initialized its system clock with.  In 
effect, as soon as the special settimeofday() call was executed, the 
system clock would jump backwards by one or more hours, to what UTC 
actually was.

But the ext2/3/4 filesystem format has last checked/mounted/written 
timestamps in its superblock.  Part of the checking to see whether a 
full fsck is needed at bootstrap is comparing them to the current time.  
If they are in the future by hours or more, something is clearly wrong, 
thinks fsck, and it runs the full check.  At bootstrap, when the initial 
fsck (of at least the root volume and sometimes other volumes as well) 
is run, the system clock is not UTC yet.  Comedy results.

Both systemd and the nosh system manager have to ensure that they do the 
special settimeofday() system call before they start off service 
management and thus run mount/fsck services, or indeed anything else 
that might have a closet dependency from not stepping the system time by 
hours partway through bootstrap.  The nosh system-manager's manual page 
has a whole section on this subject.

FreeBSD/PC-BSD has a mechanism for correctly reading a RTC that is 
erroneously in local time.  One sets up the RTC's offset from UTC in the 
machdep.adjkerntz variable in /boot/loader.conf{,.local} and the system 
clock never has to jump by hours during bootstrap.  I've yet to 
experience a FreeBSD/PC-BSD system where the installer actually 
configures this, though.

Interestingly, FreeBSD/PC-BSD also has a fallback mechanism that uses 
the latest volume mount timestamp that it can find as the initial system 
time when no hardware clock device registers at bootstrap.  Presumably 
you have a clock device that registers but it is not battery-backed, 
your volumes don't preserve (or reset) their mount timestamps, or you 
are encountering the comedy situation where FreeBSD/PC-BSD is setting 
the system clock to 1970-01-01 because the last time around it mounted 
the filesystems before the clock was corrected.
_______________________________________________
freebsd-hackers@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"