[announce] skarnet.org Winter 2021-2022 release

[announce] skarnet.org Winter 2021-2022 release

[Laurent Bercot]

  Hello,

  New versions of all the skarnet.org packages are available.

  The changes are, for the most part, minimal: essentially, the new
versions  fix a bug in the build system that made cross-building under
slashpackage more difficult than intended. Very few people should
have been impacted by this bug.
  Some packages had a few more bugfixes; and some packages have
additional functionality. No major update; no compatibility break.

  The new versions are the following:

skalibs-2.11.1.0             (minor)
nsss-0.2.0.1                 (release)
utmps-0.1.1.0                (minor)
execline-2.8.2.0             (minor)
s6-2.11.0.1                  (release)
s6-rc-0.5.3.0                (minor)
s6-portable-utils-2.2.3.4    (release)
s6-linux-utils-2.5.1.7       (release)
s6-linux-init-1.0.7.0        (minor)
s6-dns-2.3.5.3               (release)
s6-networking-2.5.1.0        (minor)
mdevd-0.1.5.1                (release)
bcnm-0.0.1.5                 (release)
dnsfunnel-0.0.1.3            (release)
smtpd-starttls-proxy-0.0.1.1 (release)

  Dependencies have all been updated to the latest versions. They are not
strict: libraries and binaries may build with older releases of their
dependencies, although this is not guaranteed.

  You do not need to recompile your s6-rc service databases. To make use
of the new s6-linux-init functionality, however, you will have to
recreate your run-image.
  You do not need to restart your supervision tree, unless you're 
deleting
your old s6 binaries.

  Details of minor package changes follow.

* skalibs-2.11.1.0
   ----------------

  - New function: opendir_at()


* utmps-0.1.1.0
   ------------

  - New binary: utmps-write, a generic utmp client that can write
user-crafted records to the utmp and/or wtmp databases.


* execline-2.8.2.0
   ----------------

  - New -s option to the case binary, enabling fnmatch() (shell)
expression matching instead of regular expression matching.


* s6-rc-0.5.3.0
   -------------

  - Bundle contents are now read in a "contents.d/" subdirectory, one
file per content, instead of one per line in a "contents" file. In
the same way, service dependencies are now read in a "dependencies.d/"
subdirectory, one file per dependency. Old "contents" and "dependencies"
files are still supported, but deprecated. This change allows better
integration of s6-rc service definitions with package managers.


* s6-linux-init-1.0.7.0
   ---------------------

  - New -S option to s6-linux-init-maker, forcing a sync on halt even
in a container.


* s6-networking-2.5.1.0
   ---------------------

  - SNI wildcarding is implemented, as well as a workaround for a
bearssl bug causing errors on certificate signatures in certain cases.


  Enjoy,
  Bug-reports welcome.
  And happy holidays to you all!

--
  Laurent

Re: [announce] skarnet.org Winter 2021-2022 release

[Steve Litt]

Hi Laurent,

Congratulations on your updated software!

I searched through my execline documentation page
( http://troubleshooters.com/linux/execline.htm ) to see whether
revisions were needed. They weren't, but I found and fixed several
errors on that page.

I've been doing other things and haven't paid much attention to the
progress of s6. I haven't found any simple documentation on it. Let me
see if I understand...

* Before first installation, back up /sbin/init so if things go
  wrong it's easy to boot with your prior init

* Before running any s6 components, do all the correct config so that
  the s6-rc-compile program to produce a proper s6-rc. Do this every
  time you add or subtract services

* Grub boots to s6-linux-init-maker, which creates several things,
  including a new /sbin/init, and then execs /sbin/init

* /sbin/init does the necessary stuff about signals and the like, and
  then forks the stage1 script

* When the stage 1 script completes, /sbin/init execs the stage 2
  script, so the stage 2 script is PID1 now

* The stage 2 script forks s6-rc-init

* When s6-rc-init completes, the stage 2 script execs s6-rc, so s6-rc
  is PID1

* s6-rc supervises s6-svscan, which supervises all the other services,
  both longrun and shortrun.

The preceding is the best interpretation I could put together from
https://skarnet.org/software/s6-rc/overview.html,
https://skarnet.org/software/s6-rc/s6-rc.html, and discussions with
you. What do I need to do to make the preceding sequence accurate?

Thanks,

SteveT

Steve Litt 
Spring 2021 featured book: Troubleshooting Techniques of the Successful
Technologist http://www.troubleshooters.com/techniques



Laurent Bercot said on Tue, 21 Dec 2021 08:19:38 +0000

>  Hello,
>
>  New versions of all the skarnet.org packages are available.
>
>  The changes are, for the most part, minimal: essentially, the new
>versions  fix a bug in the build system that made cross-building under
>slashpackage more difficult than intended. Very few people should
>have been impacted by this bug.
>  Some packages had a few more bugfixes; and some packages have
>additional functionality. No major update; no compatibility break.
>
>  The new versions are the following:
>
>skalibs-2.11.1.0             (minor)
>nsss-0.2.0.1                 (release)
>utmps-0.1.1.0                (minor)
>execline-2.8.2.0             (minor)
>s6-2.11.0.1                  (release)
>s6-rc-0.5.3.0                (minor)
>s6-portable-utils-2.2.3.4    (release)
>s6-linux-utils-2.5.1.7       (release)
>s6-linux-init-1.0.7.0        (minor)
>s6-dns-2.3.5.3               (release)
>s6-networking-2.5.1.0        (minor)
>mdevd-0.1.5.1                (release)
>bcnm-0.0.1.5                 (release)
>dnsfunnel-0.0.1.3            (release)
>smtpd-starttls-proxy-0.0.1.1 (release)
>
>  Dependencies have all been updated to the latest versions. They are
> not
>strict: libraries and binaries may build with older releases of their
>dependencies, although this is not guaranteed.
>
>  You do not need to recompile your s6-rc service databases. To make
> use
>of the new s6-linux-init functionality, however, you will have to
>recreate your run-image.
>  You do not need to restart your supervision tree, unless you're 
>deleting
>your old s6 binaries.
>
>  Details of minor package changes follow.
>
>* skalibs-2.11.1.0
>   ----------------
>
>  - New function: opendir_at()
>
>
>* utmps-0.1.1.0
>   ------------
>
>  - New binary: utmps-write, a generic utmp client that can write
>user-crafted records to the utmp and/or wtmp databases.
>
>
>* execline-2.8.2.0
>   ----------------
>
>  - New -s option to the case binary, enabling fnmatch() (shell)
>expression matching instead of regular expression matching.
>
>
>* s6-rc-0.5.3.0
>   -------------
>
>  - Bundle contents are now read in a "contents.d/" subdirectory, one
>file per content, instead of one per line in a "contents" file. In
>the same way, service dependencies are now read in a "dependencies.d/"
>subdirectory, one file per dependency. Old "contents" and
>"dependencies" files are still supported, but deprecated. This change
>allows better integration of s6-rc service definitions with package
>managers.
>
>
>* s6-linux-init-1.0.7.0
>   ---------------------
>
>  - New -S option to s6-linux-init-maker, forcing a sync on halt even
>in a container.
>
>
>* s6-networking-2.5.1.0
>   ---------------------
>
>  - SNI wildcarding is implemented, as well as a workaround for a
>bearssl bug causing errors on certificate signatures in certain cases.
>
>
>  Enjoy,
>  Bug-reports welcome.
>  And happy holidays to you all!
>
>--
>  Laurent
>

Re: [announce] skarnet.org Winter 2021-2022 release

[Carlos Eduardo]

From my own experience setting up s6 on a Linux distro where it isn't supported:

> Before first installation, back up /sbin/init so if things go wrong it's easy to boot with your prior init. Before running any s6 components, do all the correct config so that the s6-rc-compile program to produce a proper s6-rc. Do this every time you add or subtract services.

Correct.

> Grub boots to s6-linux-init-maker, which creates several things, including a new /sbin/init, and then execs /sbin/init

s6-linux-init maker is actually run by the user as part of the process
of setting up s6. It generates a /etc/s6-linux-init/current folder,
including the contents of an initial, barebones /run folder
(./run-image/), the "stage" scripts
(./scripts/rc.{init,shutdown,shutdown.final}) and ./bin subfolder with
poweroff, shutdown, halt, reboot, and the stars of the show, init and
telinit. The latter must be symlinked or copied to /sbin. A possible
way to not overwrite the current init is symlinking them with a
prefix, for instance /sbin/s6-init →
/etc/s6-linux-init/current/bin/init.

> * /sbin/init does the necessary stuff about signals and the like, and  then forks the stage1 script

I think trying to explain s6-linux-init + s6-rc through the lens of
runit's stages isn't a good idea.

Unlike runit, the stage 1 is directly spawned by the kernel as init.
In addition to handling signals, it sets up an initial environment,
mounts /run as an RAM filesystem and copies the contents of
/etc/s6-linux-init/current/run-image to it, and it then forks, marking
the beginning of stage 2.

In the second stage (after forking), the PID 1 becomes (execs) an
s6-svscan instance watching over /run/service, and the child ("PID 2",
so to speak) calls s6-rc to bring services up. First "s6-rc-init",
which will symlink service directories (with a down file) from the
compiled s6-rc database to /run/service, including an s6-sudo
responsible for spawning oneshots, and after that "s6-rc -u change
default", to bring the default bundle of services up.

Stage 1 is shrunk in s6, and stage 2 takes more responsibilities.
Stage 3 isn't even handled by PID1 - it's an independent
s6-linux-init-shutdownd service (which, like the catch-all logger, is
included as part of /etc/s6-linux-init/current/run-image when you run
s6-linux-init-maker, so you don't need to worry about it when
compiling the s6-rc database).

> * When s6-rc-init completes, the stage 2 script execs s6-rc, so s6-rc  is PID 1. s6-rc supervises s6-svscan, which supervises all the other services,  both longrun and shortrun.

s6-svscan is the PID 1, and supervises longrunning services.

s6-rc is itself a short lived program, living only long enough to
bring services up or down (though, it reports if oneshots fail).

Since it's not a permanent process, it relies on s6-svscan to keep
restarting services and not letting the machine's state change.
Oneshots aren't really supervised, it's expected that, if an admin
wants to undo what a oneshot did, they'd call "s6-rc -d change
$oneshot".

Em ter., 21 de dez. de 2021 às 20:54, Steve Litt
<slitt@troubleshooters.com> escreveu:
>
> Hi Laurent,
>
> Congratulations on your updated software!
>
> I searched through my execline documentation page
> ( http://troubleshooters.com/linux/execline.htm ) to see whether
> revisions were needed. They weren't, but I found and fixed several
> errors on that page.
>
> I've been doing other things and haven't paid much attention to the
> progress of s6. I haven't found any simple documentation on it. Let me
> see if I understand...
>
> * Before first installation, back up /sbin/init so if things go
>   wrong it's easy to boot with your prior init
>
> * Before running any s6 components, do all the correct config so that
>   the s6-rc-compile program to produce a proper s6-rc. Do this every
>   time you add or subtract services
>
> * Grub boots to s6-linux-init-maker, which creates several things,
>   including a new /sbin/init, and then execs /sbin/init
>
> * /sbin/init does the necessary stuff about signals and the like, and
>   then forks the stage1 script
>
> * When the stage 1 script completes, /sbin/init execs the stage 2
>   script, so the stage 2 script is PID1 now
>
> * The stage 2 script forks s6-rc-init
>
> * When s6-rc-init completes, the stage 2 script execs s6-rc, so s6-rc
>   is PID1
>
> * s6-rc supervises s6-svscan, which supervises all the other services,
>   both longrun and shortrun.
>
> The preceding is the best interpretation I could put together from
> https://skarnet.org/software/s6-rc/overview.html,
> https://skarnet.org/software/s6-rc/s6-rc.html, and discussions with
> you. What do I need to do to make the preceding sequence accurate?
>
> Thanks,
>
> SteveT
>
> Steve Litt
> Spring 2021 featured book: Troubleshooting Techniques of the Successful
> Technologist http://www.troubleshooters.com/techniques
>
>
>
> Laurent Bercot said on Tue, 21 Dec 2021 08:19:38 +0000
>
> >  Hello,
> >
> >  New versions of all the skarnet.org packages are available.
> >
> >  The changes are, for the most part, minimal: essentially, the new
> >versions  fix a bug in the build system that made cross-building under
> >slashpackage more difficult than intended. Very few people should
> >have been impacted by this bug.
> >  Some packages had a few more bugfixes; and some packages have
> >additional functionality. No major update; no compatibility break.
> >
> >  The new versions are the following:
> >
> >skalibs-2.11.1.0             (minor)
> >nsss-0.2.0.1                 (release)
> >utmps-0.1.1.0                (minor)
> >execline-2.8.2.0             (minor)
> >s6-2.11.0.1                  (release)
> >s6-rc-0.5.3.0                (minor)
> >s6-portable-utils-2.2.3.4    (release)
> >s6-linux-utils-2.5.1.7       (release)
> >s6-linux-init-1.0.7.0        (minor)
> >s6-dns-2.3.5.3               (release)
> >s6-networking-2.5.1.0        (minor)
> >mdevd-0.1.5.1                (release)
> >bcnm-0.0.1.5                 (release)
> >dnsfunnel-0.0.1.3            (release)
> >smtpd-starttls-proxy-0.0.1.1 (release)
> >
> >  Dependencies have all been updated to the latest versions. They are
> > not
> >strict: libraries and binaries may build with older releases of their
> >dependencies, although this is not guaranteed.
> >
> >  You do not need to recompile your s6-rc service databases. To make
> > use
> >of the new s6-linux-init functionality, however, you will have to
> >recreate your run-image.
> >  You do not need to restart your supervision tree, unless you're
> >deleting
> >your old s6 binaries.
> >
> >  Details of minor package changes follow.
> >
> >* skalibs-2.11.1.0
> >   ----------------
> >
> >  - New function: opendir_at()
> >
> >
> >* utmps-0.1.1.0
> >   ------------
> >
> >  - New binary: utmps-write, a generic utmp client that can write
> >user-crafted records to the utmp and/or wtmp databases.
> >
> >
> >* execline-2.8.2.0
> >   ----------------
> >
> >  - New -s option to the case binary, enabling fnmatch() (shell)
> >expression matching instead of regular expression matching.
> >
> >
> >* s6-rc-0.5.3.0
> >   -------------
> >
> >  - Bundle contents are now read in a "contents.d/" subdirectory, one
> >file per content, instead of one per line in a "contents" file. In
> >the same way, service dependencies are now read in a "dependencies.d/"
> >subdirectory, one file per dependency. Old "contents" and
> >"dependencies" files are still supported, but deprecated. This change
> >allows better integration of s6-rc service definitions with package
> >managers.
> >
> >
> >* s6-linux-init-1.0.7.0
> >   ---------------------
> >
> >  - New -S option to s6-linux-init-maker, forcing a sync on halt even
> >in a container.
> >
> >
> >* s6-networking-2.5.1.0
> >   ---------------------
> >
> >  - SNI wildcarding is implemented, as well as a workaround for a
> >bearssl bug causing errors on certificate signatures in certain cases.
> >
> >
> >  Enjoy,
> >  Bug-reports welcome.
> >  And happy holidays to you all!
> >
> >--
> >  Laurent
> >

Re: [announce] skarnet.org Winter 2021-2022 release

[Laurent Bercot]

>I think trying to explain s6-linux-init + s6-rc through the lens of
>runit's stages isn't a good idea.

  Carlos is correct - both here and in his explanation of s6-linux-init
stages.

  When designing s6-linux-init, I kept runit's "stage" terminology
because at the time it was a useful framework to see where things
should go; but in retrospect, it was probably a mistake, prone to
confusion, as exemplified by Steve's message. Even though, functionally,
the "stages" in runit and s6-l-i have similarities, the way they're
implemented and work under the hood is fundamentally different.

runit's stage 1 and stage 2 are similar, their difference is only
conventional: traditionally, runit runs all the oneshots in stage 1,
and stage 2 is just an execution of runsvdir - but a setup where
/etc/runit/1 is empty and all the oneshots are performed in
/etc/runit/2 prior to the execution of runsvdir would work as well.

s6-linux-init's stage 1 and stage 2 do not have the same similarity.
Stage 1 is the early init, running as pid 1; this is only the
/sbin/init script produced by s6-linux-init-maker, which executes
into the s6-linux-init binary (which sets up the system and execs
into s6-svscan); and users are not supposed to do anything with it.
  Stage 2, on the other hand, is the real boot sequence, running as
not-pid-1; it is only run when stage 1 has completed, which means
the system has an adequate long-running pid 1 process, a supervision
tree, and a catch-all logger - all the basic infrastructure is in
place and the services can be started. With s6-linux-init, stage 2
is where all the real work happens; and when the system's boot
sequence is complete, the stage 2 script simply exits and the
system keeps running until a shutdown command is issued.

  I want to keep the "stages" framing for s6-linux-init, because I think
it is useful: these are qualitatively different parts of the init
process. (There's a "stage 3" and a "stage 4" as well, at shutdown
time: they're handled by the s6-linux-init-shutdownd daemon.) The
framing is actually *more* useful here than in runit, where "stages"
are only sequence points and the only real hardcoded meaning is that
the system shuts down when /etc/runit/3 exits.


>>  The preceding is the best interpretation I could put together from
>>https://skarnet.org/software/s6-rc/overview.html,
>>https://skarnet.org/software/s6-rc/s6-rc.html, and discussions with
>>  you. What do I need to do to make the preceding sequence accurate?

  I don't know, honestly, Steve. At this point I cannot tell whether
you're acting in good or bad faith.

  You seem to be talking about s6 and s6-linux-init, yet only mention
the s6-rc documentation. You do not seem to have read the
https://skarnet.org/software/s6-linux-init/quickstart.html page,
which explains that s6-linux-init-maker is run offline, or the
https://skarnet.org/software/s6-linux-init/s6-linux-init.html page,
where the "Early preparation" part explains how stage 1 works. You
do not seem to have watched my FOSDEM 2017 video at
https://archive.fosdem.org/2017/schedule/event/s6_supervision/ where
I describe the various duties of an init system and how the components
in the s6 suite fill the boxes.

  Despite your claims to be interested, you have not put in s6 the
tenth of the effort you put in runit. It's been going on for ages.
You say you haven't paid much attention to the progress of s6, but
over the years the fundamentals have not changed, they've been the
same for a while now; the truth is that you have never paid much
attention to s6 at all. You come to the list once in a while and
ask a question that shows you are still lacking a basic understanding
of s6, an understanding that comes from two hours of experimenting
while having a browser open with 3-4 tabs to the documentation.
And then you seem to ignore the answers, and go away until the
next time when you come back just as helpless.

  You are clearly not dumb. So either you are a troll, or you need to
get a grip and realize that if you're really interested, you can do
the work of looking up and finding the relevant documentation,
experimenting, and finally getting the reward of feeling the pieces
of the puzzle fall into place, and acquiring the understanding that
has eluded you for so long and that you seem to crave. As a technical
writer, that is *your job*, so you can make the process easier for
other people.

  s6 is not as complex as you seem to think it is, far from it. There
are real, living people who understand how it works, and they're not
all acne-ridden nerds living in a basement. The documentation may
not be perfect, but it seems to be adequate. It lacks tutorials, yes,
but I expect tutorials to be written by *people like you*, who could
do a much better job of it than I ever would, if only they'd stop
acting like damsels in distress at the first mention of a Unix pipe.

  And if you're not interested, or simply not enough to really get
into it, that's okay too; you just need to own it and stop
breadcrumbing.

--
  Laurent

Re: [announce] skarnet.org Winter 2021-2022 release

[Casper Ti. Vector]

On Wed, Dec 22, 2021 at 08:48:26AM +0000, Laurent Bercot wrote:
>  I want to keep the "stages" framing for s6-linux-init, because I think
> it is useful: these are qualitatively different parts of the init
> process. (There's a "stage 3" and a "stage 4" as well, at shutdown
> time: they're handled by the s6-linux-init-shutdownd daemon.) The
> framing is actually *more* useful here than in runit, where "stages"
> are only sequence points and the only real hardcoded meaning is that
> the system shuts down when /etc/runit/3 exits.
 
I would like to note that the old-fashioned way to shut down -- letting
`s6-svscan' exec() into a "stage-3" script, which performs the halting
procedure -- is still available from slew and legacy s6-linux-init.
I personally find it simpler to understand, in the cost of a lack of
full compatibility with `shutdown' without relying on some `atd':
<https://skarnet.org/lists/supervision/2779.html>.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2022.09.20)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C