Re: [Announce] s6.rc: a distribution-friendly init/rc framework

Re: [Announce] s6.rc: a distribution-friendly init/rc framework

[Laurent Bercot]

>s6.rc [1] is an attempt to bridge the gap between the elegant 
>foundation
>provided by s6/s6-rc and an init/rc system that implements the main
>functionalities beneficial for distributions. s6.rc features a
>preprocessor that generates source directories for use with s6-rc from
>given templates. The preprocessing procedure is composed of multiple
>tiny passes, which makes the preprocessor not only clean and powerful,
>but also extensible. Using the preprocessor, s6.rc supports instanced
>supervision, optional dependencies, in-place `up'/`down' scripts,
>automatic connection between services and loggers, as well as
>package-specific passes that can be plugged into the preprocessing
>phase.

  Impressive work! Congratulations, Casper :)


>[1] <https://gitlab.com/CasperVector/s6rc-dotrc>.

  I think the correct URL is: https://gitlab.com/CasperVector/s6-dot-rc

  I'm feeling bad asking about this seeing the amount of work you've put
into it, and I hope there's an easy way to achieve it, but: would it
be possible for you to change the name of the project?

  Rationale:

  - I feel that "s6.rc" is very easy to confuse with "s6-rc"; not when
it's pronounced, but when it's written. The s6 ecosystem is confusing
enough as is for non-specialists; I think I should write a general
documentation page explaining what package does what (and an index
saying which package the s6-foobar binary belongs to, for every value
of foobar). If s6 is to become popular, the confusion needs to be
reduced, not entertained; and it would be easier if software such
as s6.rc would have a more distinctive name.
  For instance, "anopa" does a good job of being pretty distinctive. :)

  - I obviously have no way of enforcing this, but I would very much
like to keep control of the "s6" prefix, for clarity: every package
starting with "s6" should come from skarnet.org and have a place in
the integrated s6 ecosystem. (Not to say that s6.rc does not have its
place, far from it, but I'd like people to be able to easily make the
distinction between "core" and "community".) So you're obviously
welcome to use "s6" as *part* of the name of your package, but I'd
appreciate it if the name didn't *start* with "s6".

  A few miscellaneous notes:

  - I now recommend using socklog[1] to implement syslogd. This is
not as elegant as having one process per syslog connection, but
socklog binds to a datagram socket, so it can be used with musl.

  Having one stream per syslog client is a good thing per se because
it obsoletes the need to identify the client in every log record;
but the killer advantage would be to do away with system-wide
regular expressions for log filtering, and that's not something
we have yet. Even when you pipe "s6-ipcserver ucspilogd" into
s6-log, your s6-log script is the same for every client, so the
regexes need to be global. A real improvement would be to have
a different log script for every client connection, so the log
filtering would really be local; but I haven't yet thought about a way
to design such an architecture. That's the main reason why I haven't
much pushed for SOCK_STREAM syslog() in musl; if we can come up with a
syslogd scheme that works without any global regexes, then we'll
have a real case for SOCK_STREAM adoption. Until then, socklog works.

[1]: http://smarden.org/socklog/

  - The next piece of the s6 ecosystem that I plan to write is precisely
an integration layer for all the various existing parts: s6-linux-init,
s6, s6-rc. It will likely be called s6-frontend. It will be written
in C, and provide a user-friendly interface for setting up a complete
init system, as well as a one-stop-shop "s6" command; the goal is to
improve usability and simplicity for users who are not longtime
members of the process supervision gang.
  When I went and asked distributions what functionality they needed
in order to use s6 as their default init system, this is by far the
thing they mentioned the most. So, it's in the plans. Unfortunately,
it will be some time before I can get to it, because I need to work
on money-making projects first. There is a possibility that work on
s6-frontend will be funded, but in any case it wouldn't be before one
year or two, so don't hold your breath. In the meantime, I'm very
happy that initiatives like s6.rc come to light.

  - Everyone knows that I insist on the separation between mechanism
and policy, that I try to write software implementing mechanism, and
that I believe that policy is the realm of distributions. Unfortunately,
the success of a few pieces of software that embed a lot of policy -
systemd for instance, but also OpenRC to some extent - has shown that
this distinction isn't always a key factor in success, and on the
contrary, making distributors' jobs easier by embedding policy could
be an advantage. As such, s6-frontend will include a lot of policy
(where to put s6 service directories, how to organize them, where to
put s6-rc source and compiled databases, where the main scandir is,
etc.) However, if decisions about "where to put stuff" are easy enough
to take, decisions about "what init scripts should I provide, what
external software should I assume is present on the system" are
definitely not.

  I see that s6.rc comes with a lot of pre-written scripts, from acpid
to wpa_supplicant. Like Avery's supervision-scripts package, this is
something that I think goes above and beyond simple "policy": this is
seriously the beginning of a distribution initiative. I have no wish
at all to do this outside of a distribution, and am convinced that
the software base must come first, and then service-specific
scripts must be written in coordination with distributions that use
it; that is what I plan to do for s6-frontend in coordination with
Adélie or Alpine (which are the most likely to use it first). But there
is a large gray area here: what is "reusable policy" (RP) and what is
"distribution-specific" (DS)? For instance, look at the way the
network is brought up - in s6.rc, in OpenRC, in sysvinit scripts,
in systemd. Is bringing up the network RP or DS? If it involves
choosing between several external software packages that provide
equivalent functionality, is it okay to hardcode a dependency, or
should we provide flexibility (with a complexity cost)?

  This is very much the kind of discussion that I think is important
to have, that I would like to have in the relatively near future, and
since more and more people are getting experience with semi-packaging
external software, and flirting with the line between software
development and distro integration - be it Olivier, Avery, Casper,
Jonathan, or others - I think we're now in a good position to have it.

--
  Laurent

Re: [Announce] s6.rc: a distribution-friendly init/rc framework

[Casper Ti. Vector]

On Thu, Mar 22, 2018 at 05:10:58PM +0000, Laurent Bercot wrote:
> would it be possible for you to change the name of the project?

What about using "slew.rc" and changing the installation path from
`/etc/s6' to `/etc/slew'?  The change is not exactly trivial but already
much smaller than the `/etc/s6-init' / `/etc/s6-rc' merge I did before
releasing this project, and can be done in a few hours (the project itself
is not complex; the complexity on my part comes from the several machines
where it is delpoyed and has to be migrated :().

(Hint for the etymology: note how "6" is pronounced in Chinese ;)

>  - I now recommend using socklog[1] to implement syslogd.

I see.

> It will likely be called s6-frontend. It will be written in C, and
> provide a user-friendly interface for setting up a complete init
> system, as well as a one-stop-shop "s6" command; the goal is to
> improve usability and simplicity for users who are not longtime
> members of the process supervision gang.

To be honest, I find the idea not very appealing to me.  I did not want
to say about this before, but now I find it necessary before more energy
is spent in goals that I consider less meaningful.  Fortunately, only
`s6-linux-init-maker' is affected by this issue as of now, and I will
use it as an example.

I think the direct downstream of init/rc systems is distributions, not
end users which demand "usability" in the usual sense (actually, one of
the many problems with systemd is that it attempts to bypass distros
and directly provide "unified" features to end users).  After s6/s6-rc
becomes mainstream, `s6-linux-init-maker' will perhaps have, IMHO, an
embarrassing role: most end users would begin with scripts provided by
their distros, which might often be very different from those generated
by `s6-linux-init-maker'.

And the difference is not only in the utilities used, but also in the
functionalities offered: eg. conditional (re-)mounting of `/run'
according to results of `mountpoint -q /run', loading/saving the clock,
saving the "uncaught" logs.  This is honestly not to show off the
features provided by my project, but to show that attempts to encapsule
factors in real-world use cases of an init/rc system into a code
generator can be largely unproductive.

Instead, I think a better way is to provide a full "reference
implementation" of what you have in mind, with the code generation knobs
converted into comments (cf. the `devtmpfs' line in `rc.boot' in my
project) or notes in the documentation.  Let distros (and some
adventurous users), i.e. those who understand their requirements for
init/rc systems better, decide how to customise their systems: after
all, text editors are the ultimate code generators for plain text files.
If it is agreed that the Unix philosophy is essentially about minimising
the total complexity of a system, I guess people would consider this to
be the more Unix-ish way.

> However, if decisions about "where to put stuff" are easy enough
> to take, decisions about "what init scripts should I provide, what
> external software should I assume is present on the system" are
> definitely not.

> But there is a large gray area here: what is "reusable policy" (RP)
> and what is "distribution-specific" (DS)?

First of all, I would like to note that the files in my project are not
to be used exactly as is; instead, they are intended as "reasonable
approximations" considering a majority of use cases.  I currently use
Alpine and Void on my servers and desktops, and I deviate from the
published version of my project, more or less, on each and every
machine, and their versions differ from each other.  I consider this
phenomenon normal and healthy.

Considering the adoption of systemd, I think most longrun definitions
are reusable, at least in an approximate sense; oneshots can be more
volatile, but most are still reusable with a reasonable amount of
distro-specific customisation.  I do not consider package dependencies
to be a big problem: distros (and those adventurous users) will
naturally handle them when customising the init/rc system.  No better
way has been proposed to my knowledge: `s6-linux-init-maker' basically
solves it by depending on skaware, and systemd "solves" it by using its
own extremely bloated implementations.

> If it involves choosing between several external software packages
> that provide equivalent functionality, is it okay to hardcode a
> dependency, or should we provide flexibility (with a complexity cost)?

Regarding networking, I consider netifrc to be, though bloated, a
successful example.  As I mentioned, the functionalities of netifrc can
in principle be implemented using preprocessors in much cleaner ways,
and I guess the complexity cost that comes with the flexibility would
not be too big in this case.

Of course, this means you probably cannot use C exclusively.  If Unix
used a small language with S-expressions that can be both compiled and
interpreted, and with garbage collection enabled at
compile/interpret-time but optional for compiled binaries, we would be
perhaps free from those "flexible {whatever language} or efficient C"
dilemmas.  Unfortunately, no such language exists, at least to my
knowledge.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

slew: renamed from "s6.rc"

[Casper Ti. Vector]

On Fri, Mar 23, 2018 at 12:00:22PM +0800, Casper Ti. Vector wrote:
> What about using "slew.rc" and changing the installation path from
> `/etc/s6' to `/etc/slew'?  The change is not exactly trivial but already
> much smaller than the `/etc/s6-init' / `/etc/s6-rc' merge I did before
> releasing this project, and can be done in a few hours (the project itself
> is not complex; the complexity on my part comes from the several machines
> where it is delpoyed and has to be migrated :().

Done; new project URL: <https://gitlab.com/CasperVector/slew>.
(You will be redirected if you visit the `s6-dot-rc' URL.)

The changes turned out to be very simple: batch changing `/etc/s6' to
`/etc/slew' (including the symlinks in `run/service/.s6-svscan'),
`s6.rc' (filename) to `slew.rc', and `s6.rc' (project name) to `slew';
the rest were cosmetics.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: [Announce] s6.rc: a distribution-friendly init/rc framework

[Laurent Bercot]

>What about using "slew.rc" and changing the installation path from
>`/etc/s6' to `/etc/slew'?

  That's all fine with me, but it may have connotations in English
that you don't want to associate with a project aimed at stability
and friendliness :)


>To be honest, I find the idea not very appealing to me.  I did not want
>to say about this before, but now I find it necessary before more 
>energy
>is spent in goals that I consider less meaningful.  Fortunately, only
>`s6-linux-init-maker' is affected by this issue as of now, and I will
>use it as an example.
>
>I think the direct downstream of init/rc systems is distributions, not
>end users which demand "usability" in the usual sense (actually, one of
>the many problems with systemd is that it attempts to bypass distros
>and directly provide "unified" features to end users).  After s6/s6-rc
>becomes mainstream, `s6-linux-init-maker' will perhaps have, IMHO, an
>embarrassing role: most end users would begin with scripts provided by
>their distros, which might often be very different from those generated
>by `s6-linux-init-maker'.

  Yes, but I don't see this as a problem. The target audience for
s6-frontend would precisely be *distributions*, not end users; the
goal is to factor the work of putting all the s6 pieces together and
make it easier for distributions to tailor those pieces to their own
policies. For instance, instead of having to provide s6-rc source
definition directories for services, they would provide a descriptive
file, similar to systemd unit files, that would then be automatically
processed and turned into a set of s6-rc definition directories. So
they can focus on the functionality they want to achieve in the
service without having to learn the nuts and bolts of the s6
mechanisms.

  Bear in mind that - this is a simplified, but descriptive enough view
of the political landscape of the current Linux ecosystem - distribution
maintainers are *lazy*. They already know systemd, or openrc, or
sysvinit; they don't want to put in more effort. They don't want to have
to significantly modify their packaging scripts to accommodate a wildly
different format. They don't want to have to learn how things operate
under the hood. Ideally, they don't want to have to come up with
policy decisions themselves!
  You may not find the idea of s6-frontend appealing, but distro
maintainers really do. I've had distro maintainers say to me verbatim
that the *main*, and in some cases *only*, obstacle to s6 adoption was
its interface and how different it is from what they already have; but
apart from that, they love the idea.

  Remember that a few years ago I started a quest to learn what kind
of functionality people wanted in s6 in order for them to switch to
it? I expected technical answers, I expected weird systemd features;
I got literally none of those. What I did get, though, is requests
for a friendly UI and accessibility for "average" users (read: lazy
distro maintainers included). This is not the answer I wanted, but
it's the crystal clear answer I got; so it would be foolish of me to
ignore it. I don't like it any more than you do, and interface work
would definitely not be my first choice, but there it is.

  However, I'm absolutely committed to keeping the individual s6
pieces independent and maintaining them separately, and the frontend
will always be optional. So, power users like you will always be able
to use s6-linux-init, s6 and s6-rc separately, with your own scripting
around them; slew.rc or similar initiatives will not be obsoleted.
I'm not turning s6 into a monolithic behemoth, don't worry. ;)


>And the difference is not only in the utilities used, but also in the
>functionalities offered: eg. conditional (re-)mounting of `/run'
>according to results of `mountpoint -q /run', loading/saving the clock,
>saving the "uncaught" logs.  This is honestly not to show off the
>features provided by my project, but to show that attempts to encapsule
>factors in real-world use cases of an init/rc system into a code
>generator can be largely unproductive.

  Well, obviously the frontend should offer knobs and settings to
allow customization of those aspects. And in the worst case, there
should be a "run this user-provided script as is" function, which
allows users to bypass the UI and provide snippets of code themselves.


>Instead, I think a better way is to provide a full "reference
>implementation" of what you have in mind, with the code generation 
>knobs
>converted into comments (cf. the `devtmpfs' line in `rc.boot' in my
>project) or notes in the documentation.

  These two approaches are not mutually exclusive. :)


>  I do not consider package dependencies
>to be a big problem: distros (and those adventurous users) will
>naturally handle them when customising the init/rc system.  No better
>way has been proposed to my knowledge: `s6-linux-init-maker' basically
>solves it by depending on skaware, and systemd "solves" it by using its
>own extremely bloated implementations.

  That is precisely the discussion that I want to have. To a
distribution, dependencies are not a problem because they know what
they are packaging and they are responsible for their integration
choices. To a software author, however, dependencies are an issue -
because I can take responsibility for software I provide, but I
cannot take responsibility for external software; this is why
s6-linux-init depends on skaware, because I cannot depend on, say, a
coreutils implementation - but I'm well aware that distributions will
choose to depend on some coreutils instead, and they will be right;
so obviously distributions will probably not depend on the verbatim
output of s6-linux-init-maker, and that's okay.
  systemd has the exact same approach to external dependencies: it
provides all the functionality itself so it doesn't depend on anything
else.

  The difficult part is that writing an integrated init system is
riding the fine line between "software author" and "distro packager".
At one end of the spectrum, you have s6, which is pure mechanism,
self-reliant, but that needs to be paired with other programs to
get a complete turnkey init system; at the other end, you have
kitchen sinks like systemd that include everything they need by
reinventing the square wheel. To provide a reasonably lightweight
but immediately usable and distro-adoptable init system, there is
a golden mean to be found, and I think it's important to find it.


>Regarding networking, I consider netifrc to be, though bloated, a
>successful example.  As I mentioned, the functionalities of netifrc can
>in principle be implemented using preprocessors in much cleaner ways,
>and I guess the complexity cost that comes with the flexibility would
>not be too big in this case.

  But is it the software's job to determine the format of the
network configuration file, or is it purely the distribution's?
Is it an advantage for an init system to come with its own
network configuration format, or a drawback? Is it possible, and
worth it, to write hooks that call pre-existing network scripts,
or should the whole network interface config mess be torn down and
rebuilt from scratch? Those are the exact kind of questions we need
to ask ourselves.


>Of course, this means you probably cannot use C exclusively.  If Unix
>used a small language with S-expressions that can be both compiled and
>interpreted, and with garbage collection enabled at
>compile/interpret-time but optional for compiled binaries, we would be
>perhaps free from those "flexible {whatever language} or efficient C"
>dilemmas.  Unfortunately, no such language exists, at least to my
>knowledge.

  Language is an implementation detail. I usually go for C because
it doesn't depend on anything and keeps things lightweight, so it's a
natural choice for low-level software, but obviously when doing
integration you need a lot of scripting. I will go with autogenerated
execline scripts, in the spirit of s6-rc; you chose to use the rc shell,
which is a small, acceptable dependency; all this is fine, and it's
really not worth spending time on.

--
  Laurent

Re: [Announce] s6.rc: a distribution-friendly init/rc framework

[Alex Suykov]

Fri, Mar 23, 2018 at 10:51:57AM +0000, Laurent Bercot wrote:

>  Bear in mind that - this is a simplified, but descriptive enough view
> of the political landscape of the current Linux ecosystem - distribution
> maintainers are *lazy*. They already know systemd, or openrc, or
> sysvinit; they don't want to put in more effort. They don't want to have
> to significantly modify their packaging scripts to accommodate a wildly
> different format.

In their defence, I don't think any mainstream distribution makes this
kind of modifications easy. IMO it's safe to assume a new init system
means a new distribution (possibly derived from something larger).

> For instance, instead of having to provide s6-rc source definition
> directories for services, they would provide a descriptive file,
> similar to systemd unit files, that would then be automatically
> processed and turned into a set of s6-rc definition directories.

Extra layers generally make things harder to work with, not easier.

Whoever may be building something new with s6 would probably benefit
more from a reference system that's simple enough to understand
and which could be used as a base for extending into a full blown
distribution. Having examples at hand would likely matter much more
than the format of service description.

The reaction to the slew manual I'm afraid will likely be along the
lines of "that's all cool and stuff but how do I actually run this?".

Re: [Announce] s6.rc: a distribution-friendly init/rc framework

[Casper Ti. Vector]

On Fri, Mar 23, 2018 at 10:51:57AM +0000, Laurent Bercot wrote:
>  That's all fine with me, but it may have connotations in English
> that you don't want to associate with a project aimed at stability
> and friendliness :)

See also project names like "git", "curses" and "snort"... :)

>  Bear in mind that - this is a simplified, but descriptive enough view
> of the political landscape of the current Linux ecosystem - distribution
> maintainers are *lazy*.

Now I understand.  What a pity for distro developers / users and us.

> I'm not turning s6 into a monolithic behemoth, don't worry. ;)

I (and probably many others) never worried about this :)

> To provide a reasonably lightweight but immediately usable and
> distro-adoptable init system, there is a golden mean to be found, and I
> think it's important to find it.

Well, slew.rc does not technically require much more than those provided
by what are usually shipped with the base system in common distros,
except for rc(1) and s6/s6-rc/execline.  But it requires the user to be
sufficiently familiar with s6/s6-rc and rc(1), so the main problem is
what you called the current political landscape.

>  But is it the software's job to determine the format of the
> network configuration file, or is it purely the distribution's?
> Is it an advantage for an init system to come with its own
> network configuration format, or a drawback? Is it possible, and
> worth it, to write hooks that call pre-existing network scripts,
> or should the whole network interface config mess be torn down and
> rebuilt from scratch? Those are the exact kind of questions we need
> to ask ourselves.

Good questions, and I certainly need more time to think about them.
I guess that, for the slew.rc-based netifrc replacement I imagined,
assuming it is sufficiently well-designed, it would not be too
difficult to port it to another daemontools-like init/rc framework
according to the underlying logic.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: [Announce] s6.rc: a distribution-friendly init/rc framework

[Casper Ti. Vector]

On Fri, Mar 23, 2018 at 03:05:53PM +0200, Alex Suykov wrote:
> The reaction to the slew manual I'm afraid will likely be along the
> lines of "that's all cool and stuff but how do I actually run this?".

I again confess that I am not good at writing tutorials; the current
manual is really more suitable for those who know the backgrounds of
s6/s6-rc.  For a reference system, see the attached tarball and online
VM image from the first message in this thread; the only difference
between its system and slew is that all references to `/etc/s6' /
`/etc/s6/lib/s6.rc' should now be `/etc/slew' /
`/etc/slew/lib/slew.rc'.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: [Announce] s6.rc: a distribution-friendly init/rc framework

[Laurent Bercot]

>In their defence, I don't think any mainstream distribution makes this
>kind of modifications easy. IMO it's safe to assume a new init system
>means a new distribution (possibly derived from something larger).

  And that is why I intend to start with smaller, more flexible,
less inertia-driven distributions such as Alpine and Adélie (and
maybe Void or Manjaro or... someday). Mainstream distributions will
only show interest when the work has already been done somewhere else,
multiple times, and has been proven to be significantly better than
what they already have.


>Extra layers generally make things harder to work with, not easier.

  It really depends on the quality of the automation here. If the
abstraction is leaky and you have to look into the lower layer, then
yes, the upper layer is a hindrance more than anything. But if the
abstraction is airtight, as I intend to make it, then people can just
work with the upper layer and not worry about the inner mechanisms.
The inner mechanisms are still there and tweakable for power users,
but the point is to get a smooth, simplified interface on top.


>Whoever may be building something new with s6 would probably benefit
>more from a reference system that's simple enough to understand
>and which could be used as a base for extending into a full blown
>distribution. Having examples at hand would likely matter much more
>than the format of service description.

  That is what I would naturally lean towards, but it is explicitly
*not* what I was told by distro maintainers. You need to be realistic
here and admit that file-based configuration with key=value lines is
more human-friendly than directory-based configuration. Old habits
die hard, and people who are used to the systemd or openrc way of doing
things just do not want to completely switch their mental model and
deal with something with directories and small files everywhere, and
what the heck is a service directory and why can't you tune your
daemon settings in a config file like everyone else.

  Fortunately, the s6 and s6-rc formats are easy to autogenerate; some
may even argue that they were *purposefully* designed to be easy to
autogenerate. Who knows. So, slapping a UI on top of them is, like,
exactly the kind of thing they're good at, so if that's what people 
want,
there's no reason not to give people what they want.


>The reaction to the slew manual I'm afraid will likely be along the
>lines of "that's all cool and stuff but how do I actually run this?".

  That is exactly most people's reaction when they first discover the
supervision paradigm. You need to somewhat know how it works to fully
appreciate its quality.

--
  Laurent

Re: [Announce] s6.rc: a distribution-friendly init/rc framework

[Casper Ti. Vector]

On Fri, Mar 23, 2018 at 09:20:58PM +0800, Casper Ti. Vector wrote:
> Now I understand.  What a pity for distro developers / users and us.
On a second thought, what about (at least a attempt at) solving the
human (political) problems by human means (propaganda, but of the
factually correct type)?

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: [Announce] s6.rc: a distribution-friendly init/rc framework

[Laurent Bercot]

>On a second thought, what about (at least a attempt at) solving the
>human (political) problems by human means (propaganda, but of the
>factually correct type)?

  It's not a political problem, it's a religious problem. You just cannot
convince people that your UI is better, any more than you can convince
them that your text editor is better. People have tastes, we may find
them bad, but we're not going to change them; trying only paints you
as an annoying zealot, and is counter-productive.

  The factually correct, technically correct solution is already there,
and they accept it, they believe it - we don't need to convince them
of that. It's simply a UI issue. And the right answer to "I like your
engine, but I don't like your UI" isn't "You're wrong, my UI is the
best", it is to put up - and write a UI they like - or shut up.
A major selling point of the s6 and s6-rc formats is that they're easy
to autogenerate, and a frontend would be proof of that. We claim we're
technically better than everyone else, and that our paradigm is more
flexible than others - well there's the opportunity to prove it.

--
  Laurent

Re: [Announce] s6.rc: a distribution-friendly init/rc framework

[Casper Ti. Vector]

On Sun, Mar 25, 2018 at 06:38:33AM +0000, Laurent Bercot wrote:
> A major selling point of the s6 and s6-rc formats is that they're easy
> to autogenerate, and a frontend would be proof of that. We claim we're
> technically better than everyone else, and that our paradigm is more
> flexible than others - well there's the opportunity to prove it.
Fine if you think so.  Religions can really be annoying and wasteful.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: [Announce] s6.rc: a distribution-friendly init/rc framework

[Casper Ti. Vector]

On Thu, Mar 22, 2018 at 05:10:58PM +0000, Laurent Bercot wrote:
>  Having one stream per syslog client is a good thing per se because
> it obsoletes the need to identify the client in every log record;
> but the killer advantage would be to do away with system-wide
> regular expressions for log filtering, and that's not something
> we have yet. Even when you pipe "s6-ipcserver ucspilogd" into
> s6-log, your s6-log script is the same for every client, so the
> regexes need to be global. A real improvement would be to have
> a different log script for every client connection, so the log
> filtering would really be local; but I haven't yet thought about a way
> to design such an architecture. That's the main reason why I haven't
> much pushed for SOCK_STREAM syslog() in musl; if we can come up with a
> syslogd scheme that works without any global regexes, then we'll
> have a real case for SOCK_STREAM adoption. Until then, socklog works.
 
Assuming the number of syslog logging scripts is fairly small (a few for
daemons in an anticipated list, and perhaps one for the rest; I think
this scheme is actually already in use by most syslog users), what about
setting up a group of s6-log consumer services, and use a chainloading
program (akin to s6-tcpserver-access) with s6-ipcserver to dynamically
decide which consumer to connect to (by interacting with s6rc-fdholder)?

(I think this scheme, with some variations, can also be usable with
services that produce multiple output streams, like Apache which is
recently discussed on this mailing list.  This would be particularly
easy if these services can be configured to log to /proc/self/fd/[num]:
then the user can simply use s6-fdholder-retrieve to chainload the
daemons for the services.)

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: [Announce] s6.rc: a distribution-friendly init/rc framework

[Laurent Bercot]

>Assuming the number of syslog logging scripts is fairly small (a few for
>daemons in an anticipated list, and perhaps one for the rest; I think
>this scheme is actually already in use by most syslog users), what about
>setting up a group of s6-log consumer services, and use a chainloading
>program (akin to s6-tcpserver-access) with s6-ipcserver to dynamically
>decide which consumer to connect to (by interacting with s6rc-fdholder)?

Yes, that sounds good. Also, it feels similar to something I'm
proto-thinking about for instanced services ; I need to dwell on it
more, to see whether there are more common points.
Unfortunately, I don't have much time to work on it at the moment.

--
Laurent

Re: [Announce] s6.rc: a distribution-friendly init/rc framework

[Casper Ti. Vector]

On Thu, Mar 22, 2018 at 09:23:34PM +0800, Casper Ti. Vector wrote:
> [1] <https://gitlab.com/CasperVector/s6rc-dotrc>.

Should be <https://gitlab.com/CasperVector/s6-dot-rc>.  The project used
`/etc/s6-init' / `/etc/s6-rc', and `/etc/s6/lib/s6.rc' used to be
`/etc/s6-rc/lib/s6-rc.rc', hence the old name.  Many thank to mobinmob
for notifying me of this :)

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

[Announce] s6.rc: a distribution-friendly init/rc framework

[Casper Ti. Vector]

[-- Attachment #1: Type: text/plain, Size: 3300 bytes --]

s6.rc [1] is an attempt to bridge the gap between the elegant foundation
provided by s6/s6-rc and an init/rc system that implements the main
functionalities beneficial for distributions.  s6.rc features a
preprocessor that generates source directories for use with s6-rc from
given templates.  The preprocessing procedure is composed of multiple
tiny passes, which makes the preprocessor not only clean and powerful,
but also extensible.  Using the preprocessor, s6.rc supports instanced
supervision, optional dependencies, in-place `up'/`down' scripts,
automatic connection between services and loggers, as well as
package-specific passes that can be plugged into the preprocessing
phase.

[1] <https://gitlab.com/CasperVector/s6rc-dotrc>.

The init part of s6.rc is, in many aspects, similar to what one would
expect from s6-linux-init.  The stage 1/2/3 scripts shipped with s6.rc
additionally support features like setting the time using hwclock(8) or
other means before exec()ing `s6-svscan' (cf. the documentation for the
rationale), duplicating (while writing to `/run/uncaught-logs') verbose
`s6-rc' output to `/dev/console', automatic reboot under certain
circumstances (eg. when required by fsck(8)), and automatically saving
`/run/uncaught-logs/current' on shutdown.

The attached tarball is an s6.rc-based example setup tailored for Alpine
3.7.x; a VM image of the setup is available at [2].  Due to a lack of
expressiveness in execline, most scripts in s6.rc are written for the
Byron Rakitzis implementation [3] of the rc(1) shell from Plan 9, and
static-linked binaries for x86 [4] and x86_64 [5] are available for
download.  See the attachments for SHA512 checksums (signed using my
OpenPGP public key) of the tarball, the VM image and the rc(1) binaries.

[2] <https://drive.google.com/open?id=1x3BM11C5clU9DTGbgsBmaBssMUtMUItM>.
[3] <https://github.com/rakitzis/rc>.
[4] <https://drive.google.com/open?id=0B3FGvKEMCkmXLUJDQll2VFNsNVE>.
[5] <https://drive.google.com/open?id=0B3FGvKEMCkmXMm9odWhVdVR1Znc>.

Of course, this is just the beginning; a lot of work has to be done:

* Although s6.rc can be easily configured to be rock solid, it is fairly
  fragile to PEBKACs: eg. if you accidentally delete `s6.rc' and then
  reboot the system, the system can get into quite serious troubles.
  Sprinkling related scripts with a lot of "if something is unset then
  default to blah" clauses can avoid the problem, but seems to make
  the code bloated.  Is there a better solution?

* Many more service definitions and ancillary files are necessary for
  s6.rc to be useful for the general public instead of just a few users,
  so any contribution / suggestion is welcome.  BTW, I do not consider
  myself to be a really good documentation writer, so please tell me if
  you have a good idea on improving the s6.rc documentation.

* In principle, it is possible to implement something like Gentoo's
  netifrc [6] using s6.rc's preprocessor, and I believe it would be much
  cleaner than netifrc.  However, since I am not exactly familiar with
  the networking stuffs, perhaps it had better be done by someone else.

[6] <https://wiki.gentoo.org/wiki/Netifrc>.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C


[-- Attachment #2: alpine-s6rc-conf.tgz --]
[-- Type: application/x-gtar, Size: 14022 bytes --]

[-- Attachment #3: SHA512SUMS.asc --]
[-- Type: application/pgp-signature, Size: 1474 bytes --]