runit patches to fix compiler warnings on RHEL 7

runit patches to fix compiler warnings on RHEL 7

[J. Lewis Muir]

Hello!

Is runit hosted in a public source code repo?  If so, where?

Are patches to fix runit compiler warnings on RHEL 7 welcome?

Thank you!

Lewis

Re: runit patches to fix compiler warnings on RHEL 7

[J. Lewis Muir]

On 11/25, J. Lewis Muir wrote:
> Is runit hosted in a public source code repo?  If so, where?
> 
> Are patches to fix runit compiler warnings on RHEL 7 welcome?

I have patches now.  Is there a public source code repo I can contribute
against?  Or would it be helpful to just send the patches to the list?

Lewis

Re: runit patches to fix compiler warnings on RHEL 7

[Ben Franksen]

[-- Attachment #1: Type: text/plain, Size: 791 bytes --]

Am 27.11.19 um 21:33 schrieb J. Lewis Muir:
> On 11/25, J. Lewis Muir wrote:
>> Is runit hosted in a public source code repo?  If so, where?
>>
>> Are patches to fix runit compiler warnings on RHEL 7 welcome?
> 
> I have patches now.  Is there a public source code repo I can contribute
> against?  Or would it be helpful to just send the patches to the list?

Hi Lewis

it's cool to see you are interested in runit. May I ask whether you are
using it for controls / EPICS?

AFAIK there is no public repository for runit. You may want to ping
Gerrit Pape personally to get his attention. His last message to the
list is from march this year where he said he was "looking forward to do
a maintenance release of runit eventually and [...] collecting patches".

Cheers
Ben

[-- Attachment #2: pEpkey.asc --]
[-- Type: application/pgp-keys, Size: 4279 bytes --]

Re: runit patches to fix compiler warnings on RHEL 7

[Laurent Bercot]

[-- Attachment #1: Type: text/plain, Size: 1711 bytes --]


  I am reluctant to bring this up because I am not a neutral third party,
but this is frankly a conversation that needs to be had.

  - This mailing-list accepts all discussions about process supervision
software. It also accepts patches to such software (but rather than cold
sending patches, please engage in a tech discussion first - it doesn't
have to be long.)

  - The original author or runit is still subscribed to this list, and
comes from time to time. However, I'm not aware of an official repo for
runit, and runit's latest official version has been 2.1.2 for many a 
year
now.
  It looks like several distributions have their own version of runit;
they are maintained by the distros themselves.

  - We on the list will gladly help with any question with runit, but to
be honest, I'm not exactly sure what to do with patch upstream requests
for runit. Is anyone processing them and integrating them into a new
release?

  - I host this list. I'm also the author of s6, a supervision software
suite that is very similar to runit in many ways. s6 is actively
maintained and has a public git repo, and we generally have a quick
response time with requests.

  - My opinion is that the most sustainable path forward, for runit
users who need a centrally maintained supervision software suite, is to
just switch to s6 - and it comes with several other benefits as well.

  - But again, I'm not impartial, and alternatives are a good thing.
So no matter what individual decisions are made, it would definitely be
a net positive if the exact state and workflow of runit could be
clarified, and if a real development/maintenance structure was in place.

--
  Laurent

Re: runit patches to fix compiler warnings on RHEL 7

[Steve Litt]

On Thu, 28 Nov 2019 19:04:37 +0000
"Laurent Bercot" <ska-supervision@skarnet.org> wrote:

>   - We on the list will gladly help with any question with runit, but
> to be honest, I'm not exactly sure what to do with patch upstream
> requests for runit. Is anyone processing them and integrating them
> into a new release?

For submitting patches, I'd recommend working with the Void Linux
project. They can be found at #xbps on Freenode IRC. Void Linux has
used runit as their init system for the past 5 years: Their
implementation is very reliable and mature.

> 
>   - I host this list. I'm also the author of s6, a supervision
> software suite that is very similar to runit in many ways. s6 is
> actively maintained and has a public git repo, and we generally have
> a quick response time with requests.
> 
>   - My opinion is that the most sustainable path forward, for runit
> users who need a centrally maintained supervision software suite, is
> to just switch to s6 - and it comes with several other benefits as
> well.

IMHO not necessarily. There are people whose top priority is
simplicity. They value simplicity over guaranteeing against a machine
whose supervisor has died, and is now incommunicado. They value
simplicity over PID1's ability to supervise one program; the process
supervisor (did I get that right?). Such people would prefer runit.

Additionally, if a person uses sysvinit as PID1 and only PID1, and puts
"respawn runsvdir" in /etc/inittab, then they do get PID1 supervising
the supervisor.

One other observation: If I wanted the Cadillac of the industry, I'd go
with s6. But on a day to day basis, the Chevy of the industry, runit, is
good enough for the driving I do.

Which leads to the next point: One reason runit has such a large
mindshare is because Void Linux and maybe some others ship with runit
as their init. s6 has an opportunity to leapfrog. Right now, the Devuan
project is discussing supplying run scripts for runit and for s6.
Assuming Debian ship with a working s6 (only has to work as a
supervisor: sysvinit could be PID1), if the s6 run scripts arrive
first, I think s6 would be in a position to become Devuan's default
supervisor a year or two from now. I spoke the preceding sentence as an
individual, not as a member of the Devuan community. Anyway, if you
have a bunch of known-good s6 run (and finish) scripts curated
somewhere, everyone would be pleased if you let the Devuan user mailing
list know where they are.

Thanks,

SteveT

Steve Litt
November 2019 featured book: Manager's Guide to Technical
Troubleshooting Second edition
http://www.troubleshooters.com/mgr

runit or s6 (was: runit patches to fix compiler warnings on RHEL 7)

[Laurent Bercot]

>For submitting patches, I'd recommend working with the Void Linux
>project. They can be found at #xbps on Freenode IRC. Void Linux has
>used runit as their init system for the past 5 years: Their
>implementation is very reliable and mature.

Yes, but OP wants to fix compiler warnings on RHEL7. Do you think Red 
Hat
uses Void as their runit upstream?


>IMHO not necessarily. There are people whose top priority is
>simplicity.

You love to prop up the discourse that runit is a lot simpler than s6,
but I reject this assumption, and you're doing a disservice to me *and*
to users by propagating it. Core runit and core s6 are very similar,
and the complex parts of s6 are entirely optional.

The part where runit is definitely simpler than s6 is when both are used
as init systems, which is entirely optional too. Again, OP is talking
about RHEL7: what are the odds runit is used as an *init system* on 
RHEL7?
so the bulk of the "simplicity" argument falls here.


>Which leads to the next point: One reason runit has such a large
>mindshare is because Void Linux and maybe some others ship with runit
>as their init. s6 has an opportunity to leapfrog. Right now, the Devuan
>project is discussing supplying run scripts for runit and for s6.

The Devuan people know me well. They know where to contact me. They know
I'm interested in helping as much as I can if they choose s6. I haven't
heard from them. So, I'm assuming they're not really interested.
If I'm wrong, my mailbox is open.


>Anyway, if you
>have a bunch of known-good s6 run (and finish) scripts curated
>somewhere, everyone would be pleased if you let the Devuan user mailing
>list know where they are.

  It's been the same tune for years, so please believe that I know it 
well.
*Everybody* is in love with s6 as long as everything is turnkey (read: 
as
long as software authors, who are supposed to provide mechanism, are 
also
providing policy, i.e. doing the work of the distribution). But as soon
as the distribution actually has to *do its job*, stop being lazy, and
write policy scripts, suddenly there's no one around. Crickets.

  I've learned the lesson. I will provide a complete set of init scripts.
And I'm slowly getting to the point where I'll be able to do it. But 
I'll
still need 2 more years, because I need to feed myself, too.

  If more people/distros were *really* interested in this, instead of 
just
nodding their heads and paying lip service as long as they don't have to
lift a finger (yes, I'm looking at you too, Steve), it would go a lot
faster. It would most likely already be done.

  You can help by cutting the nagging.

--
  Laurent

Re: runit patches to fix compiler warnings on RHEL 7

[Jan Braun]

[-- Attachment #1: Type: text/plain, Size: 2358 bytes --]

Hi,

Laurent Bercot schrob:
>  - My opinion is that the most sustainable path forward, for runit
> users who need a centrally maintained supervision software suite, is to
> just switch to s6 - and it comes with several other benefits as well.

As a relatively new convert to supervision software, my reasons for
preferring runit over s6 are, in order of priority:

1) Debian ships with a working and maintained runit-init package. It
   provides pid 1 and hooks into /etc/rcS.d/* to integrate with other
   Debian packages. s6-linux-init and s6-rc are not packaged in Debian.

2) runit has manpages. s6 has HTML. :(

3) s6 executables are somehow worse named than runit's. This may be
   highly subjective, but I can recall and recognize the runit commands
   far easier than the s6 ones. Possibly it's the "s6-" prefix getting
   in the way of my brain pattern matching on visual appearance of glyph
   sequences.
   This point is exacerbated by #2 and the number of s6 executables.
   Compare chpst with s6-envdir s6-envuidgid s6-fghack s6-setsid
   s6-setuidgid s6-applyuidgid s6-softlimit. Yes, I know about the
   historical reasons, but still.

4) s6 seems more complex (hello execline!), and I don't (yet?) see any
   benefit/feature I'd appreciate except minimizing wakeups.

OTOH, an active and responsive upstream is obviously a big plus for s6.

>  - But again, I'm not impartial, and alternatives are a good thing.
> So no matter what individual decisions are made, it would definitely be
> a net positive if the exact state and workflow of runit could be
> clarified, and if a real development/maintenance structure was in place.

Agreed.

Brainstorming possible ways forward:

A) Gerrit Pape becomes more active in maintianing runit, at least
   acknowledging patches posted here.
B) Somebody else steps in as (co-)maintainer.
C) We get a dumping ground (wiki or somesuch) for patches to allow
   - contributors to publish their patches (after discussing them here)
   - users to easily find and download patches they'd be interested in
   - Gerrit Pape to review and apply patches at his leisure when he
     feels like making a new release.
D) The maintainers of distros shipping runit work out a patch-sharing
   scheme among them.


Just my 0.02€, I hope it helps.

cheers,
    Jan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: runit patches to fix compiler warnings on RHEL 7

[Dewayne Geraghty]

Jan,

I'm also a virgin to process/service management software, learning 
s6-rc, s6, execlineb is not for the faint-hearted nor the time-poor. 
Getting a handle on the concepts, and the naming conventions - its 
really hard work.

Execline enforces a discipline, a rigor demanding anticipatory planning 
(to get right).  I ran some performance tests and execlineb is 
marginally better.  So why persist?  Largely because an execline script 
is immediately obvious and explicit.  Seriously, at a glance you know 
what the script will achieve.  Could I write a sh script to do the same 
task?  Yes, and probably do it a lot quicker.  But.  I would loose the 
elegance and readability - where sh has an equivalence with assembler, 
execline is akin to BASIC, it makes you think differently :)

I'm developing solutions for PROTECTED level security (its an Australian 
Govt thing), and skarnet's service management provides assurance, and 
s6-log provides near-certainty of logging completeness. I'm very happy 
with the toolset, worth the time investment.

Re: runit patches to fix compiler warnings on RHEL 7

[Colin Booth]

On Fri, Nov 29, 2019 at 03:09:01PM +0100, Jan Braun wrote:
> Hi,
> 
> Laurent Bercot schrob:
> >  - My opinion is that the most sustainable path forward, for runit
> > users who need a centrally maintained supervision software suite, is to
> > just switch to s6 - and it comes with several other benefits as well.
> 
> As a relatively new convert to supervision software, my reasons for
> preferring runit over s6 are, in order of priority:
> 
> 1) Debian ships with a working and maintained runit-init package. It
>    provides pid 1 and hooks into /etc/rcS.d/* to integrate with other
>    Debian packages. s6-linux-init and s6-rc are not packaged in Debian.
runit-init is slowly becoming less functional and it wouldn't surprise
me if it fails entirely after Debian 10. As for s6-rc and s6-l-i, you
don't need either of those to emulate runit-init and if you do want them
you should talk to the current maintainer of s6 and execline about
adding them.

The rcS.d hooks can be covered with a shim program, though the LSB
compatibility layer in runit is pretty poor to start with and I tend to
suggest people avoid it if they can help it.
> 
> 2) runit has manpages. s6 has HTML. :(
Yeah, this sucks. I know Laurent isn't going to do it but I've been
meaning to take some time off and dedicate it to rewriting all of the
documentation into something that an compile into both mandoc and html.
> 
> 3) s6 executables are somehow worse named than runit's. This may be
>    highly subjective, but I can recall and recognize the runit commands
>    far easier than the s6 ones. Possibly it's the "s6-" prefix getting
>    in the way of my brain pattern matching on visual appearance of glyph
>    sequences.
>    This point is exacerbated by #2 and the number of s6 executables.
>    Compare chpst with s6-envdir s6-envuidgid s6-fghack s6-setsid
>    s6-setuidgid s6-applyuidgid s6-softlimit. Yes, I know about the
>    historical reasons, but sti
chpst is a monster of a program and at least with runscripts written in
execline it's generally easier to understand 3-4 process state
manipulators than a pile of chpst options.
> 
> 4) s6 seems more complex (hello execline!), and I don't (yet?) see any
>    benefit/feature I'd appreciate except minimizing wakeups.
s6 is more complex in terms of total bits than runit, but that
complexity brings objective benefits like having the supervision system
itself know when supervised services are ready and being able to query
the supervisor for that status. If you're talking about comparisons with
the supervisory portions of runit, besides the differences with chpst
mentioned earlier the two are pretty comparible. As for the execline
dependency, the only place where it's really required is if you use
s6-log processing directives though it's a pretty nice language for
writing run scripts in.
> 
> OTOH, an active and responsive upstream is obviously a big plus for s6.
> 
> >  - But again, I'm not impartial, and alternatives are a good thing.
> > So no matter what individual decisions are made, it would definitely be
> > a net positive if the exact state and workflow of runit could be
> > clarified, and if a real development/maintenance structure was in place.
> 
> Agreed.
> 
> Brainstorming possible ways forward:
> 
> A) Gerrit Pape becomes more active in maintianing runit, at least
>    acknowledging patches posted here.
> B) Somebody else steps in as (co-)maintainer.
> C) We get a dumping ground (wiki or somesuch) for patches to allow
>    - contributors to publish their patches (after discussing them here)
>    - users to easily find and download patches they'd be interested in
>    - Gerrit Pape to review and apply patches at his leisure when he
>      feels like making a new release.
> D) The maintainers of distros shipping runit work out a patch-sharing
>    scheme among them.
> 
> 
> Just my 0.02€, I hope it helps.
> 
> cheers,
>     Jan



-- 
Colin Booth

Re: runit patches to fix compiler warnings on RHEL 7

[Colin Booth]

On Sat, Nov 30, 2019 at 08:46:28AM +1100, Dewayne Geraghty wrote:
> Jan,
> 
> I'm also a virgin to process/service management software, learning s6-rc,
> s6, execlineb is not for the faint-hearted nor the time-poor. Getting a
> handle on the concepts, and the naming conventions - its really hard work.
If you want to ease yourself into process supervision I suggest starting
just with s6 and a smattering of execline when you are trying to
describe things that are really annoying to express in shell. Usually
you only need to do this when your run scripts are turning into a mess
of line continuations because of the chain load utilities but there are
other reasons to do it as well. 

s6-rc is absolutely not necessary for basic service management. It's a
very nice helper utility when you're mixing non-idempotent oneshots with
long-running services or handling deep dependency tries that are fairly
touchy (like system bootstrap stuff), but if you don't need deep levels
of dependency ordering and any initial environmental setup for a service
can be handled in an idempotent way, s6-rc is intense overkill. If you
need to synchronize between two services, you can hard-code startup and
shutdown dependencies with s6-svwait and/or s6-svc calls reaching into
the other service directory. It's a pretty low-rent but it's very easy
to think about and manage.
> 
> Execline enforces a discipline, a rigor demanding anticipatory planning (to
> get right).  I ran some performance tests and execlineb is marginally
> better.  So why persist?  Largely because an execline script is immediately
> obvious and explicit.  Seriously, at a glance you know what the script will
> achieve.  Could I write a sh script to do the same task?  Yes, and probably
> do it a lot quicker.  But.  I would loose the elegance and readability -
> where sh has an equivalence with assembler, execline is akin to BASIC, it
> makes you think differently :)
I personally like execline for run scripts because there's very little
magic and it takes a lot of work to fail to get the service that you
want supervised correctly parented at the end of the day. Also, since it
execs into each program you end up not having to do shenanigans like
execing into nested shells if you need to modify state after dropping
privileges (like you do with chpst).
> 
> I'm developing solutions for PROTECTED level security (its an Australian
> Govt thing), and skarnet's service management provides assurance, and s6-log
> provides near-certainty of logging completeness. I'm very happy with the
> toolset, worth the time investment.

-- 
Colin Booth

Re: runit patches to fix compiler warnings on RHEL 7

[Steve Litt]

On Sat, 30 Nov 2019 00:21:59 +0000
Colin Booth <colin@heliocat.net> wrote:

 
> runit-init is slowly becoming less functional and it wouldn't surprise
> me if it fails entirely after Debian 10. 

By what mechanism is runit-init slowly becoming less functional, and
what changes in Debian might cause it to fail entirely?

Thanks,
 
SteveT

Steve Litt
November 2019 featured book: Manager's Guide to Technical
Troubleshooting Second edition
http://www.troubleshooters.com/mgr

s6 usability (was: runit patches to fix compiler warnings on RHEL 7)

[Laurent Bercot]

[-- Attachment #1: Type: text/plain, Size: 7170 bytes --]


>As a relatively new convert to supervision software, my reasons for
>preferring runit over s6 are, in order of priority:

Hi Jan,

Thank you a lot for this feedback. This is very useful UX return.
Let me address the points one by one.


>1) Debian ships with a working and maintained runit-init package. It
>    provides pid 1 and hooks into /etc/rcS.d/* to integrate with other
>    Debian packages. s6-linux-init and s6-rc are not packaged in Debian.

I hear you. Unfortunately, I have no control over what Debian does.
Debian isn't even able to ship a not-broken execline package, so I'm at
a loss on what to do with them. I'm working on a version of execline 
that
they *might* accept to package correctly, but it's doubtful as long as
the people in charge are prejudiced against the "lots of small binaries
in /bin" approach. :(


>2) runit has manpages. s6 has HTML. :(

This is far from the first time I hear this, so it would be foolish to
ignore it, but I *really* have a hard time understanding how in 2019 
it's
so difficult for people to launch a browser to read a web page. I just
can't get it. Launching a browser and reading a web page is something 
that
we all do, every day, even the least computer-savvy ones among us, and 
for
the life of me I can't fathom how this is *not* a friendly user 
interface.
HTML even has the advantages of hyperlinks, so you can jump around in 
the
documentation as needed, which you can't do with man pages!

Do you work offline? don't you have access to a web browser? do you like
reading stuff in a terminal *better*? (Text-based web browsers still
exist, if you do.)

It really stumps me. I did learn "man" too, 25 years ago, before the
Web existed. It was nice when man pages were all we had. But a couple of
years later, we had something that seems unarguably better to me, and
I see exactly zero reason to go back, apart from the force of habit of
typing "man".

Nevertheless, if people like man pages, they should have man pages, so
it's been a few years that I have appealed to the community for this.
I'm not going to learn nroff, ever, but we have a relatively large user
community, so they could probably contribute man pages, right? We've had
a few people who seemed interested, some of them even started thinking
about it *really hard*. And one of them even wrote a tool to convert
text into nroff, à la markdown (but simpler). But when it was about 
doing
the actual work of writing the man pages (even if all the material is
already here, in the HTML doc)? You guessed it: crickets.

So, yeah, I want s6 to be accessible, but I figure that if people
really wanted man pages, they'd write man pages ¯\_(ツ)_/¯ Or maybe I was
wrong all along and the spirit of Open Source really is "If the one 
person
doing the work doesn't do exactly what I want, then I'm just gonna sit 
on
my ass and blame them".

If I'm sounding a bit jaded, it's because I am.


>3) s6 executables are somehow worse named than runit's. This may be
>    highly subjective, but I can recall and recognize the runit commands
>    far easier than the s6 ones. Possibly it's the "s6-" prefix getting
>    in the way of my brain pattern matching on visual appearance of glyph
>    sequences.
>    This point is exacerbated by #2 and the number of s6 executables.
>    Compare chpst with s6-envdir s6-envuidgid s6-fghack s6-setsid
>    s6-setuidgid s6-applyuidgid s6-softlimit. Yes, I know about the
>    historical reasons, but still.

This is very interesting. I thought that having a s6- prefix was a 
*good*
thing, because I valued clarity above everything, and especially above
terseness. I understand the advantages of having commands named "sv" and
"chpst", but I believed, in my naïveté, that it wasn't a good idea for a
specialized package to tread on a large namespace; and to me the s6-
prefix would help users recognize exactly the domain of the command
they're using, and then they could abstract it away and focus on the
real command name.
Now you're saying that the s6- prefix *impedes* your pattern recognition
and detracts you from easily mapping the command name to its 
functionality;
that having it is worse UI than not having it. I had not heard this
before, but it quite makes sense.

The number of executables is a choice; I like to have more, smaller,
executables than less, bigger ones. One function, one tool. It makes
code easier to write; this is not really for historical reasons, it's a
design choice. Personally, it's easier for me to remember several
process state change command names than all the options to chpst.
whenever I use chpst, I always need to check the doc; when I use
something like softlimit or setuidgid, I may need to check the doc for
specific options, but I always remember which command I want and its
general syntax. So, I suppose it comes down to individual preference
there.

Would a generic "s6" command, that takes alternative syntax and rewrites
itself into "internal" commands, help? It could emulate runit syntax,
among other things.

s6 runsv ... -> s6-supervise ...
s6 sv ... -> s6-svc ...
s6 chpst ... -> various s6-prefixed process state change commands

My plan is for the future s6-frontend package to include such a
one-stop-shop command that controls various aspects of an s6 
installation,
but if this command can help with s6 adoption, I can work on it much
earlier than the rest of the s6-frontend functionality.

Or, if you have other ideas that could help with easier assimilation of
the s6 commands, I'm very open to suggestions.


>4) s6 seems more complex (hello execline!), and I don't (yet?) see any
>    benefit/feature I'd appreciate except minimizing wakeups.

This, on the other hand, is a misconception that really needs to
disappear. Understanding execline is *not needed* to run s6. s6 depends
on execline in two places (there were more, but I scrapped them because
nobody used the commands that were involved):
- at build-time, in s6-ftrig-listen
- at run-time, in s6-log, for processor invocation

Would entirely removing s6's dependency on execline help clear that
misunderstanding and help with s6 adoption? This could be made possible
by:
- duplicating el_semicolon() functionality in s6-ftrig-listen.c
(it's not elegant, but clearing the dep may be worth it)
- adding an alternative '?' processor directive to s6-log, that spawns
the processor using /bin/sh instead of execlineb. (The '!' directive
would still be there; processor invocations using '!' would just fail
if execline is not installed.)

I don't like this, but if "execline" is a scarecrow that keeps people
away from s6 for no other reason than perception, it's a possibility.
Savvy users will still install execline for use in run scripts.

s6-rc, however, absolutely cannot do without execline, since it uses
autogenerated execline scripts. But s6-rc is a different beast, that
requires a lot more involvement than s6 anyway, and that isn't needed
at all if we're just talking about runit-like functionality.

--
Laurent

Re: runit patches to fix compiler warnings on RHEL 7

[Jeff]

30.11.2019, 01:22, "Colin Booth" <colin@heliocat.net>:
>>  2) runit has manpages. s6 has HTML. :(
> Yeah, this sucks. I know Laurent isn't going to do it but I've been
> meaning to take some time off and dedicate it to rewriting all of the
> documentation into something that an compile into both mandoc and html.

what about writing the docs in Perl's POD format or Markdown ?
it is easy to convert POD to html AND manpages (pod2(html,man))
and to deliver the generated docs in the source releases.

>>  3) s6 executables are somehow worse named than runit's. This may be
>>     highly subjective, but I can recall and recognize the runit commands
>>     far easier than the s6 ones. Possibly it's the "s6-" prefix getting
>>     in the way of my brain pattern matching on visual appearance of glyph
>>     sequences.
>>     This point is exacerbated by #2 and the number of s6 executables.
>>     Compare chpst with s6-envdir s6-envuidgid s6-fghack s6-setsid
>>     s6-setuidgid s6-applyuidgid s6-softlimit. Yes, I know about the
>>     historical reasons, but sti

totally agreed.

> chpst is a monster of a program and at least with runscripts written in
> execline it's generally easier to understand 3-4 process state
> manipulators than a pile of chpst options.

this is more complicated to use, though.
therefore i personally prefer perp's approach of providing both:
the single process state manipulators and their combination into one
single tool ("run..." vs "runtool").

>>  Brainstorming possible ways forward:
>>
>>  A) Gerrit Pape becomes more active in maintianing runit, at least
>>     acknowledging patches posted here.
>>  B) Somebody else steps in as (co-)maintainer.
>>  C) We get a dumping ground (wiki or somesuch) for patches to allow
>>     - contributors to publish their patches (after discussing them here)
>>     - users to easily find and download patches they'd be interested in
>>     - Gerrit Pape to review and apply patches at his leisure when he
>>       feels like making a new release.
>>  D) The maintainers of distros shipping runit work out a patch-sharing
>>     scheme among them.

runit is dead, i recommend against using it at all, the only tools of interest
this supervision suite provides are "chpst" and "utmpset"
(though the latter is indeed not as powerful as it should to make it really
useful).

besides waking up to poll for rundir changes shutting it down really
sucks since it has problems closing the log files properly, i have not seen
this with any of daemontools encore, perp, and s6.

consider switching, daemontools encore and perp were not meant to run
as process #1, but they can be supervised by (Busy,Toy)Box- or SysV init easily.

daemontools encore's "svscan" utility wakes up to poll the rundir for changes
frequently, though, unlike s6 and perp it does not provide any option to only
do this on request (maybe by just listing for a given signal).

so the final conclusion and recommendation here is:
stop using runit for supervision (and as process #1) and switch to s6.

Re: runit patches to fix compiler warnings on RHEL 7

[Jeff]

>>  chpst is a monster of a program and at least with runscripts written in
>>  execline it's generally easier to understand 3-4 process state
>>  manipulators than a pile of chpst options.
>
> this is more complicated to use, though.

it is even unnecessary, inefficient, and wasteful.

why exec chaining into several different utils where doing all the
requested process state changes in one go by using the same
utility to achieve them would suffice ?

Re: s6 usability (was: runit patches to fix compiler warnings on RHEL 7)

[Jeff]

30.11.2019, 11:15, "Laurent Bercot" <ska-supervision@skarnet.org>:
> This is very interesting. I thought that having a s6- prefix was a *good*
> thing, because I valued clarity above everything, and especially above
> terseness. I understand the advantages of having commands named "sv" and
> "chpst", but I believed, in my naïveté, that it wasn't a good idea for a
> specialized package to tread on a large namespace; and to me the s6-
> prefix would help users recognize exactly the domain of the command
> they're using, and then they could abstract it away and focus on the
> real command name.

totally agreed, Laurent.

using a dedicated namespace prefix like "s6-" is a very good idea.
this avoids nameclashes (i. e. overwriting on installation) with similar
utilities of other supervision suites and frees Laurent from the task
of coming up with proper AND unique command names. consider
nameclashes of several "init" program for example.

the solution here could be a simple symlink to the original s6 tool without
the prefix if you prefer (maybe even located in an other dir than /bin).

> The number of executables is a choice; I like to have more, smaller,
> executables than less, bigger ones. One function, one tool. It makes
> code easier to write; this is not really for historical reasons, it's a
> design choice. Personally, it's easier for me to remember several
> process state change command names than all the options to chpst.
> whenever I use chpst, I always need to check the doc; when I use
> something like softlimit or setuidgid, I may need to check the doc for
> specific options, but I always remember which command I want and its
> general syntax. So, I suppose it comes down to individual preference
> there.

using a single combined tool is more efficient since it avoids wasteful further
exec chaining steps, though.

> Would a generic "s6" command, that takes alternative syntax and rewrites
> itself into "internal" commands, help? It could emulate runit syntax,
> among other things.
>
> s6 runsv ... -> s6-supervise ...
> s6 sv ... -> s6-svc ...
> s6 chpst ... -> various s6-prefixed process state change commands
>
> My plan is for the future s6-frontend package to include such a
> one-stop-shop command that controls various aspects of an s6
> installation,
> but if this command can help with s6 adoption, I can work on it much
> earlier than the rest of the s6-frontend functionality.
>
> Or, if you have other ideas that could help with easier assimilation of
> the s6 commands, I'm very open to suggestions.

Busy/ToyBox style ?

> Would entirely removing s6's dependency on execline help clear that
> misunderstanding and help with s6 adoption? This could be made possible
> by:
> - duplicating el_semicolon() functionality in s6-ftrig-listen.c
> (it's not elegant, but clearing the dep may be worth it)
> - adding an alternative '?' processor directive to s6-log, that spawns
> the processor using /bin/sh instead of execlineb. (The '!' directive
> would still be there; processor invocations using '!' would just fail
> if execline is not installed.)

sounds not too bad, IMO. though i personally can live without it,
especially since the other suites also provide loggers (without any
execline deps of course), he can use dt encore's "multilog" utility.

> s6-rc, however, absolutely cannot do without execline, since it uses
> autogenerated execline scripts.

could you document the way s6-rc works (i. e. its architecture) somewhere ?
or are users requested to follow your C code to find out how it works
exactly ?

> But s6-rc is a different beast, that
> requires a lot more involvement than s6 anyway, and that isn't needed
> at all if we're just talking about runit-like functionality.

indeed.

Re: s6 usability (was: runit patches to fix compiler warnings on RHEL 7)

[Laurent Bercot]

>the solution here could be a simple symlink to the original s6 tool without
>the prefix if you prefer (maybe even located in an other dir than /bin).

That would be a decision for users, not software authors - else it would
defeat the point of not invading the namespace. Daemontools is still
around with names such as "svc".


>using a single combined tool is more efficient since it avoids wasteful further exec chaining steps, though.

Sure, but if we're talking about UI, optimization at this level is a 
very
moot point. A human choosing between "chpst" and "s6-applyuidgid" will
*not* notice the extra millisecond taken by an execve() step. The
primary focus should be usability.


>Busy/ToyBox style ?

No, I don't like multicall binaries. I was more thinking of a wrapper
that rewrites its command line and execs into it, like s6-svc does when
given the -w option. The goal is simply to have a unified UI thing that
makes it easier for users to find the command they need.


>could you document the way s6-rc works (i. e. its architecture) somewhere ?
>or are users requested to follow your C code to find out how it works
>exactly ?

  I am reluctant to make the ABI details public because I want the 
freedom
to change them. If people start relying on internals, their stuff may
break when updating, which would be bad.
  There are *some* details that I could document as official and stable,
but I'd need to go through all of it and decide with precision what can 
be
guaranteed and what cannot - and that's extra work, so it will have to 
wait.

--
  Laurent

Re: s6 usability (was: runit patches to fix compiler warnings on RHEL 7)

[Steve Litt]

On Sat, 30 Nov 2019 10:15:27 +0000
"Laurent Bercot" <ska-supervision@skarnet.org> wrote:


> I hear you. Unfortunately, I have no control over what Debian does.
> Debian isn't even able to ship a not-broken execline package, so I'm
> at a loss on what to do with them. I'm working on a version of
> execline that
> they *might* accept to package correctly, but it's doubtful as long as
> the people in charge are prejudiced against the "lots of small
> binaries in /bin" approach. :(

Would it be acceptable to you and them to put the binaries in /bin/s6
and then very early in the boot add /bin/s6 to the path? This isn't a
lot different from what djb did with /command, except it's not off the
root, which everyone seems to hate.

[snip] 

> >3) s6 executables are somehow worse named than runit's. This may be
> >    highly subjective, but I can recall and recognize the runit
> > commands far easier than the s6 ones. Possibly it's the "s6-"
> > prefix getting in the way of my brain pattern matching on visual
> > appearance of glyph sequences.
> >    This point is exacerbated by #2 and the number of s6 executables.
> >    Compare chpst with s6-envdir s6-envuidgid s6-fghack s6-setsid
> >    s6-setuidgid s6-applyuidgid s6-softlimit. Yes, I know about the
> >    historical reasons, but still.  
> 
> This is very interesting. I thought that having a s6- prefix was a 
> *good*
> thing, because I valued clarity above everything, and especially above
> terseness. 

As a guy who has both daemontools and s6 installed on the same box, I
thank you from the bottom of my heart for:

1) Prepending s6- to each command so they don't clash with djb's
2) Except for the s6-, naming them the same as djb's so I have less to
   remember.

[snip]
> 
> >4) s6 seems more complex (hello execline!), and I don't (yet?) see
> >any
> >    benefit/feature I'd appreciate except minimizing wakeups.  
> 
> This, on the other hand, is a misconception that really needs to
> disappear. Understanding execline is *not needed* to run s6. s6
> depends on execline in two places (there were more, but I scrapped
> them because nobody used the commands that were involved):
> - at build-time, in s6-ftrig-listen
> - at run-time, in s6-log, for processor invocation
> 
> Would entirely removing s6's dependency on execline help clear that
> misunderstanding and help with s6 adoption? This could be made
> possible by:
> - duplicating el_semicolon() functionality in s6-ftrig-listen.c
> (it's not elegant, but clearing the dep may be worth it)
> - adding an alternative '?' processor directive to s6-log, that spawns
> the processor using /bin/sh instead of execlineb. (The '!' directive
> would still be there; processor invocations using '!' would just fail
> if execline is not installed.)
> 
> I don't like this, but if "execline" is a scarecrow that keeps people
> away from s6 for no other reason than perception, it's a possibility.
> Savvy users will still install execline for use in run scripts.

I don't think it's necessary to remove the dependency unless ordinary
users would be altering the code of s6-ftrig-listen or s6-log.

A simple change change I think would do it is to change the
documentation to imply that, for the *user*, execlineb is a way to get
just a little extra whatever. Currently, when I read it, I thought I'd
be missing a lot by using /bin/sh.


> 
> s6-rc, however, absolutely cannot do without execline, since it uses
> autogenerated execline scripts. But s6-rc is a different beast, that
> requires a lot more involvement than s6 anyway, and that isn't needed
> at all if we're just talking about runit-like functionality.

Does the *user* need to code execline scripts, or is it just
something the program does? If the former, then make a point that one
doesn't need to use execline for s6-rc to be a very powerful startup
system.

If anybody would make an execline tutorial, that would help a lot. For
a guy like me who only does procedural programming (C, C++, Pascal,
Perl, Python, Ruby, Lua, etc), execline is difficult to understand.

SteveT

Steve Litt
November 2019 featured book: Manager's Guide to Technical
Troubleshooting Second edition
http://www.troubleshooters.com/mgr

Re: s6 usability

[Dewayne Geraghty]

Hi Steve,

> Does the *user* need to code execline scripts, or is it just
> something the program does? If the former, then make a point that one
> doesn't need to use execline for s6-rc to be a very powerful startup
> system.

No the user doesn't need to write execline scripts. The following 
equally applies to s6-rc.

Refer to:https://skarnet.org/software/s6/overview.html
for:
"execline makes it natural to handle long command lines made of massive 
amounts of chain loading. This is by no means mandatory, though: a run 
script can be any executable file you want, provided that running it 
eventually results in a long-lived process with the same PID."


Regarding creating a s6 subdir of bin.  I have some 1325 applications 
(FreeBSD people call them ports), only 1 has a separate directory under bin.

Re: s6 usability

[fungal-net]

As a test for portability for the 66 software, other than its display
case based on arch - called obarun - a couple of devs recently tried it
in a few distributions that have the s6 in their repository.

Funtoo was the first target
https://forum.obarun.org/viewtopic.php?id=956 and this was done by a dev
who used plasma on obarun and used plasma on funtoo.

Void has s6 and 66 is under testing but boot-66serv has to be installed
separately with a modification to match void's choice of placing
services in /usr/share instead of the original /usr/lib.  With 6-7
commands and it booted to a tty of choice.  I've been using it daily
since (I was shown the way) in musl flavor.  A delight and a more
responsive system (feels that way) than the OEM runit.  I wish I had one
of the 7-9 alternative void architectures to do further testing but all
I have is refurbished tired x64s.
https://forum.obarun.org/viewtopic.php?id=957

Then there was adelie and kiss (an alpha system with similarities to
gentoo but installs in little time)
https://forum.obarun.org/viewtopic.php?id=961
https://forum.obarun.org/viewtopic.php?id=959 with similar ease.

With debian there is the obstacle of s6 being intentionally broken by
dislocated skalibs but I am speculating there is more to it.  How often
has a broken package been available in the 4 level hierarchy of debian?
If it was on experimental I can understand, but there are two versions,
recent and previous s6 and libs, both broken, in sid,testing,stable.  I
may be paranoid but isn't this also preventing devuan and antiX (and its
MX derivative) from effectively trying s6 unless they uniquely rename
the packages confusing users?

I know that the scope of this list is in hacking experimental init and
service supervision software but what better way to test the work other
than displaying it in more popular distributions?
All one needs past s6 is boot-66serv
# git clone https://framagit.org/obarun/boot-66serv
# cd boot-66serv
# ./configure --bindir=/usr/bin --shebangdir=/usr/bin
--with-system-service=/usr/lib-{or share or whatever}/66/service
# make install

I think it would take an earth-shaking new development for someone to
pry me away from 66 for years to come, if I have that much left in me.

PS  I am writing this still shaking in anger from reading Jesse Smith's
Distrowatch review of Obarun, using an old image with a previous
installer, declining its initial prompt to update the installer and
theme, failing the installation and moving on to bluestar, a desktop
theme customizer of arch, and finding it wonderful.  This is after
listing Obarun for 1.5 years, declining to correct its information for
more than 7 months, and bad mouthing it, in the same article where its
headline lists Devuan as Debuan2.1.  Same guy reviewed Adelie (near its
beta life end) and failed to see s6 in it.

Has RH and its mother... IBM converted the open and free software world
into a head-butting ring?  No, there are others at fault for that as
well.  Arch for example is pushing facebook's compression algorithm into
its packaging by default.  I am now going to shut off this machine and
take my 30yo Bridgestone cycle with friction shifters out for a ride in
this fine sunny wintery day, because I am getting too disgusted from
doing any real work, hopefully no big truck will run in my way.  :)
Did you know there are now about 3 corporations responsible for about
90% of the bikes made?  A "free" choice among 1300 models.

Re: s6 usability (was: runit patches to fix compiler warnings on RHEL 7)

[Jeff]

30.11.2019, 19:58, "Laurent Bercot" <ska-supervision@skarnet.org>:
>> the solution here could be a simple symlink to the original s6 tool without
>> the prefix if you prefer (maybe even located in an other dir than /bin).
>
> That would be a decision for users, not software authors - else it would
> defeat the point of not invading the namespace. Daemontools is still
> around with names such as "svc".

sure, that was just an idea for Jan, he could just create a dir somewhere,
populate it with symlinks he prefers to the original s6 tools and put this dir
in front of the PATH when running s6 since it seems the utilities do not bother
under what name they run.
 
>> using a single combined tool is more efficient since it avoids wasteful
>> further exec chaining steps, though.
>
> Sure, but if we're talking about UI, optimization at this level is a
> very
> moot point. A human choosing between "chpst" and "s6-applyuidgid" will
> *not* notice the extra millisecond taken by an execve() step. The
> primary focus should be usability.

i prefer short names like "chpst" (change process state ?) with multiple
command line options from a usability perspective. but the usage of single
tools with descriptive names is of course easier to read (not to write) and
hence understand when they occur in a script, that's true.

> I am reluctant to make the ABI details public because I want the freedom
> to change them. If people start relying on internals, their stuff may
> break when updating, which would be bad.
> There are *some* details that I could document as official and stable,
> but I'd need to go through all of it and decide with precision what can
> be guaranteed and what cannot - and that's extra work, so it will have
> to wait.

ok. i was more about insights into the design of the whole s6-rc toolset.
are the up/down scripts run by a dedicated service from within the supervision
tree? what exactly is the task of the "s6-rc-oneshot-run" and
"s6-rc-fdholder-filler" internal programs ? how is the startup organized,
how are "longruns" and "oneshots" intertwined ?

having to read the sources to get this information is somewhat inconvenient.
:-(

Re: runit patches to fix compiler warnings on RHEL 7

[J. Lewis Muir]

[-- Attachment #1: Type: text/plain, Size: 982 bytes --]

On 11/27, Martin Castillo wrote:
> On 27.11.19 21:33, J. Lewis Muir wrote:
> > On 11/25, J. Lewis Muir wrote:
> >> Is runit hosted in a public source code repo?  If so, where?
> >>
> >> Are patches to fix runit compiler warnings on RHEL 7 welcome?
> > 
> > I have patches now.  Is there a public source code repo I can contribute
> > against?  Or would it be helpful to just send the patches to the list?
> 
> AFAIK you have to post them to this list.

OK, attached is the patch series.

To apply it against runit 2.1.2:

$ cd admin/runit-2.1.2
$ patch -p2 < /path/to/runit-2.1.2-rhel-7-compiler-fixes.diff

The patch series eliminates all GCC 4.8.5 warnings on x86_64 RHEL 7.7.
I haven't tested on other platforms, so it could be that some of the
fixes just make it compile cleanly on RHEL 7 but not on other platforms.
If this is the case, then I think more logic would be needed at compile
time to test the available APIs and choose the appropriate APIs to use.

Regards,

Lewis

[-- Attachment #2: runit-2.1.2-rhel-7-compiler-fixes.diff --]
[-- Type: text/plain, Size: 16710 bytes --]

# HG changeset patch
# User J. Lewis Muir <jlmuir@imca-cat.org>
# Date 1574790428 21600
#      Tue Nov 26 11:47:08 2019 -0600
# Branch 2.1.2_compiler-warning-fixes
# Node ID bdaf3416fb1b1010546d8a52b75dc764ca8a80a5
# Parent  6001387b32501dd5c76c969467cf4f7a655e9dcf
Change getgroups arg 2 type in chkshsgr.c to gid_t

This change eliminates the following GCC 4.8.5 warning on RHEL 7.7:

  ./compile chkshsgr.c
  chkshsgr.c: In function ‘main’:
  chkshsgr.c:10:3: warning: passing argument 2 of ‘getgroups’ from incompatible pointer type [enabled by default]
     if (getgroups(1,x) == 0) if (setgroups(1,x) == -1) _exit(1);
     ^
  In file included from chkshsgr.c:3:0:
  /usr/include/unistd.h:711:12: note: expected ‘__gid_t *’ but argument is of type ‘short int *’
   extern int getgroups (int __size, __gid_t __list[]) __THROW __wur;
              ^

diff -r 6001387b3250 -r bdaf3416fb1b external/src/chkshsgr.c
--- a/external/src/chkshsgr.c	Tue Nov 26 10:48:03 2019 -0600
+++ b/external/src/chkshsgr.c	Tue Nov 26 11:47:08 2019 -0600
@@ -4,7 +4,7 @@
 
 int main()
 {
-  short x[4];
+  gid_t x[4];
 
   x[0] = x[1] = 0;
   if (getgroups(1,x) == 0) if (setgroups(1,x) == -1) _exit(1);
# HG changeset patch
# User J. Lewis Muir <jlmuir@imca-cat.org>
# Date 1574791174 21600
#      Tue Nov 26 11:59:34 2019 -0600
# Branch 2.1.2_compiler-warning-fixes
# Node ID a12ee55203125f8ef926dbe72e66a1bd45235619
# Parent  bdaf3416fb1b1010546d8a52b75dc764ca8a80a5
Include grp.h in chkshsgr.c

This change eliminates the following GCC 4.8.5 warning on RHEL 7.7:

  ./compile chkshsgr.c
  chkshsgr.c: In function ‘main’:
  chkshsgr.c:10:3: warning: implicit declaration of function ‘setgroups’ [-Wimplicit-function-declaration]
     if (getgroups(1,x) == 0) if (setgroups(1,x) == -1) _exit(1);
     ^

diff -r bdaf3416fb1b -r a12ee5520312 external/src/chkshsgr.c
--- a/external/src/chkshsgr.c	Tue Nov 26 11:47:08 2019 -0600
+++ b/external/src/chkshsgr.c	Tue Nov 26 11:59:34 2019 -0600
@@ -1,5 +1,6 @@
 /* Public domain. */
 
+#include <grp.h>
 #include <unistd.h>
 
 int main()
# HG changeset patch
# User J. Lewis Muir <jlmuir@imca-cat.org>
# Date 1574791401 21600
#      Tue Nov 26 12:03:21 2019 -0600
# Branch 2.1.2_compiler-warning-fixes
# Node ID b0034c0716a2f8496d17d5c4a1d8d77193b3594f
# Parent  a12ee55203125f8ef926dbe72e66a1bd45235619
Include grp.h in chpst.c

This change eliminates the following GCC 4.8.5 warning on RHEL 7.7:

  ./compile chpst.c
  chpst.c: In function ‘suidgid’:
  chpst.c:80:3: warning: implicit declaration of function ‘setgroups’ [-Wimplicit-function-declaration]
     if (setgroups(ugid.gids, ugid.gid) == -1) fatal("unable to setgroups");
     ^

diff -r a12ee5520312 -r b0034c0716a2 external/src/chpst.c
--- a/external/src/chpst.c	Tue Nov 26 11:59:34 2019 -0600
+++ b/external/src/chpst.c	Tue Nov 26 12:03:21 2019 -0600
@@ -2,6 +2,7 @@
 #include <time.h>
 #include <sys/time.h>
 #include <sys/resource.h>
+#include <grp.h>
 #include <unistd.h>
 #include "sgetopt.h"
 #include "error.h"
# HG changeset patch
# User J. Lewis Muir <jlmuir@imca-cat.org>
# Date 1574791671 21600
#      Tue Nov 26 12:07:51 2019 -0600
# Branch 2.1.2_compiler-warning-fixes
# Node ID 396b02da6d0e56256ac5c1cf00ed0af690dbd8f1
# Parent  b0034c0716a2f8496d17d5c4a1d8d77193b3594f
Avoid op sequence order ambiguity in chpst.c

This change eliminates the following GCC 4.8.5 warning on RHEL 7.7:

  ./compile chpst.c
  chpst.c: In function ‘main’:
  chpst.c:312:33: warning: operation on ‘subgetoptarg’ may be undefined [-Wsequence-point]
             if (optarg[scan_ulong(++optarg, &ul)]) usage(); nicelvl =ul;
                                   ^

diff -r b0034c0716a2 -r 396b02da6d0e external/src/chpst.c
--- a/external/src/chpst.c	Tue Nov 26 12:03:21 2019 -0600
+++ b/external/src/chpst.c	Tue Nov 26 12:07:51 2019 -0600
@@ -309,7 +309,8 @@
     case 'n':
       switch (*optarg) {
         case '-':
-          if (optarg[scan_ulong(++optarg, &ul)]) usage(); nicelvl =ul;
+          ++optarg;
+          if (optarg[scan_ulong(optarg, &ul)]) usage(); nicelvl =ul;
           nicelvl *=-1;
           break;
         case '+': ++optarg;
# HG changeset patch
# User J. Lewis Muir <jlmuir@imca-cat.org>
# Date 1574792032 21600
#      Tue Nov 26 12:13:52 2019 -0600
# Branch 2.1.2_compiler-warning-fixes
# Node ID 81ee1629db40cb6c87fa327fadcb7fcf1aeb4c53
# Parent  396b02da6d0e56256ac5c1cf00ed0af690dbd8f1
Include unistd.h in compile pathexec_run.c

This change eliminates the following GCC 4.8.5 warning on RHEL 7.7:

  ./compile pathexec_run.c
  pathexec_run.c: In function ‘pathexec_run’:
  pathexec_run.c:18:5: warning: implicit declaration of function ‘execve’ [-Wimplicit-function-declaration]
       execve(file,argv,envp);
       ^

diff -r 396b02da6d0e -r 81ee1629db40 external/src/pathexec_run.c
--- a/external/src/pathexec_run.c	Tue Nov 26 12:07:51 2019 -0600
+++ b/external/src/pathexec_run.c	Tue Nov 26 12:13:52 2019 -0600
@@ -1,5 +1,6 @@
 /* Public domain. */
 
+#include <unistd.h>
 #include "error.h"
 #include "stralloc.h"
 #include "str.h"
# HG changeset patch
# User J. Lewis Muir <jlmuir@imca-cat.org>
# Date 1574805376 21600
#      Tue Nov 26 15:56:16 2019 -0600
# Branch 2.1.2_compiler-warning-fixes
# Node ID e19a22494c69e30e3cd384ae6780f4667a67e449
# Parent  81ee1629db40cb6c87fa327fadcb7fcf1aeb4c53
Cast execve argv arg in pathexec_run.c

This change eliminates the following GCC 4.8.5 warnings on RHEL 7.7:

  ./compile pathexec_run.c
  pathexec_run.c: In function ‘pathexec_run’:
  pathexec_run.c:19:5: warning: passing argument 2 of ‘execve’ from incompatible pointer type [enabled by default]
       execve(file,argv,envp);
       ^
  In file included from pathexec_run.c:3:0:
  /usr/include/unistd.h:551:12: note: expected ‘char * const*’ but argument is of type ‘const char * const*’
   extern int execve (const char *__path, char *const __argv[],
              ^
  pathexec_run.c:36:5: warning: passing argument 2 of ‘execve’ from incompatible pointer type [enabled by default]
       execve(tmp.s,argv,envp);
       ^
  In file included from pathexec_run.c:3:0:
  /usr/include/unistd.h:551:12: note: expected ‘char * const*’ but argument is of type ‘const char * const*’
   extern int execve (const char *__path, char *const __argv[],
              ^

diff -r 81ee1629db40 -r e19a22494c69 external/src/pathexec_run.c
--- a/external/src/pathexec_run.c	Tue Nov 26 12:13:52 2019 -0600
+++ b/external/src/pathexec_run.c	Tue Nov 26 15:56:16 2019 -0600
@@ -16,7 +16,7 @@
   int savederrno;
 
   if (file[str_chr(file,'/')]) {
-    execve(file,argv,envp);
+    execve(file,(char * const *)argv,envp);
     return;
   }
 
@@ -33,7 +33,7 @@
     if (!stralloc_cats(&tmp,file)) return;
     if (!stralloc_0(&tmp)) return;
 
-    execve(tmp.s,argv,envp);
+    execve(tmp.s,(char * const *)argv,envp);
     if (errno != error_noent) {
       savederrno = errno;
       if ((errno != error_acces) && (errno != error_perm) && (errno != error_isdir)) return;
# HG changeset patch
# User J. Lewis Muir <jlmuir@imca-cat.org>
# Date 1574807490 21600
#      Tue Nov 26 16:31:30 2019 -0600
# Branch 2.1.2_compiler-warning-fixes
# Node ID eea70f9d621273abdca6d0bd27f8729ec4f175ab
# Parent  e19a22494c69e30e3cd384ae6780f4667a67e449
Cast execve envp arg in pathexec_run.c

This change eliminates the following GCC 4.8.5 warnings on RHEL 7.7:

  ./compile pathexec_run.c
  pathexec_run.c: In function ‘pathexec_run’:
  pathexec_run.c:19:5: warning: passing argument 3 of ‘execve’ from incompatible pointer type [enabled by default]
       execve(file,(char * const *)argv,envp);
       ^
  In file included from pathexec_run.c:3:0:
  /usr/include/unistd.h:551:12: note: expected ‘char * const*’ but argument is of type ‘const char * const*’
   extern int execve (const char *__path, char *const __argv[],
              ^
  pathexec_run.c:36:5: warning: passing argument 3 of ‘execve’ from incompatible pointer type [enabled by default]
       execve(tmp.s,(char * const *)argv,envp);
       ^
  In file included from pathexec_run.c:3:0:
  /usr/include/unistd.h:551:12: note: expected ‘char * const*’ but argument is of type ‘const char * const*’
   extern int execve (const char *__path, char *const __argv[],
              ^

diff -r e19a22494c69 -r eea70f9d6212 external/src/pathexec_run.c
--- a/external/src/pathexec_run.c	Tue Nov 26 15:56:16 2019 -0600
+++ b/external/src/pathexec_run.c	Tue Nov 26 16:31:30 2019 -0600
@@ -16,7 +16,7 @@
   int savederrno;
 
   if (file[str_chr(file,'/')]) {
-    execve(file,(char * const *)argv,envp);
+    execve(file,(char * const *)argv,(char * const *)envp);
     return;
   }
 
@@ -33,7 +33,7 @@
     if (!stralloc_cats(&tmp,file)) return;
     if (!stralloc_0(&tmp)) return;
 
-    execve(tmp.s,(char * const *)argv,envp);
+    execve(tmp.s,(char * const *)argv,(char * const *)envp);
     if (errno != error_noent) {
       savederrno = errno;
       if ((errno != error_acces) && (errno != error_perm) && (errno != error_isdir)) return;
# HG changeset patch
# User J. Lewis Muir <jlmuir@imca-cat.org>
# Date 1574808043 21600
#      Tue Nov 26 16:40:43 2019 -0600
# Branch 2.1.2_compiler-warning-fixes
# Node ID 6d61f595fb22a0ee4bdb09d727c300828fc76a3e
# Parent  eea70f9d621273abdca6d0bd27f8729ec4f175ab
Include grp.h in prot.c

This change eliminates the following GCC 4.8.5 warning on RHEL 7.7:

  ./compile prot.c
  prot.c: In function ‘prot_gid’:
  prot.c:13:3: warning: implicit declaration of function ‘setgroups’ [-Wimplicit-function-declaration]
     if (setgroups(1,&gid) == -1) return -1;
     ^

diff -r eea70f9d6212 -r 6d61f595fb22 external/src/prot.c
--- a/external/src/prot.c	Tue Nov 26 16:31:30 2019 -0600
+++ b/external/src/prot.c	Tue Nov 26 16:40:43 2019 -0600
@@ -1,5 +1,6 @@
 /* Public domain. */
 
+#include <grp.h>
 #include "hasshsgr.h"
 #include "prot.h"
 
# HG changeset patch
# User J. Lewis Muir <jlmuir@imca-cat.org>
# Date 1574874991 21600
#      Wed Nov 27 11:16:31 2019 -0600
# Branch 2.1.2_compiler-warning-fixes
# Node ID 3714c914f8803b8235329a5438ce83c5241a0070
# Parent  6d61f595fb22a0ee4bdb09d727c300828fc76a3e
Change setgroups arg 2 type in prot.c to gid_t

This change eliminates the following GCC 4.8.5 warning on RHEL 7.7:

  ./compile prot.c
  prot.c: In function ‘prot_gid’:
  prot.c:14:3: warning: pointer targets in passing argument 2 of ‘setgroups’ differ in signedness [-Wpointer-sign]
     if (setgroups(1,&gid) == -1) return -1;
     ^
  In file included from prot.c:3:0:
  /usr/include/grp.h:181:12: note: expected ‘const __gid_t *’ but argument is of type ‘int *’
   extern int setgroups (size_t __n, const __gid_t *__groups) __THROW;
              ^

diff -r 6d61f595fb22 -r 3714c914f880 external/src/prot.c
--- a/external/src/prot.c	Tue Nov 26 16:40:43 2019 -0600
+++ b/external/src/prot.c	Wed Nov 27 11:16:31 2019 -0600
@@ -7,11 +7,13 @@
 int prot_gid(int gid)
 {
 #ifdef HASSHORTSETGROUPS
-  short x[2];
-  x[0] = gid; x[1] = 73; /* catch errors */
+  gid_t x[2];
+  x[0] = (gid_t)gid; x[1] = 73; /* catch errors */
   if (setgroups(1,x) == -1) return -1;
 #else
-  if (setgroups(1,&gid) == -1) return -1;
+  gid_t x;
+  x = (gid_t)gid;
+  if (setgroups(1,&x) == -1) return -1;
 #endif
   return setgid(gid); /* _should_ be redundant, but on some systems it isn't */
 }
# HG changeset patch
# User J. Lewis Muir <jlmuir@imca-cat.org>
# Date 1574879582 21600
#      Wed Nov 27 12:33:02 2019 -0600
# Branch 2.1.2_compiler-warning-fixes
# Node ID abd85fc8cbb6632f78d8973a673df11bda56a82a
# Parent  3714c914f8803b8235329a5438ce83c5241a0070
Include unistd.h in proto.c

This change eliminates the following GCC 4.8.5 warning on RHEL 7.7:

  ./compile prot.c
  prot.c: In function ‘prot_gid’:
  prot.c:18:3: warning: implicit declaration of function ‘setgid’ [-Wimplicit-function-declaration]
     return setgid(gid); /* _should_ be redundant, but on some systems it isn't */
     ^

diff -r 3714c914f880 -r abd85fc8cbb6 external/src/prot.c
--- a/external/src/prot.c	Wed Nov 27 11:16:31 2019 -0600
+++ b/external/src/prot.c	Wed Nov 27 12:33:02 2019 -0600
@@ -1,6 +1,7 @@
 /* Public domain. */
 
 #include <grp.h>
+#include <unistd.h>
 #include "hasshsgr.h"
 #include "prot.h"
 
# HG changeset patch
# User J. Lewis Muir <jlmuir@imca-cat.org>
# Date 1574880136 21600
#      Wed Nov 27 12:42:16 2019 -0600
# Branch 2.1.2_compiler-warning-fixes
# Node ID 2f9db40fe5a069b5c9e7498a45e0ecff3f0ad518
# Parent  abd85fc8cbb6632f78d8973a673df11bda56a82a
Use gid_t/uid_t types in setgid/setuid in proto.c

GCC 4.8.5 on RHEL 7.7 did not emit warnings about these type
differences, but these changes are consistent with the type changes that
were necessary in proto.c to quash other warnings.

diff -r abd85fc8cbb6 -r 2f9db40fe5a0 external/src/prot.c
--- a/external/src/prot.c	Wed Nov 27 12:33:02 2019 -0600
+++ b/external/src/prot.c	Wed Nov 27 12:42:16 2019 -0600
@@ -16,10 +16,10 @@
   x = (gid_t)gid;
   if (setgroups(1,&x) == -1) return -1;
 #endif
-  return setgid(gid); /* _should_ be redundant, but on some systems it isn't */
+  return setgid((gid_t)gid); /* _should_ be redundant, but on some systems it isn't */
 }
 
 int prot_uid(int uid)
 {
-  return setuid(uid);
+  return setuid((uid_t)uid);
 }
# HG changeset patch
# User J. Lewis Muir <jlmuir@imca-cat.org>
# Date 1574880328 21600
#      Wed Nov 27 12:45:28 2019 -0600
# Branch 2.1.2_compiler-warning-fixes
# Node ID 6fb94ec18d5277e100432f5780f6f9cd5902bb2c
# Parent  2f9db40fe5a069b5c9e7498a45e0ecff3f0ad518
Include unistd.h in seek_set.c

This change eliminates the following GCC 4.8.5 warning on RHEL 7.7:

  ./compile seek_set.c
  seek_set.c: In function ‘seek_set’:
  seek_set.c:9:1: warning: implicit declaration of function ‘lseek’ [-Wimplicit-function-declaration]
   { if (lseek(fd,(off_t) pos,SET) == -1) return -1; return 0; }
   ^

diff -r 2f9db40fe5a0 -r 6fb94ec18d52 external/src/seek_set.c
--- a/external/src/seek_set.c	Wed Nov 27 12:42:16 2019 -0600
+++ b/external/src/seek_set.c	Wed Nov 27 12:45:28 2019 -0600
@@ -1,6 +1,7 @@
 /* Public domain. */
 
 #include <sys/types.h>
+#include <unistd.h>
 #include "seek.h"
 
 #define SET 0 /* sigh */
# HG changeset patch
# User J. Lewis Muir <jlmuir@imca-cat.org>
# Date 1574881770 21600
#      Wed Nov 27 13:09:30 2019 -0600
# Branch 2.1.2_compiler-warning-fixes
# Node ID 942ba9fb2086c5157b70fb62e34b7e6c393dc127
# Parent  6fb94ec18d5277e100432f5780f6f9cd5902bb2c
Use time_t type in {u,w}tmp_logout in utmpset.c

This change eliminates the following GCC 4.8.5 warnings on RHEL 7.7:

  ./compile utmpset.c
  utmpset.c: In function ‘utmp_logout’:
  utmpset.c:38:5: warning: passing argument 1 of ‘time’ from incompatible pointer type [enabled by default]
       if (time(&ut.ut_time) == -1) break;
       ^
  In file included from utmpset.c:2:0:
  /usr/include/time.h:192:15: note: expected ‘time_t *’ but argument is of type ‘int32_t *’
   extern time_t time (time_t *__timer) __THROW;
                 ^
  utmpset.c: In function ‘wtmp_logout’:
  utmpset.c:68:3: warning: passing argument 1 of ‘time’ from incompatible pointer type [enabled by default]
     if (time(&ut.ut_time) == -1) {
     ^
  In file included from utmpset.c:2:0:
  /usr/include/time.h:192:15: note: expected ‘time_t *’ but argument is of type ‘int32_t *’
   extern time_t time (time_t *__timer) __THROW;
                 ^

diff -r 6fb94ec18d52 -r 942ba9fb2086 external/src/utmpset.c
--- a/external/src/utmpset.c	Wed Nov 27 12:45:28 2019 -0600
+++ b/external/src/utmpset.c	Wed Nov 27 13:09:30 2019 -0600
@@ -24,6 +24,7 @@
 int utmp_logout(const char *line) {
   int fd;
   uw_tmp ut;
+  time_t t;
   int ok =-1;
 
   if ((fd =open(UW_TMP_UFILE, O_RDWR, 0)) < 0)
@@ -35,7 +36,8 @@
     if (!ut.ut_name[0] || (str_diff(ut.ut_line, line) != 0)) continue;
     memset(ut.ut_name, 0, sizeof ut.ut_name);
     memset(ut.ut_host, 0, sizeof ut.ut_host);
-    if (time(&ut.ut_time) == -1) break;
+    if (time(&t) == (time_t)-1) break;
+    ut.ut_time = t;
 #ifdef DEAD_PROCESS
     ut.ut_type =DEAD_PROCESS;
 #endif
@@ -52,6 +54,7 @@
   int len;
   struct stat st;
   uw_tmp ut;
+  time_t t;
 
   if ((fd = open_append(UW_TMP_WFILE)) == -1)
     strerr_die4sys(111, FATAL, "unable to open ", UW_TMP_WFILE, ": ");
@@ -65,10 +68,11 @@
   memset(&ut, 0, sizeof(uw_tmp));
   if ((len =str_len(line)) > sizeof ut.ut_line) len =sizeof ut.ut_line -2;
   byte_copy(ut.ut_line, len, line);
-  if (time(&ut.ut_time) == -1) {
+  if (time(&t) == (time_t)-1) {
     close(fd);
     return(-1);
   }
+  ut.ut_time = t;
 #ifdef DEAD_PROCESS
   ut.ut_type =DEAD_PROCESS;
 #endif

Re: runit patches to fix compiler warnings on RHEL 7

[J. Lewis Muir]

On 11/28, Laurent Bercot wrote:
>  - This mailing-list accepts all discussions about process supervision
> software. It also accepts patches to such software (but rather than cold
> sending patches, please engage in a tech discussion first - it doesn't
> have to be long.)

OK, great!  I just sent my patch series in a separate reply.  I'm happy
to have a tech discussion about it, and I could possibly change the
patch series based on the discussion, or if it is determined that my
patch series should not be accepted, I would accept that as well.

>  - The original author or runit is still subscribed to this list, and
> comes from time to time. However, I'm not aware of an official repo for
> runit, and runit's latest official version has been 2.1.2 for many a year
> now.
>  It looks like several distributions have their own version of runit;
> they are maintained by the distros themselves.
> 
>  - We on the list will gladly help with any question with runit, but to
> be honest, I'm not exactly sure what to do with patch upstream requests
> for runit. Is anyone processing them and integrating them into a new
> release?
 
This is unfortunate, but I understand and know that things can happen,
priorities can change, time availability can change, etc.  A proper
upstream would be useful.  A bunch of forks does not seem useful to me.
The versions maintained by distros do not seem useful to me because I
suspect that they would not be interested in patches related to distros
or OSes other than their own.

>  - I host this list. I'm also the author of s6, a supervision software
> suite that is very similar to runit in many ways. s6 is actively
> maintained and has a public git repo, and we generally have a quick
> response time with requests.
> 
>  - My opinion is that the most sustainable path forward, for runit
> users who need a centrally maintained supervision software suite, is to
> just switch to s6 - and it comes with several other benefits as well.

OK, I'm open to that.  Thanks for the suggestion!

I don't need an init replacement, and I initially chose daemontools
because it was the original toolset, and I wanted something that could
start and stop a server process that did not daemonize itself.  But
the server that I wanted to manage took a while to actually become
"available," and daemontools didn't support the concept of a service
becoming available sometime after when it was started.  That's when
I found runit which did support the concept of a service becoming
available with its "sv start" command and a "./check" service script.
This way, I could start it and wait for it to become available before
starting something else.

So, if s6 can do something similar, I'd be happy to try it out!  Can it?

My use case is actually to run it as a systemd service, so briefly
looking at

  https://skarnet.org/software/

I see something called sdnotify-wrapper, so maybe I should have a look
at that?  (It was mentioned to me off-list.)

>  - But again, I'm not impartial, and alternatives are a good thing.
> So no matter what individual decisions are made, it would definitely be
> a net positive if the exact state and workflow of runit could be
> clarified, and if a real development/maintenance structure was in place.

+1.  Agreed.

Lewis

Re: runit patches to fix compiler warnings on RHEL 7

[J. Lewis Muir]

On 12/02, J. Lewis Muir wrote:
> So, if s6 can do something similar, I'd be happy to try it out!  Can it?

Reading more, it seems the answer is yes:

  https://skarnet.org/software/s6/notifywhenup.html

So, s6 has a built-in mechanism where, when the supervised process is
available ("ready" in the s6 nomenclature), it writes a line to a file
descriptor and closes it.

If the supervised process does not support the file descriptor
notification mechanism, a workaround is the s6-notifyoncheck program:

  https://skarnet.org/software/s6/s6-notifyoncheck.html

This all looks promising!  Thank you, Laurent, for writing and
maintaining s6!

Lewis

Re: s6 usability (was: runit patches to fix compiler warnings on RHEL 7)

[Laurent Bercot]

>Would it be acceptable to you and them to put the binaries in /bin/s6
>and then very early in the boot add /bin/s6 to the path? This isn't a
>lot different from what djb did with /command, except it's not off the
>root, which everyone seems to hate.

s6 binaries aren't a problem for Debian; but apparently, execline
binaries are. They're doing exactly that: they're storing execline
binaries under /usr/lib/execline - except they're not putting that
directory in the default PATH, they're only adding it to PATH when
execlineb is invoked. Which breaks some s6 commands in older s6 
versions,
and breaks s6-rc commands (any version), because those assume that
execline binaries can be found in their PATH, and don't call execlineb.

It's completely acceptable to me to put binaries in a separate 
directory.
But said separate directory *needs* to be in the default PATH. If it is
not, it means that those binaries are considered second-class citizens,
and that is *not* acceptable - and even more importantly, it breaks
things.


>As a guy who has both daemontools and s6 installed on the same box, I
>thank you from the bottom of my heart for:
>
>1) Prepending s6- to each command so they don't clash with djb's
>2) Except for the s6-, naming them the same as djb's so I have less to
>    remember.

Yes, there are a good number of people, me included, who prefer that
naming scheme. However, Jan's UX return is valid, and if I want to make
s6 adoption as easy as possible, it needs to be taken into account too.


>A simple change change I think would do it is to change the
>documentation to imply that, for the *user*, execlineb is a way to get
>just a little extra whatever. Currently, when I read it, I thought I'd
>be missing a lot by using /bin/sh.

I think it's already the case, but I'll make a pass on the documentation
to make it abundantly clear.


>Does the *user* need to code execline scripts, or is it just
>something the program does? If the former, then make a point that one
>doesn't need to use execline for s6-rc to be a very powerful startup
>system.

  1. execline is used *internally* by s6-rc, in autogenerated scripts. 
The
user doesn't have to know anything about it.

  2. source oneshot up/down scripts are parsed by execlineb, so this is a
place where users interact with it. However, the interaction can be made
pretty minimal if the up/down script just calls a shell script stored
somewhere else (typically /etc/init.d), which is common, and generally
good, practice.
  For instance, the "source/network-interfaces/up" script could simply
contain:

  /etc/init.d/network start

and that's it. Technically, this is an execline script, made of two 
words,
but nobody needs to know that; and the complex logic of the script can 
be
implemented in the /etc/init.d/network program, which can be written in
any language. Even in a compiled language, if you're a masochist.

Advanced users can write more complex logic in the
source/network-interfaces/up script itself, using execline binaries for
control flow, etc. But it's by no means essential. For all intents and
purposes, that script is just a small command line that execs into 
another
script, hosted wherever you want, written in whatever language you want.
So I'd say the necessary interaction between s6-rc users and execline is
really tiny.


>If anybody would make an execline tutorial, that would help a lot. For
>a guy like me who only does procedural programming (C, C++, Pascal,
>Perl, Python, Ruby, Lua, etc), execline is difficult to understand.

Well, for one, the execline documentation is bad (it was my first real
attempt at writing software documentation, and it shows), so revamping 
it
would probably help a lot. And second, there *are* execline tutorials
online, you just haven't looked.

--
Laurent

Re: runit patches to fix compiler warnings on RHEL 7

[Laurent Bercot]

[-- Attachment #1: Type: text/plain, Size: 2397 bytes --]


>OK, great!  I just sent my patch series in a separate reply.  I'm happy
>to have a tech discussion about it, and I could possibly change the
>patch series based on the discussion, or if it is determined that my
>patch series should not be accepted, I would accept that as well.

Eh, "fix old compiler warnings" doesn't require much discussion :)


>That's when
>I found runit which did support the concept of a service becoming
>available with its "sv start" command and a "./check" service script.
>This way, I could start it and wait for it to become available before
>starting something else.
>
>So, if s6 can do something similar, I'd be happy to try it out!  Can it?

Yes it can, and it does you one better: if your service supports it, s6
can tell you that it's ready as soon as the service says it is, without
the need for polling at all.
(https://skarnet.org/software/s6/notifywhenup.html for the tech 
details.)

And s6 has, for instance, commands that block until a service is ready,
so you can write your startup sequence without "sv check" shenanigans.


>I see something called sdnotify-wrapper, so maybe I should have a look
>at that?  (It was mentioned to me off-list.)

sdnotify-wrapper will only be useful if your daemon is using the
NOTIFY_SOCKET mechanism (aka sd_notify()) to tell systemd when it is
ready. If it's the case, then yes, you can use sdnotify-wrapper in your
s6 run script and it should automatically do the right thing.

But if you have control over your daemon's source, or the daemon 
supports
it natively (typically via a command line option), the s6 mechanism for
readiness notification is much simpler and lighter. It's just "the
daemon writes a line of text when it's ready, on any file descriptor it
wants, and then closes that file descriptor". For instance, if your
daemon writes "ok" to its stdout when it's ready (and doesn't write
any newline beforehand), you can just use that to inform s6.

And if your daemon doesn't do anything of the sort, you can still poll
it with a "./check" script (or "./data/check" in s6 parlance) as you do
with runit: just prepend your run script with
https://skarnet.org/software/s6/s6-notifyoncheck.html
and your check script will be automatically invoked as necessary, and
the result will be directly piped into the readiness notification
mechanism.

--
Laurent

Re: s6 usability (was: runit patches to fix compiler warnings on RHEL 7)

[Laurent Bercot]

>sure, that was just an idea for Jan, he could just create a dir somewhere,
>populate it with symlinks he prefers to the original s6 tools and put this dir
>in front of the PATH when running s6 since it seems the utilities do not bother under what name they run.

Right, but I've heard enough people complain about s6's UI that a 
one-stop
wrapper command sounds like a good idea anyway.


>ok. i was more about insights into the design of the whole s6-rc toolset.
>are the up/down scripts run by a dedicated service from within the supervision
>tree? what exactly is the task of the "s6-rc-oneshot-run" and
>"s6-rc-fdholder-filler" internal programs ? how is the startup organized,
>how are "longruns" and "oneshots" intertwined ?
>
>having to read the sources to get this information is somewhat inconvenient.

  This is exactly what I was saying: I'm not documenting those details
because I don't want to be bound by them. The s6-rc-fdholder-filler API
changed right before 0.2.0.0; making this necessary change would have 
been
a lot more difficult if people had relied on details of the interface.

  If you're doing something that requires knowledge of the internal 
programs,
you're definitely good enough to read the code. :P

--
  Laurent

Re: runit patches to fix compiler warnings on RHEL 7

[Laurent Bercot]

>Reading more, it seems the answer is yes:

Our replies crossed. Glad you found what you needed in the doc. :)

--
Laurent

Re: s6 usability

[Samuel Holland]

On 12/2/19 3:32 PM, Laurent Bercot wrote:
>> As a guy who has both daemontools and s6 installed on the same box, I
>> thank you from the bottom of my heart for:
>>
>> 1) Prepending s6- to each command so they don't clash with djb's
>> 2) Except for the s6-, naming them the same as djb's so I have less to
>>    remember.
> 
> Yes, there are a good number of people, me included, who prefer that
> naming scheme. However, Jan's UX return is valid, and if I want to make
> s6 adoption as easy as possible, it needs to be taken into account too.

From a Linux distribution perspective, there's also the question of if s6 can be
made a drop-in replacement for daemontools, since it does follow djb's naming
scheme. In gentoo, there are various packages that depend on
virtual/daemontools; for example, the nullmailer test suite uses ipcserver. From
a quick comparison of the documentation, it looks like s6 only adds options, and
remains compatible with the daemontools options.

So would it be valid/acceptable for a distribution to create unprefixed symlinks
to the s6-* binaries? It looks like this would mostly only work for the subset
of the binaries that implement daemontools functionality; some others
(s6-setsid, s6-sudo) would have naming conflicts if they were not prefixed.

Then, with the symlinks, s6 could "provide" virtual/daemontools. Maybe this
would also help discoverability (the issue at hand). Maybe the inconsistency
would cause more harm than good, and the symlinks should be "for compatibility
only".

Thoughts?

Samuel

Re: runit patches to fix compiler warnings on RHEL 7

[Benjamin Franksen]

Am 02.12.19 um 22:58 schrieb Laurent Bercot:
>> I see something called sdnotify-wrapper, so maybe I should have a look
>> at that?  (It was mentioned to me off-list.)
> 
> sdnotify-wrapper will only be useful if your daemon is using the
> NOTIFY_SOCKET mechanism (aka sd_notify()) to tell systemd when it is
> ready. If it's the case, then yes, you can use sdnotify-wrapper in your
> s6 run script and it should automatically do the right thing.

Hi,

I was the one who told Lewis about sdnotify-wrapper and it seems I got
it completely wrong: I had thought it was a wrapper for daemons that use
the s6 notification mechanism to make them compatible with the systemd
convention. Your description above suggests it does the exact opposite.

Cheers
Ben

Re: s6 usability

[Steve Litt]

On Mon, 2 Dec 2019 17:17:50 -0600
Samuel Holland <samuel@sholland.org> wrote:

> On 12/2/19 3:32 PM, Laurent Bercot wrote:
> >> As a guy who has both daemontools and s6 installed on the same
> >> box, I thank you from the bottom of my heart for:
> >>
> >> 1) Prepending s6- to each command so they don't clash with djb's
> >> 2) Except for the s6-, naming them the same as djb's so I have
> >> less to remember.  
> > 
> > Yes, there are a good number of people, me included, who prefer that
> > naming scheme. However, Jan's UX return is valid, and if I want to
> > make s6 adoption as easy as possible, it needs to be taken into
> > account too.  
> 
> From a Linux distribution perspective, there's also the question of
> if s6 can be made a drop-in replacement for daemontools, since it
> does follow djb's naming scheme. In gentoo, there are various
> packages that depend on virtual/daemontools; for example, the
> nullmailer test suite uses ipcserver. From a quick comparison of the
> documentation, it looks like s6 only adds options, and remains
> compatible with the daemontools options.
> 
> So would it be valid/acceptable for a distribution to create
> unprefixed symlinks to the s6-* binaries? It looks like this would
> mostly only work for the subset of the binaries that implement
> daemontools functionality; some others (s6-setsid, s6-sudo) would
> have naming conflicts if they were not prefixed.
> 
> Then, with the symlinks, s6 could "provide" virtual/daemontools.
> Maybe this would also help discoverability (the issue at hand). Maybe
> the inconsistency would cause more harm than good, and the symlinks
> should be "for compatibility only".
> 
> Thoughts?

In my opinion, 99% of all people currently using daemontools are highly
technically proficient and could easily either rename commands or
prepath to a directory containing prefixless symlinked synonyms. IMHO
if the distro does it, they'll find a way to screw it up. And even if
the distro does it right, it will screw that 1 in a million people
(like me) who occasionally use daemontools and s6 on the same box,
switching between them regularly.

Personally, I'd leave well enough alone.

SteveT

Steve Litt 
December 2019 featured book: Rapid Learning for the 21st Century
http://www.troubleshooters.com/rl21

Re: s6 usability (was: runit patches to fix compiler warnings on RHEL 7)

[Casper Ti. Vector]

On Sun, Dec 01, 2019 at 09:47:52PM -0500, Steve Litt wrote:
> Would it be acceptable to you and them to put the binaries in /bin/s6
> and then very early in the boot add /bin/s6 to the path? This isn't a
> lot different from what djb did with /command, except it's not off the
> root, which everyone seems to hate.

Or can we consider the Plan 9 style [1], which searches all relative
paths (in the broad sense, i.e. all which do not begin with `/', `./' or
`../') from $PATH, so we can install chainloaders into /bin/s6 and then
use `s6/cd' to run `/bin/s6/cd' in execline?

[1] <https://github.com/rakitzis/rc/issues/44>

I fully understand that this convention is not followed in most Unix
shells (except for rc(1) which is from Plan 9, and perhaps some others),
but execline is not a shell, and we can additionally symlink /bin/s6/*
into /bin for backward compatibility.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: runit patches to fix compiler warnings on RHEL 7

[Jonathan de Boyne Pollard]

Laurent Bercot:

> It looks like several distributions have their own version of runit; 
> they are maintained by the distros themselves.
>
Further to all that:  I believe, although things may have changed, that 
the Debian maintainer for runit is open to patches.

* https://tracker.debian.org/pkg/runit

Re: runit patches to fix compiler warnings on RHEL 7

[Jonathan de Boyne Pollard]

Some of those are common.  I looked at similar stuff in daemontools, 
where runit got some of this code from, when I packaged it up with some 
of the other Bernstein softwares some years ago.  However, you have 
missed the point of HASSHORTSETGROUPS.  There's no point in having 
conditionally compiled code selected by it that uses gid_t in both 
codepaths.  If you are going to use gid_t, you do not need the 
HASSHORTSETGROUPS mechanism in its entirety.  Think about what it does.

* http://jdebp.uk./Softwares/djbwares/

In the nosh toolset, I have no such mechanisms.  The code uses the 
<unistd.h> types, with a std::vector<gid_t> for the call to setgroups() 
in setuidgid-fromenv for example.  The only similar mechanism picks 
between the very old waitpid() function and the newer (but still old, 
because it was SVR4) waitid() function.  And only OpenBSD ever needs the 
code to use the former, in practice.

Re: runit patches to fix compiler warnings on RHEL 7

[Jonathan de Boyne Pollard]

Jan Braun:

> 2) runit has manpages. s6 has HTML. :(
>
Daniel J. Bernstein had something to say on that subject, two decades 
ago.  See the "Notes" section of http://cr.yp.to/slashdoc.html .

I generate both manual pages and HTML from a common DocBook XML master 
in the nosh toolset.  And the DocBook XML is itself readable directly 
with a WWW browser. 
http://jdebp.uk./Softwares/nosh/guide/commands/setuidgid-fromenv.xml is 
a copy of one such DocBook XML master, for example.  It's on the WWW, 
and the packages also install it locally, for off-line reading.

M. Pape did some of the manual pages for some operating system's 
versions of daemontools, converting M. Bernstein's HTML pages into 
roff.  For djbwares I converted everything into DocBook XML, and the 
same holds for djbwares as for the nosh toolset.  There is a DocBook XML 
master that one can view in a WWW browser directly (both on-line and 
off-line), generated HTML pages, and generated manual pages readable 
with man. 
http://jdebp.uk./Softwares/djbwares/guide/commands/setuidgid.xml is a 
copy of one such DocBook XML master, for example.  This is the source 
for the "man setuidgid" manual and the source for 
http://jdebp.uk./Softwares/djbwares/guide/setuidgid.html .

I even filled in the manual pages that M. Pape hadn't done and that M. 
Bernstein hadn't originally written in HTML, and updated some of the 
doco.  See 
http://jdebp.uk./Softwares/djbwares/guide/commands/caldate_easter.xml 
and http://jdebp.uk./Softwares/djbwares/guide/commands/dnscache.xml for 
examples of that.

Re: s6 usability

[Jonathan de Boyne Pollard]

Samuel Holland:

> Then, with the symlinks, s6 could "provide" virtual/daemontools.
>
I do this with the nosh toolset already, for Debian.  Several of the 
"shim" and "-run" packages, which are separate from the main tools 
packages, "provide" stuff.  I even have  a things such as a dummy 
"nosh-logrotate-shims" package that provides "logrotate", which is 
required by things like the "rabbitmq-server" package but isn't actually 
needed at all in order to run rabbitmq-server under nosh service 
management.  And a "nosh-run-bcron" that "provides" "bcron-run".

* http://jdebp.uk./Softwares/nosh/debian-binary-packages.html#Toolsets

* http://jdebp.uk./Softwares/nosh/debian-binary-packages.html#run

Re: runit patches to fix compiler warnings on RHEL 7

[J. Lewis Muir]

On 12/04, Jonathan de Boyne Pollard wrote:
> Jan Braun:
> 
> > 2) runit has manpages. s6 has HTML. :(
> > 
> Daniel J. Bernstein had something to say on that subject, two decades ago.
> See the "Notes" section of http://cr.yp.to/slashdoc.html .
> 
> I generate both manual pages and HTML from a common DocBook XML master in
> the nosh toolset.  And the DocBook XML is itself readable directly with a
> WWW browser.
> http://jdebp.uk./Softwares/nosh/guide/commands/setuidgid-fromenv.xml is a
> copy of one such DocBook XML master, for example.  It's on the WWW, and the
> packages also install it locally, for off-line reading.

I still like having man pages.  It's often just easier to type "man
<name>" than to find the local (or remote) HTML document and open it in
a web browser.

However, I agree that it's very nice to have HTML as well.  So, I like
to have both!  It seems good to generate them from a single source
format.  I would like DocBook except that the toolchain seems *so*
heavy.  And if you want to generate PDFs, it's even heavier.

What about mandoc?

  http://mandoc.bsd.lv/

It seems pretty lightweight, and from an mdoc source, it can generate
ASCII, HTML, man, PDF, and PostScript.

Lewis

Re: runit patches to fix compiler warnings on RHEL 7

[Laurent Bercot]

[-- Attachment #1: Type: text/plain, Size: 736 bytes --]

>What about mandoc?

The colour of this bikeshed is not up for debate.

If you want man pages for skaware, provide me with:
1. a reasonable source format (e.g. not roff, so mandoc is right out)
2. a tool that can be built using *only* a C compiler (so as to keep
bootstrapping skaware easy), that converts the source format into man
pages *and* into HTML
3. the actual contents of the man pages you want, in that source format.

(https://git.sr.ht/~sircmpwn/scdoc fits 1. and 2., but its author wasn't
motivated enough to do 3.)

Until then, any further discussion of documentation formats is pure
noise, I've heard it all before - several times - and it has the 
potential
to piss me off VERY quickly.

--
Laurent

Re: s6 usability

[Laurent Bercot]

>From a Linux distribution perspective, there's also the question of if s6 can be
>made a drop-in replacement for daemontools, since it does follow djb's naming
>scheme. In gentoo, there are various packages that depend on
>virtual/daemontools; for example, the nullmailer test suite uses ipcserver. From
>a quick comparison of the documentation, it looks like s6 only adds options, and
>remains compatible with the daemontools options.

Yes, and that was on purpose, but it's only true with the "official" API
and not with the internals. For instance, the supervise/status file
isn't compatible.


>So would it be valid/acceptable for a distribution to create unprefixed symlinks
>to the s6-* binaries? It looks like this would mostly only work for the subset
>of the binaries that implement daemontools functionality; some others
>(s6-setsid, s6-sudo) would have naming conflicts if they were not prefixed.

s6-setsid can be symlinked as pgrphack: same functionality, different
name. The same way that s6-log can be symlinked as multilog.


>Then, with the symlinks, s6 could "provide" virtual/daemontools. Maybe this
>would also help discoverability (the issue at hand). Maybe the inconsistency
>would cause more harm than good, and the symlinks should be "for compatibility
>only".

s6 can provide at least surface compatibility with daemontools, yes. 
With
the symlinks, it can still be a drop-in replacement (unless there are
interface changes I haven't thought about).

daemontools replacement is easy. The real subject is compatibility with
runit, which is possible but not quite drop-in.

--
Laurent

Re: runit patches to fix compiler warnings on RHEL 7

[Steve Litt]

On Wed, 4 Dec 2019 10:40:14 -0600
"J. Lewis Muir" <jlmuir@imca-cat.org> wrote:

> On 12/04, Jonathan de Boyne Pollard wrote:
> > Jan Braun:
> >   
> > > 2) runit has manpages. s6 has HTML. :(
> > >   
> > Daniel J. Bernstein had something to say on that subject, two
> > decades ago. See the "Notes" section of
> > http://cr.yp.to/slashdoc.html .
> > 
> > I generate both manual pages and HTML from a common DocBook XML
> > master in the nosh toolset.  And the DocBook XML is itself readable
> > directly with a WWW browser.
> > http://jdebp.uk./Softwares/nosh/guide/commands/setuidgid-fromenv.xml
> > is a copy of one such DocBook XML master, for example.  It's on the
> > WWW, and the packages also install it locally, for off-line
> > reading.  
> 
> I still like having man pages.  It's often just easier to type "man
> <name>" than to find the local (or remote) HTML document and open it
> in a web browser.

I never thought about man pages until a few days ago I did some
experiments with execline and had a quick question about syntax. I did
man execlineb and of course got "no entry". So I fired up a browser,
did a locate command, and put a path in my browser.

The browser is vastly superior for learning all about
unfamiliar or moderately familiar software, but for the quick lookup of
something you primarily know about, there's no substitute for a quick
"man execlineb".
 
SteveT

Steve Litt 
December 2019 featured book: Rapid Learning for the 21st Century
http://www.troubleshooters.com/rl21

Re: runit patches to fix compiler warnings on RHEL 7

[J. Lewis Muir]

On 12/04, Laurent Bercot wrote:
> > What about mandoc?
> 
> The colour of this bikeshed is not up for debate.
> 
> If you want man pages for skaware, provide me with:
> 1. a reasonable source format (e.g. not roff, so mandoc is right out)
> 2. a tool that can be built using *only* a C compiler (so as to keep
> bootstrapping skaware easy), that converts the source format into man
> pages *and* into HTML
> 3. the actual contents of the man pages you want, in that source format.
> 
> (https://git.sr.ht/~sircmpwn/scdoc fits 1. and 2., but its author wasn't
> motivated enough to do 3.)
 
Interesting, I hadn't heard of scdoc before.  I don't see where it says
it can convert the source format into HTML, though.  Did I miss that?

> Until then, any further discussion of documentation formats is pure
> noise, I've heard it all before - several times - and it has the potential
> to piss me off VERY quickly.

Certainly don't want to cause trouble, and not intending to bikeshed,
but I searched for "halibut" in the archive for this list as well as the
skaware list and did not find that it had been mentioned, so I'll just
mention that another system I've used in the past is Halibut which is
lightweight and written in ANSI C:

  https://www.chiark.greenend.org.uk/~sgtatham/halibut/

It can generate ASCII, HTML, PDF, PostScript, man, and info.  So, I
think it would satisfy #1 and #2, but certainly not #3.  Still, if
someone wanted to do the work to provide #3 at some point, then Halibut
may be a reasonable tool to consider.

Lewis

Re: runit patches to fix compiler warnings on RHEL 7

[Laurent Bercot]

[-- Attachment #1: Type: text/plain, Size: 283 bytes --]


>The browser is vastly superior for learning all about
>unfamiliar or moderately familiar software, but for the quick lookup of
>something you primarily know about, there's no substitute for a quick
>"man execlineb".

  https://lmgtfy.com/?q=execlineb&iie=1

--
  Laurent

Re: runit patches to fix compiler warnings on RHEL 7

[Laurent Bercot]

>  I hope you find this comment useful.

  Yes, it was a very useful UX return, thank you.

  I think I'm going to work on an early version of s6-frontend, which
will only provide daemontools-style and/or runit-style command emulation
(depending on build-time options), and maybe a preliminary version of
the "s6" generic command as well. That way, at least the need for short
commands will be met, and the daemontools/runit muscle memory can be
actually leveraged for s6.

--
  Laurent

Re: s6 usability

[Jan Braun]

[-- Attachment #1: Type: text/plain, Size: 7957 bytes --]

Hey,
and sorry for taking so long to reply.

Laurent Bercot schrob:
> > 1) Debian ships with a working and maintained runit-init package. It
> >    provides pid 1 and hooks into /etc/rcS.d/* to integrate with other
> >    Debian packages. s6-linux-init and s6-rc are not packaged in Debian.
> 
> I hear you. Unfortunately, I have no control over what Debian does.
> Debian isn't even able to ship a not-broken execline package, so I'm at
> a loss on what to do with them. I'm working on a version of execline that
> they *might* accept to package correctly,

If you're referring to
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=906250#37
then, well, you are fighting against POSIX. There's little choice for
Debian in the matter. Taking a hardline stance on such "legal" issues is
part of their identity as a distro.

> but it's doubtful as long as
> the people in charge are prejudiced against the "lots of small binaries
> in /bin" approach. :(

I don't think that's the real issue. POSIX saying "the cd/umask/wait
shell builtin must also be exec()able" is. Or execline's desire for
binaries with those names and different behaviour, depending on your
viewpoint.



> > 2) runit has manpages. s6 has HTML. :(
> 
> This is far from the first time I hear this, so it would be foolish to
> ignore it, but I *really* have a hard time understanding how in 2019 it's
> so difficult for people to launch a browser to read a web page.

It's not difficult launching the browser, it's difficult getting to the
correct webpage. Compare
| $ elinks /usr/share/doc/s6/s6-log.html
with a hypothetical
| $ man s6-log
and count keystrokes. (And then look up the K key in vim. ;)
The main advantage of manpages is that they are in a well-known place
and therefore instantly found. The displaying itself makes hardly a
difference. Especially since, thankfully, your html mimicks the man page
conventions wrt sections and so on.

Something like djb's /doc (thanks, Jonathan!) would scratch my itch too,
but as long as that's not in widespread use, I prefer man over html.

(I also have several rants about html/browsers/JS available on request,
but that's rather tangential to the issue of *finding* documentation.)



> [... 3) ...]
> The number of executables is a choice; I like to have more, smaller,
> executables than less, bigger ones. [...]
> So, I suppose it comes down to individual preference there.

I agree, it's probably personal UI taste. To me, a good metric is the
"fanout" of possible options:
When I want to call something runit-related, I got
    * chpst: change process state
        **  -u: user
        **  -n: nofiles
        ...
    - sv: manipulate a service
        ** up
        ** down
        ** term
        ...
    - runsv<tab>    supervision implemention
        ** runsv     one service
        ** runsvdir  a dir of services

    - sv-<tab> my custom scripts
        ** sv-errors  show the readproctitle logs
        ...

If any point in the tree, there's 7±2 children I can find by
tab-completing/glancing at a man page, then I can probably navigate the
whole thing pretty quickly.

On the other hand:
| $ ls /usr/bin/s6* | cut -d- -f 2 | uniq | wc -l
| 39
Even if there's still some related commands (e.g. 3x s6-sudo*), that
number is not coming down to a point where I can keep everything in
my head.


> Would a generic "s6" command, that takes alternative syntax and rewrites
> itself into "internal" commands, help? It could emulate runit syntax,
> among other things.
> 
> s6 runsv ... -> s6-supervise ...
> s6 sv ... -> s6-svc ...
> s6 chpst ... -> various s6-prefixed process state change commands

Probably yes, but if you are doing that, then why don't you look at
argv[0] and provide the runit interface proper? :D
(Or provide runsv/sv/chpst/.. as individual binaries, since you prefer
those.)


But outside of supervision, I notice that you are reimplementing a
lot of small programs. As long as they're mostly command-line compatible
with their inspiration, I think such a "s6" executable would enable a
nicer UI for s6-{linux,portable}-utils and the s6 djb reimplementations:

Have your binaries in /usr/lib/s6/ (or wherever), and named without the
"s6-" prefix. Have a "s6" binary multiplexing to these on its first
argument. That way, a user can choose any of:

| $ s6 cat foo
|
| $ PATH="/usr/lib/s6/:$PATH"
| $ cat foo
|
| # ln -sf /usr/lib/s6/cat /bin/cat
| $ cat foo

(And possibly a s6-cat compatibility symlink somewhere.)
This is similar to busybox (except for the reverse goal :) and allows
individual customization, while keeping /bin/ uncluttered by default.

And with all those reimplementations out of the way, maybe the remaining
binaries are few enough that I can actually remember them. Of course, if
you then want to rename s6-supervise to runsv, and s6-svscan to
runsvdir, that might even work. ;)



> > 4) s6 seems more complex (hello execline!), and I don't (yet?) see any
> >    benefit/feature I'd appreciate except minimizing wakeups.
> 
> This, on the other hand, is a misconception that really needs to
> disappear. Understanding execline is *not needed* to run s6.

Needing to *understand* execline wasn't my misconception, nor worry. But
when I'm told that a runit-lookalike depends on bringing its own
scripting language along, then that sounds more like systemd and less
like djb to me. :(

> s6 depends
> on execline in two places (there were more, but I scrapped them because
> nobody used the commands that were involved):
> - at build-time, in s6-ftrig-listen
> - at run-time, in s6-log, for processor invocation
> 
> Would entirely removing s6's dependency on execline help clear that
> misunderstanding and help with s6 adoption?

When you explain it like that, it sounds far more reasonable.

> This could be made possible
> by:
> - duplicating el_semicolon() functionality in s6-ftrig-listen.c
> (it's not elegant, but clearing the dep may be worth it)

I can't judge that.

> - adding an alternative '?' processor directive to s6-log, that spawns
> the processor using /bin/sh instead of execlineb. (The '!' directive
> would still be there; processor invocations using '!' would just fail
> if execline is not installed.)

I'd appreciate this regardless of the internal dependency. I know you
want to popularize execline, but "you must use it if you want to use my
other tools" is not a helpful form of advocacy.

> I don't like this, but if "execline" is a scarecrow that keeps people
> away from s6 for no other reason than perception, it's a possibility.
> Savvy users will still install execline for use in run scripts.

That's perfectly fine. Small modular building blocks you can use,
disregard, or replace as you see fit are Unixy.



> s6-rc, however, absolutely cannot do without execline, since it uses
> autogenerated execline scripts. But s6-rc is a different beast, that
> requires a lot more involvement than s6 anyway, and that isn't needed
> at all if we're just talking about runit-like functionality.

Yes. But since you are mentioning it, that was another of my "s6 seems
more complex" issues. runit goes from "start the supervisor manually" to
"be pid 1" with very little effort. See runit(8).
Or https://www.unix.com/man-page/debian/8/runit/ I guess. ;-P

s6-linux-init and s6-rc seem extremely complicated in comparison. And
it's not clear to me how much of that complexity is optional, and what
it might buy me. Maybe a better high-level overview document might help
there.


cheers, and sorry for the wall of text,
    Jan

P.S: I stumbled over this execline oddity:
| dollarat     -0 -d a                      # separates by \0
| forbacktickx -0 -d a var {gen...} loop... # splits on a
IMHO, both should be an error, but at least treat them the same.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: s6 usability

[Jan Braun]

[-- Attachment #1: Type: text/plain, Size: 3825 bytes --]

Steve Litt schrob:
> > From a Linux distribution perspective, there's also the question of
> > if s6 can be made a drop-in replacement for daemontools, [...]
> 
> In my opinion, 99% of all people currently using daemontools are highly
> technically proficient and could easily either rename commands or
> prepath to a directory containing prefixless symlinked synonyms. IMHO
> if the distro does it, they'll find a way to screw it up. And even if
> the distro does it right, it will screw that 1 in a million people
> (like me) who occasionally use daemontools and s6 on the same box,
> switching between them regularly.

I find these claims to be rather unplausible, given my Debian
experience.

The Debian way would be creating a package "s6-daemontools" containing
just the symlinks from foo to s6-foo and declaring the following package
relationships:
    Depends: s6
    Conflicts: daemontools
    Provides: daemontools
That is, when s6-daemontools is installed:
    s6 must also be installed,
    daemontools must not be installed,
    but daemontools is treated as installed for dependency purposes.

Then, you can choose to install s6 and daemontools together, getting
the default naming for each.

And I can install s6-daemontools and get tai64n/tai64nlocal binaries
that produce correct timestamps despite my kernel clock being set to
UTC. (That is, the ones from s6.)

And every dependency on "daemontools" will be fulfilled, by exactly one
of either djb's daemontools, or s6 plus the symlinks in s6-daemontools.

See espeak-ng-espeak for a real package like that. This is hardly rocket
science, and has been done for *decades* by now.


Maybe non-Debian(-derived) distros are in the business of doing stupid
sh*t, but then you should either switch to a sane distro or be building
your own system. It's certainly no reason to refuse to consider the "s6
as daemontools" concept. Or to insist that people must setup the
symlinks themselves.


Onwards:
Comparing the debian daemontools package with s6 gives:

| $ dpkg -L daemontools  | grep /usr/bin/ | sed -E 's_/usr/bin/_/usr/bin/s6-_' | xargs ls -1
| ls: cannot access '/usr/bin/s6-multilog': No such file or directory
| ls: cannot access '/usr/bin/s6-pgrphack': No such file or directory
| ls: cannot access '/usr/bin/s6-readproctitle': No such file or directory
| ls: cannot access '/usr/bin/s6-svscanboot': No such file or directory
| /usr/bin/s6-envdir
| /usr/bin/s6-envuidgid
| /usr/bin/s6-fghack
| /usr/bin/s6-setlock
| /usr/bin/s6-setuidgid
| /usr/bin/s6-softlimit
| /usr/bin/s6-supervise
| /usr/bin/s6-svc
| /usr/bin/s6-svok
| /usr/bin/s6-svscan
| /usr/bin/s6-svstat
| /usr/bin/s6-tai64n
| /usr/bin/s6-tai64nlocal
| $ 

As Laurent said in the other mail:
> s6-setsid can be symlinked as pgrphack [...]
> s6-log can be symlinked as multilog.

That leaves only readproctitle and svscanboot.
The latter is just clearing env, setting $PATH and executing
    svscan /service 2>&1 | readproctitle service errors: .....
with 400 dots, according to the manpage.

But I don't think there's an equivalent of readproctitle in s6.

Plus, there remains the issue of non-compatible supervise directories.
I don't think the above packaging scheme is valid if it would mean
services (with their existing supervise dirs) fail to work because the
supervise(8) implementation got switched from djb to s6 or vice versa.

But in my quick testing they actually could re-use each other's
supervise dir without tripping over the other implementation's
artifacts.¹

Laurent, is that something that should work/you are willing to support?

In that case, readproctitle would be the only remaining obstacle.

regards,
    Jan

¹ They would not recognize each other's locking, however.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: s6 usability

[Guillermo]

El sáb., 21 dic. 2019 a las 6:26, Jan Braun escribió:
>
> > > 1) Debian ships with a working and maintained runit-init package.
> [...]
> > I hear you. Unfortunately, I have no control over what Debian does.
> [...]
> If you're referring to
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=906250#37
> then, well, you are fighting against POSIX. There's little choice for
> Debian in the matter. Taking a hardline stance on such "legal" issues is
> part of their identity as a distro.

Trying to accomodate Debian is probably a waste of time at the moment,
until the results of the ongoing General Resolution vote are known
next weekend.

> It's not difficult launching the browser, it's difficult getting to the
> correct webpage. Compare
> | $ elinks /usr/share/doc/s6/s6-log.html

'elinks /usr/share/doc/s6/index.html' and then navigate? :)

> > Would a generic "s6" command, that takes alternative syntax and rewrites
> > itself into "internal" commands, help?
> [...]
> Probably yes, but if you are doing that, then why don't you look at
> argv[0] and provide the runit interface proper? :D

That would create a 'multple personality binary'.

> (Or provide runsv/sv/chpst/.. as individual binaries, since you prefer
> those.)

That could prevent installing both s6 and runit, depending on
packaging, Same as s6 and daemontools[-encore] if the s6- prefix in
program names was dropped.

> > > 4) s6 seems more complex (hello execline!), and I don't (yet?) see any
> > >    benefit/feature I'd appreciate except minimizing wakeups.
> >
> > This, on the other hand, is a misconception that really needs to
> > disappear. Understanding execline is *not needed* to run s6.
>
> Needing to *understand* execline wasn't my misconception, nor worry. But
> when I'm told that a runit-lookalike depends on bringing its own
> scripting language along, then that sounds more like systemd and less
> like djb to me. :(
> [...]
> I know you
> want to popularize execline, but "you must use it if you want to use my
> other tools" is not a helpful form of advocacy.

If there is no misconception about the need to understand execline,
then I find this criticism quite odd. It's like complaining that a GUI
application 'imposes' e.g. Qt, or that Xorg 'imposes' X11 video and
input drivers. As long as it is a dependency (i.e. an implementation
detail from the POV of a user), if fail to see the problem. I would
understand if it was e.g. a big and intrusive dependency, or a
dependency that prevented you from installing other packages, but
execline isn't that, so I don't see how this compares to systemd.

> But since you are mentioning it, that was another of my "s6 seems
> more complex" issues. runit goes from "start the supervisor manually" to
> "be pid 1" with very little effort. See runit(8).
> Or https://www.unix.com/man-page/debian/8/runit/ I guess. ;-P
>
> s6-linux-init and s6-rc seem extremely complicated in comparison.

s6 + s6-rc vs runit is not a good comparison. One alternative provides
a service manager, the other one doesn't. Not equivalent feature sets.
s6  + s6-linux-init vs runit would be a better comparison
feature-wise. But, if one takes 1) into account, this is kind of
abstract, because Debian currently packages neither s6-linux-init nor
s6-rc.

G.

Re: s6 usability

[Colin Booth]

On Sat, Dec 21, 2019 at 10:26:39AM +0100, Jan Braun wrote:
> > I hear you. Unfortunately, I have no control over what Debian does.
> > Debian isn't even able to ship a not-broken execline package, so I'm at
> > a loss on what to do with them. I'm working on a version of execline that
> > they *might* accept to package correctly,
> 
> If you're referring to
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=906250#37
> then, well, you are fighting against POSIX. There's little choice for
> Debian in the matter. Taking a hardline stance on such "legal" issues is
> part of their identity as a distro.
>
It doesn't help that neither Adam nor Jakub read the documentation for
the execline equivalents for cd, umask, or wait. That or they don't know
what 'execs into' means.
> 
> > but it's doubtful as long as
> > the people in charge are prejudiced against the "lots of small binaries
> > in /bin" approach. :(
> 
> I don't think that's the real issue. POSIX saying "the cd/umask/wait
> shell builtin must also be exec()able" is. Or execline's desire for
> binaries with those names and different behaviour, depending on your
> viewpoint.
> 
Within the context of a shell the builtin will always* take precedence. 

*I'm sure there's a way to break your shell so that it doesn't but I
don't know what it is and anyway it would be an impressively stupid
thing to do.
> 
> > [... 3) ...]
> > The number of executables is a choice; I like to have more, smaller,
> > executables than less, bigger ones. [...]
> > So, I suppose it comes down to individual preference there.
> 
> I agree, it's probably personal UI taste. To me, a good metric is the
> "fanout" of possible options:
> When I want to call something runit-related, I got
>     * chpst: change process state
>         **  -u: user
>         **  -n: nofiles
>         ...
>     - sv: manipulate a service
>         ** up
>         ** down
>         ** term
>         ...
>     - runsv<tab>    supervision implemention
>         ** runsv     one service
>         ** runsvdir  a dir of services
> 
>     - sv-<tab> my custom scripts
>         ** sv-errors  show the readproctitle logs
>         ...
> 
> If any point in the tree, there's 7±2 children I can find by
> tab-completing/glancing at a man page, then I can probably navigate the
> whole thing pretty quickly.
> 
> On the other hand:
> | $ ls /usr/bin/s6* | cut -d- -f 2 | uniq | wc -l
> | 39
> Even if there's still some related commands (e.g. 3x s6-sudo*), that
> number is not coming down to a point where I can keep everything in
> my head.
> 
I think the real problem here is not that there are too many things
prefixed s6- but that there isn't a super clear delineation between
user-facing commands (s6-svc, s6-svscanctl, s6-svstat, s6-svok, s6-sudo,
s6-tai64nlocal), the portions that belong directly to the supervisor
(s6-svscan, s6-supervise, s6-ftrigrd), and the parts that are only used
occasionally or are meant to be called from inside of scripts (most
everything else). FWIW, runit is actually worse in terms of separating
out what goes where from a documentation standpoint, however there are
only nine programs shipped with runit so it matters less.
> 
> But outside of supervision, I notice that you are reimplementing a
> lot of small programs. As long as they're mostly command-line compatible
> with their inspiration, I think such a "s6" executable would enable a
> nicer UI for s6-{linux,portable}-utils and the s6 djb reimplementations:
> 
> Have your binaries in /usr/lib/s6/ (or wherever), and named without the
> "s6-" prefix. Have a "s6" binary multiplexing to these on its first
> argument. That way, a user can choose any of:
> 
> | $ s6 cat foo
> |
> | $ PATH="/usr/lib/s6/:$PATH"
> | $ cat foo
> |
> | # ln -sf /usr/lib/s6/cat /bin/cat
> | $ cat foo
> 
> (And possibly a s6-cat compatibility symlink somewhere.)
> This is similar to busybox (except for the reverse goal :) and allows
> individual customization, while keeping /bin/ uncluttered by default.
> 
Have you ever considered slashpackage ;)

In all seriousness though this, with the exception of dropping the s6-
prefix (and the prefix-appender binary I guess), is what slashpackage
does. /bin stays uncluttered, commands end up in a PATH-able place, and
if you want to surprise any systemic shell scripts you have you can
symlink in replacements to the default PATH. 
> 
> > s6-rc, however, absolutely cannot do without execline, since it uses
> > autogenerated execline scripts. But s6-rc is a different beast, that
> > requires a lot more involvement than s6 anyway, and that isn't needed
> > at all if we're just talking about runit-like functionality.
> 
> Yes. But since you are mentioning it, that was another of my "s6 seems
> more complex" issues. runit goes from "start the supervisor manually" to
> "be pid 1" with very little effort. See runit(8).
> Or https://www.unix.com/man-page/debian/8/runit/ I guess. ;-P
> 
> s6-linux-init and s6-rc seem extremely complicated in comparison. And
> it's not clear to me how much of that complexity is optional, and what
> it might buy me. Maybe a better high-level overview document might help
> there.
> 
If you're trying to have a drop-in replacement for runit(8) then
s6-linux-init (specifically s6-linux-init-maker) is all you really need
to figure out, and then only when building out a system. Then you need
to do some editing of the generated and relocated scripts to get it to
work right. Of course, setting up runit's 1 takes a bit of doing if you
want it to do anything else than put you into singleuser. 

If you want a system that's modern (with service dependencies and all
that) you need to set up s6-rc but it is by no means a requirement.
s6-l-i on its own gets you a pretty compact boot system that's pretty
well suited for running purpose built VMs since you can handle all the
dependency tracking and shutdown protection using tools provided in s6
alone.

As for an overview, the why, overview, and quickstarts in s6-linux-init
should cover it. But for those who prefer it in email format, what you
get is: the supervisor as pid 1, hpr (halt/poweroff/reboot) interfaces
that understand how to work with the service model, direct IPC using
common s6 interfaces instead of the ghastly interfaces like stopit,
ctrlaltdel, reboot, nosync, a safety net of having a running supervisor
(and as such a running getty) during shutdown, and so on. runit is
impressively technically simple but that technical simplicity begats
workarounds and a lack of safety when it comes to the lifecycle of a
full system and most of the added design complexity around s6-l-i is
mostly there to make sure it can handle shutdown in a clean way.

FWIW, the original design of s6 followed much closer to the runit
design, though with stage1 execing into s6-svscan (instead of forking
it), stage2 only handling post boot initialization tasks (instead of
those tasks and then the supervisor), and stage3 also being pid1 (via
s6-svscan's finish script).

(speaking of walls of text).
> 
> P.S: I stumbled over this execline oddity:
> | dollarat     -0 -d a                      # separates by \0
> | forbacktickx -0 -d a var {gen...} loop... # splits on a
> IMHO, both should be an error, but at least treat them the same.
>
As per the docs for forbacktickx:
-0 : accept null characters from gen's output, using them as delimiters.
If this option and a -d option are used simultaneously, the rightmost
one wins.

Cheers!
-- 
Colin Booth

Re: s6 usability

[Laurent Bercot]

[-- Attachment #1: Type: text/plain, Size: 6970 bytes --]


>If you're referring to
>https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=906250#37
>then, well, you are fighting against POSIX. There's little choice for
>Debian in the matter. Taking a hardline stance on such "legal" issues is
>part of their identity as a distro.

No, you're falling for the pretext - because this is only the current
pretext they're using. Debian *does not provide* a POSIX-compliant cd
binary, so their claims at POSIX compliance are just lies.
The current version of execline includes POSIX-compliant cd and wait
binaries, and the next one will also include a POSIX-compliant umask
binary. Then we shall see what new excuse they find not to put execline
binaries in the default PATH.


>I don't think that's the real issue.

Unfortunately, it is. Debian's claim at POSIX compliance is pure
hypocrisy, only used when it furthers other goals of theirs and
promptly forgotten when it's at odds with them.

Adélie Linux *actually* aims for full POSIX compliance and passes more
VSX-PCTS2003 tests than any other Linux distribution. It includes all
execline binaries in /bin. No VSX tests are failed because of that.


>It's not difficult launching the browser, it's difficult getting to the
>correct webpage. Compare
>| $ elinks /usr/share/doc/s6/s6-log.html

https://lmgtfy.com/?q=s6-log&iie=1

Of course, you can argue that not every machine is connected to the
Internet, which is true. But the overwhelming majority of times when
I've had to tinker with an unconnected machine, and had no access to
the Web whatsoever, it was for bootstrapping, or for tinkering with an
embedded target, and the unconnected machine did not have a functional
manpages database either.

A manpages indexer is actually more difficult to bootstrap than a
network connection + a text-based Web browser!

What I would say is that if the obstacle to HTML documentation is
that there are no shortcuts to locally installed HTML docs, whereas man
comes with an index for the local manpages database, then what we need
to do as a community is a local HTML doc indexer. Then you would be able
to type
$ doc s6-log
and you'd have your favorite browser auto-launched on the local 
s6-log.html
page.
*That* would be software with some real added value, and maybe it would
start the much-needed transition away from antiquated nroff pages.

... and since nobody else is ever going to write it, maybe I should. :(


>Probably yes, but if you are doing that, then why don't you look at
>argv[0] and provide the runit interface proper? :D
>(Or provide runsv/sv/chpst/.. as individual binaries, since you prefer
>those.)

One does not preclude the other. :P
I am currently experimenting with such a thing. The *exact* runit
interfaces are difficult to provide, because runit's conception is
slightly different from s6's and emulating precise runit behaviour
requires significant effort that I feel is not a good time investment.
(For instance, runsv and s6-supervise are not exactly similar, because
runsv also handles a logger whereas s6-supervise only handles one
service. svlogd reads a config file in a logdir, s6-log does not.)

I'd rather provide close interfaces that work in the common cases
but fail in the edge cases with an explanation why and a suggestion on
how to use the related s6 command. The goal isn't to mimic runit, the
goal is to help runit users transition to s6.


>But outside of supervision, I notice that you are reimplementing a
>lot of small programs.

Those are mostly legacy, from a time when I wanted to bootstrap a
machine with no GNU code (and no busybox either). They're out of scope
for s6 or the related utilities; prefixing them with s6- was probably
my worst naming mistake.

I realize, from this conversation and others, that I need to write an
additional documentation page, to be read even before the s6 overview,
that explains what piece of the s6 ecosystem does what. That page
would, among other things, mention s6-*-utils and explain how relevant
they are (i.e. not much).


>Needing to *understand* execline wasn't my misconception, nor worry. But
>when I'm told that a runit-lookalike depends on bringing its own
>scripting language along, then that sounds more like systemd and less
>like djb to me. :(

  That is only until you realize that s6 is a collection of utilities
doing a lot of stuff, and that the scripting language is used by a
few binaries in the collection, and not by the core supervision 
programs.

  Moreover, there would be a relatively easy way not to do that: anywhere
execline is used, use a shell instead. And spawn shell scripts instead
of simple command lines. And nobody would bat an eye.
  But the reason why execline exists is to make it lighter, easier and
more secure to spawn command lines. It is to be *better* than the shell
in the s6 use cases. It's not to add Turing completeness to a parser in
pid 1 or whatever nonsense you can come up with. So it's very unfair
criticism to say "it brings its own scripting language along": I 
understand
you may have been burned by crappy software before, but s6 is not like
that. It brings its own scripting language along *in order to be 
simpler,
less buggy, more restricted and less resource-consuming* than the 
scripting
language that already exists on every Unix machine and that everyone 
uses.


>When you explain it like that, it sounds far more reasonable.

The new documentation page I'm thinking about should definitely explain
that, to nip any misconsceptions in the bud.


>I'd appreciate this regardless of the internal dependency. I know you
>want to popularize execline, but "you must use it if you want to use my
>other tools" is not a helpful form of advocacy.

That is really not the goal. I don't want to popularize execline because
it's execline. I want it to be used in the exact places where it's
beneficial to use it over a shell. And a s6-log processor definition is
precisely such a place. I care about software design first and 
popularity
second; if it weren't the case, I'd be writing commands that are easy
to type and that autoplay cute videos.
$ cute cat
$ cute puppy
$ cute blobfish

I guarantee you such a program would be more popular than s6.


>Yes. But since you are mentioning it, that was another of my "s6 seems
>more complex" issues. runit goes from "start the supervisor manually" to
>"be pid 1" with very little effort. See runit(8).
>Or https://www.unix.com/man-page/debian/8/runit/ I guess. ;-P
>
>s6-linux-init and s6-rc seem extremely complicated in comparison. And
>it's not clear to me how much of that complexity is optional, and what
>it might buy me. Maybe a better high-level overview document might help
>there.

Yes. The document I'm thinking about would explain that.
Thanks again for your UX returns. They are definitely useful.

--
Laurent

Re: s6 usability

[Jan Braun]

[-- Attachment #1: Type: text/plain, Size: 3662 bytes --]

Colin Booth schrob:
> > If you're referring to
> > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=906250#37
> > then, well, you are fighting against POSIX. There's little choice for
> > Debian in the matter. Taking a hardline stance on such "legal" issues is
> > part of their identity as a distro.
> >
> It doesn't help that neither Adam nor Jakub read the documentation for
> the execline equivalents for cd, umask, or wait.

Why would you say that? They effectively only claim that execline's
cd/umask/wait binaries don't conform to the POSIX specification for
cd/umask/wait. And I think that's uncontroversally true.

> That or they don't know what 'execs into' means.

POSIX requires:
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap01.html#tag_17_06
| However, all of the standard utilities, including the regular built-ins
| in the table, [...]  shall be implemented in a manner so that they can be
| accessed via the exec family of functions as defined in the System
| Interfaces volume of POSIX.1-2008 and can be invoked directly by those
| standard utilities that require it (env, find, nice, nohup, time,
| xargs).

i.e, if you call
    execvp("cd", "cd", /* any other args, */ NULL);
, POSIX says you MUST get the behaviour documented at
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/cd.html
. And if /bin/cd is execline's cd, then you don't.

There's also the imho sensible rationale of e.g.
| find . -type d -exec cd {} \; -exec foo {} \;
| (which invokes "foo" on accessible directories)
for that requirement, even if these are admittedly rare cases.

See Message-ID: <a8fbd02e-0265-3d59-89d1-81048665693c@NTLWorld.COM>
here on the list, from Jonathan de Boyne Pollard for more details.


> Within the context of a shell the builtin will always* take precedence.

True, but not the controversial issue.



> [placing binaries]
> Have you ever considered slashpackage ;)
> 
> In all seriousness though this, with the exception of dropping the s6-
> prefix (and the prefix-appender binary I guess), is what slashpackage
> does. /bin stays uncluttered, commands end up in a PATH-able place, and
> if you want to surprise any systemic shell scripts you have you can
> symlink in replacements to the default PATH. 

Yes, I'm aware of that. Unfortunately, I'm not aware of a unix distro
usable for my general needs implementing /package as their packaging
scheme.
Nix/NixOS does something similar, and is on the short list of distros
I'll consider if Debian goes ahead with the systemd madness.

And FWIW, if I were to create my own distro/OS, I'd do away with $PATH
entirely and have people union-mount stuff into /bin .



> > P.S: I stumbled over this execline oddity:
> > | dollarat     -0 -d a                      # separates by \0
> > | forbacktickx -0 -d a var {gen...} loop... # splits on a
> > IMHO, both should be an error, but at least treat them the same.
> >
> As per the docs for forbacktickx:
> -0 : accept null characters from gen's output, using them as delimiters.
> If this option and a -d option are used simultaneously, the rightmost
> one wins.

Yes, and as per the docs for dollarat:
-0 : use the null character as separator. Any -d argument will be
ignored.

They're both working as advertised. But they have *different* rules for
resolving the case where both -0 and -d are given.
I think that's a lack of UI consistency, and would consider it a bug in
my software.
(And, as I said, I think the best response to getting both -0 and -d
would be erroring out, but that's just an aside.)


cheers,
    Jan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: s6 usability

[Jan Braun]

[-- Attachment #1: Type: text/plain, Size: 8286 bytes --]

Laurent Bercot schrob:
> No, you're falling for the pretext - because this is only the current
> pretext they're using. Debian *does not provide* a POSIX-compliant cd
> binary, so their claims at POSIX compliance are just lies.

Including execline's (non-compliant) cd would preclude them from
becoming more POSIX-compliant if somebody actually bothers to provide a
POSIX-compliant cd in the future.

> The current version of execline includes POSIX-compliant cd and wait
> binaries, and the next one will also include a POSIX-compliant umask
> binary. Then we shall see what new excuse they find not to put execline
> binaries in the default PATH.

Oh, hey, that somebody is you. Awesome! So shipping that execline will
actually increase POSIX compliance in Debian. :)

So let's wait and see what happens.

I haven't had any reason to suspect any Debian Developers are/were
acting insincerely in dealing with Debian issues. Bad at communication,
annoyingly stubborn, overly pedantic, plain misguided: yes, but afaict
always honest¹.

I'm sorry you feel differently. :(


> Adélie Linux *actually* aims for full POSIX compliance and passes more
> VSX-PCTS2003 tests than any other Linux distribution. It includes all
> execline binaries in /bin. No VSX tests are failed because of that.

I guess that just means that
    find . -type d -exec cd {} \; -exec foo {} \;
(or similar for umask) is not part of the tests. ;)



> https://lmgtfy.com/?q=s6-log&iie=1

Ewww, no, just no.
That's far too slow, and not deterministic.

> $ doc s6-log
> and you'd have your favorite browser auto-launched on the local s6-log.html
> page.

Yes. The lack of that is exactly my issue with the s6 docs.

> *That* would be software with some real added value, and maybe it would
> start the much-needed transition away from antiquated nroff pages.
> 
> ... and since nobody else is ever going to write it, maybe I should. :(

Maybe. I don't see /much/ benefit in having hyperlinks and colors for
manpage-like docs.

However, I very much see the danger that people will start writing crap
instead of manpages-in-html once you tell them that "this is going to be
rendered by a browser".

One reason why GNU info docs are virtually unusable to me is because
they're a twisty little maze of tiny nodes hyperlinking each other, and
I found it impossible to concisely read stuff once from beginning to end
like I'd do with a new manpage.

And that's just hyperlinks. Some people will undoubtedly start doing
unprofessional things with images and Javascript and loading stuff from
the network, and then usability is completely out the window.

Man pages are not about nroff, but about having a standard template on
how to document things. But nroff serves as a filter that prevents
people from throwing random things into the man database. ;)


I think writing manpage-style docs in a simple markup language and then
compiling to nroff/html/pdf/... is the better aproach.
And then your "doc" command is just a slight variation on
"$BROWSER /usr/share/doc/html/$1", if you want it.



> I am currently experimenting with such a thing. The *exact* runit
> interfaces are difficult to provide, [...]
> The goal isn't to mimic runit, the
> goal is to help runit users transition to s6.

Well, to me, the litmus test would be whether my
 runsvdir -P /etc/service  '................'
and various ./run scripts that use chpst and sv up/down/start/stop
result in a working supervision setup with the s6 versions of
runsvdir/chpst/sv.

The lack of config file for the logger is annoying. But since runit uses
it's own matching language, and s6-log uses posix extended regexps
(thanks!), the config file wouldn't be compatible anyway.

I shall be looking forward to the results of your experiments. :)



> I realize, from this conversation and others, that I need to write an
> additional documentation page, to be read even before the s6 overview,
> that explains what piece of the s6 ecosystem does what.

Yes, please. It's been confusing to have s6 (the supervision suite)
containing a bunch of s6-prefixed binaries, and then having a bunch of
other loosely related things also with s6- prefixes. Contributes greatly
to my "too many binaries to keep in memory" issue.



> > Needing to *understand* execline wasn't my misconception, nor worry. But
> > when I'm told that a runit-lookalike depends on bringing its own
> > scripting language along, then that sounds more like systemd and less
> > like djb to me. :(
> 
>  That is only until you realize [...] I understand you may have been
>  burned by crappy software before, but s6 is not like that.

Yes. I think and hope that I was careful in wording anything about "s6
complexity" as *impressions* that I got by reading the docs &co, not as
things I believe actually apply, if you can spot the difference.

When I look at runit, I see a system that's obviously simple (yay).
When I look at s6, there are so many parts that I *cannot tell* whether
each part is reasonably simple and independent of most² others (good),
or whether it's one big interdependent mess (bad).

As I'm learning more about s6, and by seeing your attitude about
relevant issues, I'm getting more confident that it's the former. But
technically I still can't be sure yet, and I'm trying to point out the
uphill battle s6 has to fight on the PR front in my head. ;) Of course,
better docs may help, see above.



> It brings its own scripting language along *in order to be simpler,
> less buggy, more restricted and less resource-consuming* than the scripting
> language that already exists on every Unix machine and that everyone uses.

Yes, I get that.

> [quote rearranged]
> So it's very unfair
> criticism to say "it brings its own scripting language along"

Except that it, in fact, actually does so. :P

For well-meaning reasons, and you may be right about the security
benefits over shell.
But I am *extremely* unlikely to bother learning execline.

Hence, in my case, there's no security benefits, and, on my computer,
execline is just unneeded dead code lying around. Not nearly XML in pid
1, but still a step away from true minimalism.

Actually, it's not lying around at the moment. But that means I can't
use !processor directives for s6-log. Which I would need if I were to
actually use s6. And they'd all be "!/etc/jan/logmailr/mailer". So
there's no need for execline, or a shell, to get invoked. Just plain
execvp() would do.³

Can you see why that bothers the perfectionist in me?


> Thanks again for your UX returns. They are definitely useful.

Glad to hear that. And sorry if I offended with my systemd comparison or
generally come across too strong. I'm genuinely trying to give
constructive feedback. But I do have a habit of erring on the side of
too blunt phrasing.

cheers,
    Jan



¹)
Well, there is/was one particular systemd advocate stretching the bounds
of my ability to believe that they're actually *that* narrow-minded and
unable to see the other side. Still, I'd prefer to give the benefit of
the doubt.

²)
That's another thing where I prefer multibinaries: s6-fdholder-*
Having
    fdholderd
    fdholder list
    fdholder retrieve
    fdholder store
    ...
would be far more Jan-compatible UI.
And having both s6-fdholder-daemon and s6-fdholderd in $PATH also serves
to confuse me. If one is an implementation detail of the other, I'd
prefer it be hidden away into /lib or something. Then I don't have to
start by figuring out which one I should use. Or, worse, only seeing the
wrong one and trying to make that work.

³)
Note I'm not saying you should implement "?processor" that way.
There's some pros+cons to each of
 * Stuffing "processor" unchanged into the 1st (and 2nd) argument of exec[vl]p
 * splitting on whitespace and parcelling into exec[vl]p
 * passing the string on to system()
Pick what you think best. I'm unlikely to notice the difference, and if
I do, it's trivial to adapt. Since there seems to be no standard about
this kind of thing, I made a habit of assuming that anything shell
metacharacter related causes breakage.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: s6 usability

[Colin Booth]

On Sun, Dec 22, 2019 at 02:05:14AM +0100, Jan Braun wrote:
> > It doesn't help that neither Adam nor Jakub read the documentation for
> > the execline equivalents for cd, umask, or wait.
> 
> Why would you say that? They effectively only claim that execline's
> cd/umask/wait binaries don't conform to the POSIX specification for
> cd/umask/wait. And I think that's uncontroversally true.
> 
> > That or they don't know what 'execs into' means.
> 
> POSIX requires:
> https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap01.html#tag_17_06
> | However, all of the standard utilities, including the regular built-ins
> | in the table, [...]  shall be implemented in a manner so that they can be
> | accessed via the exec family of functions as defined in the System
> | Interfaces volume of POSIX.1-2008 and can be invoked directly by those
> | standard utilities that require it (env, find, nice, nohup, time,
> | xargs).
> 
> i.e, if you call
>     execvp("cd", "cd", /* any other args, */ NULL);
> , POSIX says you MUST get the behaviour documented at
> https://pubs.opengroup.org/onlinepubs/9699919799/utilities/cd.html
> . And if /bin/cd is execline's cd, then you don't.
>
Fair enough. Pedantic as it was I had mostly taken issue with the
statement that:
  Regular "cd" changes the directory for the calling process (thus it
  even can't possibly be a separate process), this "cd" takes two
  arguments and runs a _child_ with the changed directory.
Which is a false statement unless you consider a program execve'd into
another to be "running a child".
> 
> There's also the imho sensible rationale of e.g.
> | find . -type d -exec cd {} \; -exec foo {} \;
> | (which invokes "foo" on accessible directories)
> for that requirement, even if these are admittedly rare cases.
>
Not sure if this is a legitimate counter but:
find . -type d -exec cd "{}" pwd \;
does what is expected with execline's binary. In fact, I'm not sure if
your hypothetical is valid since I believe each exec is given its own
execution context and cd is not allowed to modify the callers
environment. Of course, I don't think there are any true stand-alone cd
utilities that we can test this with. If you know of one please tell me.

Similarly:
$ nohup /command/cd / pwd ; cat nohup.out
nohup: ignoring input and appending output to 'nohup.out'
/

On third thought, you're specifically talking about the case where you
fail to give a follow-on program to cd. As of v2.5.3.0, the configure
script in execline has grown the --enable-pedantic-posix option which,
when given, will generate a cd that knows how to handle the NULL case.
> 
> > [placing binaries]
> > Have you ever considered slashpackage ;)
> > 
> > In all seriousness though this, with the exception of dropping the s6-
> > prefix (and the prefix-appender binary I guess), is what slashpackage
> > does. /bin stays uncluttered, commands end up in a PATH-able place, and
> > if you want to surprise any systemic shell scripts you have you can
> > symlink in replacements to the default PATH. 
> 
> Yes, I'm aware of that. Unfortunately, I'm not aware of a unix distro
> usable for my general needs implementing /package as their packaging
> scheme.
> Nix/NixOS does something similar, and is on the short list of distros
> I'll consider if Debian goes ahead with the systemd madness.
> 
> And FWIW, if I were to create my own distro/OS, I'd do away with $PATH
> entirely and have people union-mount stuff into /bin .
> 
And yet NixOS is also primarily a systemd based distro, unless we have
different definitions of "systemd madness."
> 
> 
> > > P.S: I stumbled over this execline oddity:
> > > | dollarat     -0 -d a                      # separates by \0
> > > | forbacktickx -0 -d a var {gen...} loop... # splits on a
> > > IMHO, both should be an error, but at least treat them the same.
> > >
> > As per the docs for forbacktickx:
> > -0 : accept null characters from gen's output, using them as delimiters.
> > If this option and a -d option are used simultaneously, the rightmost
> > one wins.
> 
> Yes, and as per the docs for dollarat:
> -0 : use the null character as separator. Any -d argument will be
> ignored.
> 
> They're both working as advertised. But they have *different* rules for
> resolving the case where both -0 and -d are given.
> I think that's a lack of UI consistency, and would consider it a bug in
> my software.
> (And, as I said, I think the best response to getting both -0 and -d
> would be erroring out, but that's just an aside.)
> 
Fair enough. For some reason I was thinking you were talking about a
documentation error, not a functional inconsistency. I was assuming that
this leaked in when forbacktickx became a wrapper but that isn't the
case, the inconsistency exists as far back as the 2.0.0.0 rewrite.

-- 
Colin Booth

Re: s6 usability

[Steve Litt]

On Sat, 21 Dec 2019 23:46:34 +0000
"Laurent Bercot" <ska-supervision@skarnet.org> wrote:

> No, you're falling for the pretext - because this is only the current
> pretext they're using. Debian *does not provide* a POSIX-compliant cd
> binary, so their claims at POSIX compliance are just lies.
> The current version of execline includes POSIX-compliant cd and wait
> binaries, and the next one will also include a POSIX-compliant umask
> binary. Then we shall see what new excuse they find not to put
> execline binaries in the default PATH.

My finding in the past is that Debian will do harm to any software just
so they can comply with their arcane rules. One such rule caused them
to change the prepend key-sequence in VimOutliner from the incredibly
quick and easy double comma (,,) to the wrist-twisting, error prone,
slow, and keyboard dependent double backslash (\\), even though
VimOutliner's manifesto listed authoring speed as the top priority.

That being said, is having your stuff on the executable path such an
advantage? A lot of times I put all the executables for a system in a
directory off the path, and then create a shellscript like this:

====================================
#!/bin/sh
export PATH=/path/to/s6:$PATH
$@
====================================

Or this:

====================================
#!/bin/execlineb -s1
importas OLDPATH PATH
export PATH "/path/to/s6:$OLDPATH"
$1 $@
====================================

Symlink the shellscript from a directory on the path, and whatever is
in $@ is as good as being on the path. By doing this I avoid any
program name conflicts, I can easily delete my whole program or copy it
to another computer. And of course slashpackage is another way.

I'm not saying this is perfect.  I'm just saying it could be as simple
a opening a tarball in an arbitrary directory and typing make; make
install. That's basically how we distributed VimOutliner for a long
time. Earlier in this thread somebody suggested a method of Debian
packaging that was both breathtaking in its convolution and
preventing of other software from being installed at the same time. With
Debian's propensity to break things doing them their way, perhaps it's
time to consider the possibility of doing an end run around the Debian
Developers.

SteveT

Steve Litt 
December 2019 featured book: Rapid Learning for the 21st Century
http://www.troubleshooters.com/rl21

Re: s6 usability

[Laurent Bercot]

>That being said, is having your stuff on the executable path such an
>advantage?

I don't know, why does Unix like to have its binaries in /bin? Why does
PATH exist? What is the nature of an executable? You have two hours.


>December 2019 featured book: Rapid Learning for the 21st Century

  Ah, so that's your secret. Rapid learning and rapid forgetting.

--
  Laurent

Re: s6 usability

[Dewayne Geraghty]

On the question of PATH for BSD land (FreeBSD, TrueOS, HardenedBSD et 
al), binaries installed from packages (ports) live under /usr/local, 
with the exception of /var and /tmp.  The wars were fought and 
/usr/local can easily be mounted read-only.

Of the 1446 packages I have installed (no desktop stuff),
the breakdown is

# ls /usr/local/bin/ | wc -l
     2857
# ls /usr/local/sbin/ | wc -l
      349
# find /usr/local/bin/ -type d -depth 1
/usr/local/bin/db5; # No directories under /usr/local/sbin
# ls /usr/local/libexec|wc -l
       72

The placement of files, is more a "distribution" decision that I'm sure 
is already settled.

On the documentation front, Laurent's work is very good, but I did find 
the examples from the gentoo docset for s6 & s6-rc, a life saver for 
someone with no prior process or service management background.  (I'd 
only used monit previously, and still do to reload application configs 
and some other system state change events over s6 tools).

Re: s6 usability

[Oliver Schad]

[-- Attachment #1: Type: text/plain, Size: 1052 bytes --]

On Sun, 22 Dec 2019 23:20:03 +0000
"Laurent Bercot" <ska-supervision@skarnet.org> wrote:

> I don't know, why does Unix like to have its binaries in /bin? Why
> does PATH exist? What is the nature of an executable? You have two
> hours.

Sorry, this discussion is crazy: Laurent provides a basic booting tools
with his toolchain. A booting tools should be in /bin - full stop!

However - what is the concrete suggestion from the Debian guys what
Laurent should do?

On top, it makes sense to start a community integration program to
fullfill distro requirements. So that would include public test and
build infrastructure and pre-packaging stuff.

I can support that with free hosting for building and testing, i.e. I
can provide a public gitlab with build and test pipelines.

Best Regards
Oli

-- 
Automatic-Server AG •••••
Oliver Schad
Geschäftsführer
Turnerstrasse 2
9000 St. Gallen | Schweiz

www.automatic-server.com | oliver.schad@automatic-server.com
Tel: +41 71 511 31 11 | Mobile: +41 76 330 03 47

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: s6 usability

[Steve Litt]

On Sun, 22 Dec 2019 23:20:03 +0000
"Laurent Bercot" <ska-supervision@skarnet.org> wrote:

> >That being said, is having your stuff on the executable path such an
> >advantage?  
> 
> I don't know, why does Unix like to have its binaries in /bin? Why
> does PATH exist? What is the nature of an executable? You have two
> hours.
> 
> 
> >December 2019 featured book: Rapid Learning for the 21st Century  
> 
>   Ah, so that's your secret. Rapid learning and rapid forgetting.

You can sling all the insults you want, but the fact is, putting all
your stuff for a single software group into one tree, outside of the
path, and accessing it with a prepath script that perhaps does a cd and
sets other environment variables that you'll need makes a tidy little
package, with no name conflicts. You can "uninstall" it with a simple
tree delete.

Depending on dependencies and architecture and the way your software's
written, you might be able to copy it to another computer with cp -Rp
or rsync -va. It's very easy to examine.

The UNIX way, with /usr/bin and /usr/local/bin and the rest, is just
fine with me, as long as the distro cooperates. But if the distro fails
to carry out the intent of  your software, there are worse things than
putting it all in one directory or tree, prepathing their location, and
running it that way.

SteveT

Steve Litt 
December 2019 featured book: Rapid Learning for the 21st Century
http://www.troubleshooters.com/rl21

Re: s6 usability

[Laurent Bercot]

>You can sling all the insults you want, but the fact is

... the fact is, I have explained this multiple times on this very list,
I am tired of seeing the same fallacy come up again and again, and I am
tired of repeating myself.


>fine with me, as long as the distro cooperates. But if the distro fails
>to carry out the intent of  your software, there are worse things than
>putting it all in one directory or tree, prepathing their location, and
>running it that way.

1. There aren't any worse things than software breaking. Do I need to
explain again why software breaks when you do that?

2. Why aren't you suggesting the same thing to the ImageMagick authors?
or to the util-linux authors? Those are also dumping a lot of stuff into
/bin or /usr/bin, including a program abusively named "import" - for
which I was willing to concede, in the name of historical priority.
It's a real question. Why should I accept subpar treatment that nobody
else would consider?

--
Laurent

Re: s6 usability

[Laurent Bercot]

[-- Attachment #1: Type: text/plain, Size: 629 bytes --]

>However - what is the concrete suggestion from the Debian guys what
>Laurent should do?

  The Debian people are happy with putting execline binaries in a 
separate
directory that is not in the default PATH. They don't see a problem with
that. And the thing is, nobody will notice a problem either - except
people who try calling useful execline binaries (such as redirfd) 
outside
of execline scripts, *which is a use case I officially want to support*.

  I'll try releasing the execline version with a POSIX-compliant umask
before the end of the year. Then we'll see where Debian really stands.

--
  Laurent

Re: s6 usability

[Jonathan de Boyne Pollard]

Oliver Schad:

> A booting tools should be in /bin - full stop!
>

That is historically untrue.  The real world has not actually worked in 
the way that some people think.

* https://unix.stackexchange.com/a/448799/5132

Re: s6 usability

[Oliver Schad]

[-- Attachment #1: Type: text/plain, Size: 1443 bytes --]

On Mon, 23 Dec 2019 10:15:11 +0000
Jonathan de Boyne Pollard <J.deBoynePollard-newsgroups@NTLWorld.COM>
wrote:

> Oliver Schad:
> 
> > A booting tools should be in /bin - full stop!
> >  
> 
> That is historically untrue.  The real world has not actually worked
> in the way that some people think.

Sorry, this is a historical explanation of some things - is this a
history course here?

That is *not* what you would expect today. Scripts
for system initialization (a special case) should find all tools
in /bin, because it's in always in the path (always today).

To put tools for a lot of scripts somewhere(!) would mean, that all
scripts have a method to find these tools or would mean to add more
directories to PATH.

Because the maintainer of these scripts are completely distributed
(it's not one small team, which writes boot/start scripts), it's crazy
to do something else than put everything in /bin.

If you would say, that this is not true, then we can discuss where we
should place cp, mv, ls - maybe in /usr/lib/gnu/? Really?

Important scripting tools should always be in /bin or /usr/bin - /bin
for boot, /usr/bin for later stages is ok.

Best Regards
Oli

-- 
Automatic-Server AG •••••
Oliver Schad
Geschäftsführer
Turnerstrasse 2
9000 St. Gallen | Schweiz

www.automatic-server.com | oliver.schad@automatic-server.com
Tel: +41 71 511 31 11 | Mobile: +41 76 330 03 47

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]