From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.comp.sysutils.supervision.general/2251 Path: news.gmane.org!.POSTED!not-for-mail From: Joe Nosay Newsgroups: gmane.linux.debian.user,gmane.comp.sysutils.supervision.general,gmane.os.freebsd.devel.hackers Subject: Re: nosh version 1.28 Date: Tue, 30 Aug 2016 11:58:27 -0400 Message-ID: References: <54430B41.3010301@NTLWorld.com> <54B86FD5.3090203@NTLWorld.com> <554E53EF.4080600@NTLWorld.com> <554E93AF.3070709@NTLWorld.com> <556BA130.50708@NTLWorld.com> <55902328.8080602@NTLWorld.com> <55D5CFA2.5010402@NTLWorld.com> <55D8B9AC.6010209@NTLWorld.com> <56089268.6080007@NTLWorld.com> <56120D11.4080506@NTLWorld.com> <5636C75B.70000@NTLWorld.com> <5672BD8C.50303@NTLWorld.com> <569617F3.8000101@NTLWorld.com> <56AEAED5.4010606@NTLWorld.com> <572D266D.6080807@NTLWorld.com> <7fa206eb-9b34-de45-4474-11c04007326b@NTLWorld.com> <12f6f577-ecae-07cf-b066-c418071c7034@NTLWorld.com> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary=94eb2c14bc96304776053b4c0daf X-Trace: blaine.gmane.org 1472572760 21849 195.159.176.226 (30 Aug 2016 15:59:20 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Tue, 30 Aug 2016 15:59:20 +0000 (UTC) Cc: Julian Elischer , Debian users , Supervision , FreeBSD Hackers To: Jonathan de Boyne Pollard Original-X-From: bounce-debian-user=gldu-debian-user-2=m.gmane.org@lists.debian.org Tue Aug 30 17:59:11 2016 Return-path: Envelope-to: gldu-debian-user-2@m.gmane.org Original-Received: from bendel.debian.org ([82.195.75.100]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1belRI-0004MG-0g for gldu-debian-user-2@m.gmane.org; Tue, 30 Aug 2016 17:59:04 +0200 Original-Received: from localhost (localhost [127.0.0.1]) by bendel.debian.org (Postfix) with QMQP id 1833818C; Tue, 30 Aug 2016 15:58:59 +0000 (UTC) X-Mailbox-Line: From debian-user-request@lists.debian.org Tue Aug 30 15:58:59 2016 Old-Return-Path: Original-Received: from localhost (localhost [127.0.0.1]) by bendel.debian.org (Postfix) with ESMTP id 6A6B27A for ; Tue, 30 Aug 2016 15:58:49 +0000 (UTC) X-Virus-Scanned: at lists.debian.org with policy bank en-ht X-Amavis-Spam-Status: No, score=-4.999 tagged_above=-10000 required=5.3 tests=[BAYES_00=-2, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FOURLA=0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=2, LDO_WHITELIST=-5, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=no autolearn_force=no Original-Received: from bendel.debian.org ([127.0.0.1]) by localhost (lists.debian.org [127.0.0.1]) (amavisd-new, port 2525) with ESMTP id LhNqogMds1bh for ; Tue, 30 Aug 2016 15:58:43 +0000 (UTC) X-policyd-weight: using cached result; rate: -7 Original-Received: from mail-ua0-x230.google.com (mail-ua0-x230.google.com [IPv6:2607:f8b0:400c:c08::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (not verified)) by bendel.debian.org (Postfix) with ESMTPS id 6E4F7136 for ; Tue, 30 Aug 2016 15:58:31 +0000 (UTC) Original-Received: by mail-ua0-x230.google.com with SMTP id l94so40555332ual.0 for ; Tue, 30 Aug 2016 08:58:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=SQ7JGtWsJv2nxF67WN/1QGxycIsVnUPowbw+vsVNJNQ=; b=RqRCLO8TYeCMmjhUyjG9AKq9wjGy3vs9npbLwj3qHyI4u0/g5iYFXBDAtksdA9dUOp UUmsRLSTghBQnFgbKnbkyu0I3sPdYff1fjGSLnTlcrIuhm+S4ba+wVELYKwgnqE2eEGx kZ/pFzKPWYIDxJbuVU39DtUjBMHO4SH7LyEJ82X/Hrq4hkC5HK5bs54zGtjUOPG7hDqC 6Iya5moIUFC5jzfrmfYF1ZV98hSyq04PwY5GrqiG/NfLNfdz/RRt9N2mOmORzFzVcDeC nVPDt4mvq3PnQ8++luHlLO+sw2Pr0NbYwxjt+O6T45xPYulnfqdkc1dNI6TzFAsrlnWJ U5LA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=SQ7JGtWsJv2nxF67WN/1QGxycIsVnUPowbw+vsVNJNQ=; b=ZxDjWapHEW0d0vz9I5C3SnNjkQoFBNp7HFMdrE9U5feMDW5i2RHebZeYK3fSwc275G 9iYbdVjK4DVf2df3mdRXSrpNnKmEMqpsxN1ypx+x+nwLyYw2AkLCVQpGFqvf5U3B0vri ra7045zO2n1fZDmRaT+uq5fjRGZqPTBubnWtWnZa/tSJn6RY45gpzcn41Y2DFjfHSo0R p2U9JXkMQT4suLmetmG2ToImApvJ3jmOx8c8qBaRXNTjUNvLsJOHvr3vKQyQBLwkEphS VE7ZTu4d+H+Qo4xkSSe5FtlvNL322kcWMMd67l79/jSkcMfS1PcLL+ovtgkAwRkK0Hyq QWPg== X-Gm-Message-State: AE9vXwNcvjYenJxZTk3s7DWTh7lGSnHzBhqlMp4YeU/aOGA+HtV0nWKJMICrIbKLrKUm7eU378NcU42c2InrwQ== X-Received: by 10.31.248.130 with SMTP id w124mr2583322vkh.90.1472572708372; Tue, 30 Aug 2016 08:58:28 -0700 (PDT) Original-Received: by 10.103.137.67 with HTTP; Tue, 30 Aug 2016 08:58:27 -0700 (PDT) In-Reply-To: <12f6f577-ecae-07cf-b066-c418071c7034@NTLWorld.com> X-Rc-Virus: 2007-09-13_01 X-Rc-Spam: 2008-11-04_01 Resent-Message-ID: Resent-From: debian-user@lists.debian.org X-Mailing-List: archive/latest/711085 X-Loop: debian-user@lists.debian.org List-Id: List-URL: List-Post: List-Help: List-Subscribe: List-Unsubscribe: Precedence: list Resent-Sender: debian-user-request@lists.debian.org List-Archive: https://lists.debian.org/msgid-search/CA+WntOvsSjBwgqNbZ+SfCgnfq2OPD8C46eA=hk6oitzutzRubw@mail.gmail.com Resent-Date: Tue, 30 Aug 2016 15:58:59 +0000 (UTC) Xref: news.gmane.org gmane.linux.debian.user:516091 gmane.comp.sysutils.supervision.general:2251 gmane.os.freebsd.devel.hackers:57970 Archived-At: --94eb2c14bc96304776053b4c0daf Content-Type: text/plain; charset=UTF-8 Thank you very much. And have a blessed day. On Sun, Aug 21, 2016 at 7:04 AM, Jonathan de Boyne Pollard < J.deBoynePollard-newsgroups@ntlworld.com> wrote: > I don't know why you asked about FreeBSD rc.d just on the Debian mailing > list; but I'm going to deal in both of those and others besides, here, and > things that apply across both, so I've re-included the FreeBSD mailing > list. (-: > > 2016-08-14 15:10, Julian Elischer: > > I don't know if I just missed it, or it isn't there but I have a >> question.. >> >> You give examples of importing systemd service files. What about >> importing rc.d files with all their ability to run arbitrary shell commands. >> >> And once you have the services defined, what is the logical equivalent of >> rc.conf, which can supply parameters for each service and turn them on and >> off? can you import from rc.conf? >> >> You did miss it. (-: > > What you missed has grown to be a significant subsystem. It was actually > mentioned a couple of times in the 1.28 announcement. It's the external > configuration import subsystem. You can read about it in the nosh Guide: > > xdg-open /usr/local/share/doc/nosh/external-formats.html > > As you can see, there's a whole section on importing from rc.conf into > native service management mechanisms. ("rc.conf" covers several sources, > note, including a FreeNAS configuration database and /etc/defaults/rc.conf > .) > > The native service mangement mechanisms are the "enable" and "disable" > subcommands to the system-control command, and using the envdir command in > the normal daemontools-family style way. The enable/disable mechanism in > "rc.conf" is treated as if it were a preset (in systemd nomenclature). You > tell service management to "preset" a service, and it will look at > /etc/rc.conf and /etc/rc.conf.local (as well as some other preset > mechanisms) to determine what to set the native enable/disable state to. > The user manual page for the preset subcommand (of system-control) explains > what the preset mechanisms are in detail. > > You can set up environment directories how and where you like, but there's > a convention that is shared by the "convert-systemd-units" tool, the > "rcctl" shim, and the external configuration import subsystem as a whole. > This convention is an environment directory named "env" that is in the > service directory. The "rcctl" shim gets and sets variables there; and the > import subsystem places converted "rc.conf", /etc/fstab, /etc/ttys, > /etc/my.cnf, and other stuff there. > > One example of this in action, out of many in the import subsystem, is > jails that have been set up the version 9 way in "rc.conf". Those are > turned into service bundles, with "env" environment directories that > contain environment settings such as "hostname", "mount_devfs", and > "interface". The "run" script for the jail service very simply turns the > environment variables into arguments to the "jail" comand. In a system > with an original OpenBSD "rcctl" command, one would expect to be able to > set (version 9) jail control variables by manipulating /etc/rc.conf with > commands like "rcctl set wibble hostname wobble". The "rcctl" shim and > this shared convention mean that one need not stray that far from this if > "rcctl" is one's habit: "rcctl set v9-jail@wibble hostname wobble" does > the "native" thing of setting the "hostname" variable in the (conventional) > environment variable directory for the "v9-jail@wibble" service. > > Bonus feature for those with other habits: With nosh service management in > place, one can actually import from /etc/rc.conf settings *on Debian* (as > long as one sets up a FreeBSD/PC-BSD-style /etc/defaults/rc.conf pointing > to it with rc_conf_files). One can use /etc/ttys, too. > > As for importing scripts that run "arbitrary shell commands", there are > several points. > > First, you may not need to. Note that most of what you get out of the box > in /etc/rc.d/ and /usr/local/etc/rc.d/ on FreeBSD and PC-BSD has already > been converted. Remember that project that I had to convert 157 services? > Take a look at the nosh roadmap page. It's almost done. > > Second, you may not need to. Take a look at what actually comes in the > nosh-bundles package nowadays. Discounting the 'cyclog@' service bundles > there are just over 540 service bundles in there, from samba to ntp, from > saned to ossec@agentd. (Including the 'cyclog@' service bundles, it is > over a thousand service bundles.) The Debian world doesn't get left out, > either. Although it's a lot more difficult than in the BSD worlds to come > up with a list of "core" Debian services, a lot of the basics of Debian are > also covered by this, from kernel-vt-setfont through irqbalance to > update-binfmts. And those more-than-540 service bundles cover lots of > "non-core" stuff, from (as aforementioned) OSSEC-HIDS, Salt, and RabbitMQ > to publicfile httpd over IPV6. > > Third, you may not need to. This was mentioned in the 1.28 announcement, > in fact. The external configuration import subsystem makes *further* > service bundles, beyond the pre-made ones that come in a binary package. > It creates service bundles to run (optional) per-user service management, > per-user Desktop Bus brokers, MySQL and MariaDB servers (according to your > my.cnf), PPP and SPPP, md and pefs, jails (set up with v9 rc.conf or the > PC-BSD Warden), tinydns and dnscache services (not quite ready when 1.28 > came out, as the announcement said), static IP4/IP6/ARP/NDP setup and > teardown, and more besides. > > Fourth, you may not need to. Out of all of this, there's probably already > an existing service bundle for something similar that one can copy and > adapt. > > Fifth, you may not need to. The convert-systemd-units tool exists, after > all. If there's a system service/socket unit around, converting that may > well be simpler starting point than starting with an rc script. It's > usually significantly simpler than starting with a van Smoorenburg rc > script, although Mewburn and OpenBSD rc scripts can themselves be fairly > simple starting points. I did a "make fetch" on the PC-BSD ports tree a > couple of months ago. (As an aside: there are several broken ports that > don't do the right thing here.) There are a growing number of packages > where there's now a systemd service/socket unit in the fetched source > archive. > > Sixth, the easy cases are easy. As just noted, Mewburn and OpenBSD rc > scripts can themselves be fairly simple. (They are not *always* so, > though, contrary to popular belief.) If you have an rc script that says > "The command name is this, its arguments are that.", it is very easy indeed > to convert this into something that can execute as a "run" program. > Setting up all of the stuff around the "run" program for a complete service > bundle is merely an exercise in two-line shell scripts (for things like > "start" and "stop") and making directories and symbolic links (for things > like the "before" and "wants" directories). > > Seventh, the hard cases require a human being anyway. Parsing a shell > script that runs "arbitrary shell commands" would require creating what is > essentially a full shell script interpreter, that can handle the Almquist, > Bourne Again, and Korn shell syntaxes (because such a hypothetical > *general-case* conversion tool would have to address van Smoorenburg rc > scripts on Debian, Mewburn rc scripts on FreeBSD/PC-BSD/NetBSD and friends, > and OpenBSD rc scripts) that knows about at least five quite different sets > of "helper" commands (from start-stop-daemon to startproc) and that works > out how an entire shell script translates into the actual acts of executing > one or (in really bad cases) more services. At this point, I defer to a > human being *understanding what is needed* and writing one or more service > bundles. (-: > > And there is, of course, scads of doco, written over the past two decades > by many people, on how to write daemontools-family-style "run" scripts. > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > --94eb2c14bc96304776053b4c0daf Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Thank you very much.
And have a blessed day.=

On Sun,= Aug 21, 2016 at 7:04 AM, Jonathan de Boyne Pollard <<= a href=3D"mailto:J.deBoynePollard-newsgroups@ntlworld.com" target=3D"_blank= ">J.deBoynePollard-newsgroups@ntlworld.com> wrote:
I don't know why you asked about FreeBSD rc.d j= ust on the Debian mailing list; but I'm going to deal in both of those = and others besides, here, and things that apply across both, so I've re= -included the FreeBSD mailing list.=C2=A0 (-:

2016-08-14 15:10, Julian Elischer:

I don't know if I just missed it, or it isn't there but=C2=A0 I hav= e a question..

You give examples of importing systemd service files.=C2=A0 What about impo= rting rc.d files with all their ability to run arbitrary shell commands.
And once you have the services defined, what is the logical equivalent of r= c.conf, which can supply parameters for each service and turn them on and o= ff?=C2=A0 can you import from rc.conf?

You did miss it.=C2=A0 (-:

What you missed has grown to be a significant subsystem. It was actually me= ntioned a couple of times in the 1.28 announcement. It's the external c= onfiguration import subsystem.=C2=A0 You can read about it in the nosh Guid= e:

xdg-open /usr/local/share/doc/nosh/external-formats.html

As you can see, there's a whole section on importing from rc.conf into = native service management mechanisms.=C2=A0 ("rc.conf" covers sev= eral sources, note, including a FreeNAS configuration database and /etc/def= aults/rc.conf .)

The native service mangement mechanisms are the "enable" and &quo= t;disable" subcommands to the system-control command, and using the en= vdir command in the normal daemontools-family style way.=C2=A0 The enable/d= isable mechanism in "rc.conf" is treated as if it were a preset (= in systemd nomenclature).=C2=A0 You tell service management to "preset= " a service, and it will look at /etc/rc.conf and /etc/rc.conf.local (= as well as some other preset mechanisms) to determine what to set the nativ= e enable/disable state to.=C2=A0 The user manual page for the preset subcom= mand (of system-control) explains what the preset mechanisms are in detail.=

You can set up environment directories how and where you like, but there= 9;s a convention that is shared by the "convert-systemd-units" to= ol, the "rcctl" shim, and the external configuration import subsy= stem as a whole. This convention is an environment directory named "en= v" that is in the service directory.=C2=A0 The "rcctl" shim = gets and sets variables there; and the import subsystem places converted &q= uot;rc.conf", /etc/fstab, /etc/ttys, /etc/my.cnf, and other stuff ther= e.

One example of this in action, out of many in the import subsystem, is jail= s that have been set up the version 9 way in "rc.conf".=C2=A0 Tho= se are turned into service bundles, with "env" environment direct= ories that contain environment settings such as "hostname", "= ;mount_devfs", and "interface".=C2=A0 The "run" sc= ript for the jail service very simply turns the environment variables into = arguments to the "jail" comand.=C2=A0 In a system with an origina= l OpenBSD "rcctl" command, one would expect to be able to set (ve= rsion 9) jail control variables by manipulating /etc/rc.conf with commands = like "rcctl set wibble hostname wobble".=C2=A0 The "rcctl&qu= ot; shim and this shared convention mean that one need not stray that far f= rom this if "rcctl" is one's habit: "rcctl set v9-jail@w= ibble hostname wobble" does the "native" thing of setting th= e "hostname" variable in the (conventional) environment variable = directory for the "v9-jail@wibble" service.

Bonus feature for those with other habits: With nosh service management in = place, one can actually import from /etc/rc.conf settings *on Debian* (as l= ong as one sets up a FreeBSD/PC-BSD-style /etc/defaults/rc.conf pointing to= it with rc_conf_files).=C2=A0 One can use /etc/ttys, too.

As for importing scripts that run "arbitrary shell commands", the= re are several points.

First, you may not need to.=C2=A0 Note that most of what you get out of the= box in /etc/rc.d/ and /usr/local/etc/rc.d/ on FreeBSD and PC-BSD has alrea= dy been converted.=C2=A0 Remember that project that I had to convert 157 se= rvices?=C2=A0 Take a look at the nosh roadmap page.=C2=A0 It's almost d= one.

Second, you may not need to.=C2=A0 Take a look at what actually comes in th= e nosh-bundles package nowadays.=C2=A0 Discounting the 'cyclog@' se= rvice bundles there are just over 540 service bundles in there, from samba = to ntp, from saned to ossec@agentd.=C2=A0 (Including the 'cyclog@' = service bundles, it is over a thousand service bundles.)=C2=A0 The Debian w= orld doesn't get left out, either. Although it's a lot more difficu= lt than in the BSD worlds to come up with a list of "core" Debian= services, a lot of the basics of Debian are also covered by this, from ker= nel-vt-setfont through irqbalance to update-binfmts.=C2=A0 And those more-t= han-540 service bundles cover lots of "non-core" stuff, from (as = aforementioned) OSSEC-HIDS, Salt, and RabbitMQ to publicfile httpd over IPV= 6.

Third, you may not need to.=C2=A0 This was mentioned in the 1.28 announceme= nt, in fact.=C2=A0 The external configuration import subsystem makes *furth= er* service bundles, beyond the pre-made ones that come in a binary package= .=C2=A0 It creates service bundles to run (optional) per-user service manag= ement, per-user Desktop Bus brokers, MySQL and MariaDB servers (according t= o your my.cnf), PPP and SPPP, md and pefs, jails (set up with v9 rc.conf or= the PC-BSD Warden), tinydns and dnscache services (not quite ready when 1.= 28 came out, as the announcement said), static IP4/IP6/ARP/NDP setup and te= ardown, and more besides.

Fourth, you may not need to.=C2=A0 Out of all of this, there's probably= already an existing service bundle for something similar that one can copy= and adapt.

Fifth, you may not need to.=C2=A0 The convert-systemd-units tool exists, af= ter all.=C2=A0 If there's a system service/socket unit around, converti= ng that may well be simpler starting point than starting with an rc script.= =C2=A0 It's usually significantly simpler than starting with a van Smoo= renburg rc script, although Mewburn and OpenBSD rc scripts can themselves b= e fairly simple starting points.=C2=A0 I did a "make fetch" on th= e PC-BSD ports tree a couple of months ago.=C2=A0 (As an aside: there are s= everal broken ports that don't do the right thing here.)=C2=A0 There ar= e a growing number of packages where there's now a systemd service/sock= et unit in the fetched source archive.

Sixth, the easy cases are easy.=C2=A0 As just noted, Mewburn and OpenBSD rc= scripts can themselves be fairly simple.=C2=A0 (They are not *always* so, = though, contrary to popular belief.)=C2=A0 If you have an rc script that sa= ys "The command name is this, its arguments are that.", it is ver= y easy indeed to convert this into something that can execute as a "ru= n" program.=C2=A0 Setting up all of the stuff around the "run&quo= t; program for a complete service bundle is merely an exercise in two-line = shell scripts (for things like "start" and "stop") and = making directories and symbolic links (for things like the "before&quo= t; and "wants" directories).

Seventh, the hard cases require a human being anyway.=C2=A0 Parsing a shell= script that runs "arbitrary shell commands" would require creati= ng what is essentially a full shell script interpreter, that can handle the= Almquist, Bourne Again, and Korn shell syntaxes (because such a hypothetic= al *general-case* conversion tool would have to address van Smoorenburg rc = scripts on Debian, Mewburn rc scripts on FreeBSD/PC-BSD/NetBSD and friends,= and OpenBSD rc scripts) that knows about at least five quite different set= s of "helper" commands (from start-stop-daemon to startproc) and = that works out how an entire shell script translates into the actual acts o= f executing one or (in really bad cases) more services.=C2=A0 At this point= , I defer to a human being *understanding what is needed* and writing one o= r more service bundles.=C2=A0 (-:

And there is, of course, scads of doco, written over the past two decades b= y many people, on how to write daemontools-family-style "run" scr= ipts.

_______________________________________________
freebsd-ha= ckers@freebsd.org mailing list
https://lists.freebsd.org/mailman/li= stinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@f= reebsd.org"

--94eb2c14bc96304776053b4c0daf--