The Unix Heritage Society mailing list
 help / color / mirror / Atom feed
* Re: [TUHS] Abstractions
@ 2021-02-23 19:37 Nelson H. F. Beebe
  2021-02-23 21:02 ` Charles H. Sauer
  2021-02-24  1:51 ` Greg A. Woods
  0 siblings, 2 replies; 35+ messages in thread
From: Nelson H. F. Beebe @ 2021-02-23 19:37 UTC (permalink / raw)
  To: tuhs

The recent discussions on the TUHS list of whether /bin and /usr/bin
are different, or symlinked, brought to mind the limited disk and tape
sizes of the 1970s and 1980s.  Especially the lower-cost tape
technologies had issues with correct recognition of an end-of-tape
condition, making it hard to span a dump across tape volumes, and
strongly suggesting that directory tree sizes be limited to what could
fit on a single tape.

I made an experiment today across a broad range of operating systems
(many with multiple versions in our test farm), and produced these two
tables, where version numbers are included only if the O/S changed
practices:

------------------------------------------------------------------------
Systems with /bin a symlink to /usr/bin (or both to yet another common
directory) [42 major variants]:

	ArchLinux		Kali			RedHat 8
	Arco			Kubuntu 19, 20		Q4OS
	Bitrig			Lite			ScientificLinux 7
	CentOS 7, 8		Lubuntu 19		Septor
	ClearLinux		Mabox			Solaris 10, 11
	Debian 10, 11		Magiea			Solydk
	Deepin			Manjaro			Sparky
	DilOS			Mint 20			Springdale
	Dyson			MXLinux 19		Ubuntu 19, 20, 21
	Fedora			Neptune			UCS
	Gnuinos			Netrunner		Ultimate
	Gobolinux		Oracle Linux		Unleashed
	Hefftor			Parrot 4.7		Void
	IRIX			PureOS			Xubuntu 19, 20

------------------------------------------------------------------------
Systems with separate /bin and /usr/bin [60 major variants]:

	Alpine			Hipster			OS108
	AltLinux		KaOS			Ovios
	Antix			KFreeBSD		PacBSD
	Bitrig			Kubuntu 18		Parrot 4.5
	Bodhi			LibertyBSD		PCBSD
	CentOS 5, 6		LMDE			PCLinuxOS
	ClonOS			Lubuntu 17		Peppermint
	Debian 7--10		LXLE			Salix
	DesktopBSD		macOS			ScientificLinux 6
	Devuan			MidnightBSD		SlackEX
	DragonFlyBSD		Mint 18--20		Slackware
	ElementaryOS		MirBSD			Solus
	FreeBSD 9--13		MXLinux 17, 18		T2
	FuryBSD			NetBSD 6-1010		Trident
	Gecko			NomadBSD		Trisquel
	Gentoo			OmniOS			TrueOS
	GhostBSD		OmniTribblix		Ubuntu 14--18
	GNU/Hurd		OpenBSD			Xubuntu 18
	HardenedBSD		OpenMandriva		Zenwalk
	Helium			openSUSE		Zorinos
	
------------------------------------------------------------------------

Some names appear in both tables, indicating a transition from
separate directories to symlinked directories in more recent O/S
releases.

Many of these system names are spelled in mixed lettercase, and if
I've botched some of them, I extend my apologies to their authors.

Some of those systems run on multiple CPU architectures, and our test
farm exploits that; however, I found no instance of the CPU type
changing the separation or symbolic linking of /bin and /usr/bin.

-------------------------------------------------------------------------------
- Nelson H. F. Beebe                    Tel: +1 801 581 5254                  -
- University of Utah                    FAX: +1 801 581 4148                  -
- Department of Mathematics, 110 LCB    Internet e-mail: beebe@math.utah.edu  -
- 155 S 1400 E RM 233                       beebe@acm.org  beebe@computer.org -
- Salt Lake City, UT 84112-0090, USA    URL: http://www.math.utah.edu/~beebe/ -
-------------------------------------------------------------------------------

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-23 19:37 [TUHS] Abstractions Nelson H. F. Beebe
@ 2021-02-23 21:02 ` Charles H. Sauer
  2021-02-23 21:15   ` Henry Bent
  2021-02-24  1:51 ` Greg A. Woods
  1 sibling, 1 reply; 35+ messages in thread
From: Charles H. Sauer @ 2021-02-23 21:02 UTC (permalink / raw)
  To: tuhs

To add to the inventory below:
Dell SVR4 /bin is a symlink to /usr/bin
NEXTSTEP/486 3.3 /bin and /usr/bin are separate

On 2/23/2021 1:37 PM, Nelson H. F. Beebe wrote:
> The recent discussions on the TUHS list of whether /bin and /usr/bin
> are different, or symlinked, brought to mind the limited disk and tape
> sizes of the 1970s and 1980s.  Especially the lower-cost tape
> technologies had issues with correct recognition of an end-of-tape
> condition, making it hard to span a dump across tape volumes, and
> strongly suggesting that directory tree sizes be limited to what could
> fit on a single tape.
> 
> I made an experiment today across a broad range of operating systems
> (many with multiple versions in our test farm), and produced these two
> tables, where version numbers are included only if the O/S changed
> practices:
> 
> ------------------------------------------------------------------------
> Systems with /bin a symlink to /usr/bin (or both to yet another common
> directory) [42 major variants]:
> 
> 	ArchLinux		Kali			RedHat 8
> 	Arco			Kubuntu 19, 20		Q4OS
> 	Bitrig			Lite			ScientificLinux 7
> 	CentOS 7, 8		Lubuntu 19		Septor
> 	ClearLinux		Mabox			Solaris 10, 11
> 	Debian 10, 11		Magiea			Solydk
> 	Deepin			Manjaro			Sparky
> 	DilOS			Mint 20			Springdale
> 	Dyson			MXLinux 19		Ubuntu 19, 20, 21
> 	Fedora			Neptune			UCS
> 	Gnuinos			Netrunner		Ultimate
> 	Gobolinux		Oracle Linux		Unleashed
> 	Hefftor			Parrot 4.7		Void
> 	IRIX			PureOS			Xubuntu 19, 20
> 
> ------------------------------------------------------------------------
> Systems with separate /bin and /usr/bin [60 major variants]:
> 
> 	Alpine			Hipster			OS108
> 	AltLinux		KaOS			Ovios
> 	Antix			KFreeBSD		PacBSD
> 	Bitrig			Kubuntu 18		Parrot 4.5
> 	Bodhi			LibertyBSD		PCBSD
> 	CentOS 5, 6		LMDE			PCLinuxOS
> 	ClonOS			Lubuntu 17		Peppermint
> 	Debian 7--10		LXLE			Salix
> 	DesktopBSD		macOS			ScientificLinux 6
> 	Devuan			MidnightBSD		SlackEX
> 	DragonFlyBSD		Mint 18--20		Slackware
> 	ElementaryOS		MirBSD			Solus
> 	FreeBSD 9--13		MXLinux 17, 18		T2
> 	FuryBSD			NetBSD 6-1010		Trident
> 	Gecko			NomadBSD		Trisquel
> 	Gentoo			OmniOS			TrueOS
> 	GhostBSD		OmniTribblix		Ubuntu 14--18
> 	GNU/Hurd		OpenBSD			Xubuntu 18
> 	HardenedBSD		OpenMandriva		Zenwalk
> 	Helium			openSUSE		Zorinos
> 	
> ------------------------------------------------------------------------
> 
> Some names appear in both tables, indicating a transition from
> separate directories to symlinked directories in more recent O/S
> releases.
> 
> Many of these system names are spelled in mixed lettercase, and if
> I've botched some of them, I extend my apologies to their authors.
> 
> Some of those systems run on multiple CPU architectures, and our test
> farm exploits that; however, I found no instance of the CPU type
> changing the separation or symbolic linking of /bin and /usr/bin.
> 
> -------------------------------------------------------------------------------
> - Nelson H. F. Beebe                    Tel: +1 801 581 5254                  -
> - University of Utah                    FAX: +1 801 581 4148                  -
> - Department of Mathematics, 110 LCB    Internet e-mail: beebe@math.utah.edu  -
> - 155 S 1400 E RM 233                       beebe@acm.org  beebe@computer.org -
> - Salt Lake City, UT 84112-0090, USA    URL: http://www.math.utah.edu/~beebe/ -
> -------------------------------------------------------------------------------
> 

-- 
voice: +1.512.784.7526       e-mail: sauer@technologists.com
fax: +1.512.346.5240         Web: https://technologists.com/sauer/
Facebook/Google/Skype/Twitter: CharlesHSauer

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-23 21:02 ` Charles H. Sauer
@ 2021-02-23 21:15   ` Henry Bent
  2021-02-24  2:47     ` Greg A. Woods
  0 siblings, 1 reply; 35+ messages in thread
From: Henry Bent @ 2021-02-23 21:15 UTC (permalink / raw)
  To: Charles H. Sauer; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 4421 bytes --]

On Tue, 23 Feb 2021 at 16:03, Charles H. Sauer <sauer@technologists.com>
wrote:

> To add to the inventory below:
> Dell SVR4 /bin is a symlink to /usr/bin
> NEXTSTEP/486 3.3 /bin and /usr/bin are separate
>
> On 2/23/2021 1:37 PM, Nelson H. F. Beebe wrote:
> > The recent discussions on the TUHS list of whether /bin and /usr/bin
> > are different, or symlinked, brought to mind the limited disk and tape
> > sizes of the 1970s and 1980s.  Especially the lower-cost tape
> > technologies had issues with correct recognition of an end-of-tape
> > condition, making it hard to span a dump across tape volumes, and
> > strongly suggesting that directory tree sizes be limited to what could
> > fit on a single tape.
> >
> > I made an experiment today across a broad range of operating systems
> > (many with multiple versions in our test farm), and produced these two
> > tables, where version numbers are included only if the O/S changed
> > practices:
> >
> > ------------------------------------------------------------------------
> > Systems with /bin a symlink to /usr/bin (or both to yet another common
> > directory) [42 major variants]:
> >
> >       ArchLinux               Kali                    RedHat 8
> >       Arco                    Kubuntu 19, 20          Q4OS
> >       Bitrig                  Lite                    ScientificLinux 7
> >       CentOS 7, 8             Lubuntu 19              Septor
> >       ClearLinux              Mabox                   Solaris 10, 11
> >       Debian 10, 11           Magiea                  Solydk
> >       Deepin                  Manjaro                 Sparky
> >       DilOS                   Mint 20                 Springdale
> >       Dyson                   MXLinux 19              Ubuntu 19, 20, 21
> >       Fedora                  Neptune                 UCS
> >       Gnuinos                 Netrunner               Ultimate
> >       Gobolinux               Oracle Linux            Unleashed
> >       Hefftor                 Parrot 4.7              Void
> >       IRIX                    PureOS                  Xubuntu 19, 20
> >
> > ------------------------------------------------------------------------
> > Systems with separate /bin and /usr/bin [60 major variants]:
> >
> >       Alpine                  Hipster                 OS108
> >       AltLinux                KaOS                    Ovios
> >       Antix                   KFreeBSD                PacBSD
> >       Bitrig                  Kubuntu 18              Parrot 4.5
> >       Bodhi                   LibertyBSD              PCBSD
> >       CentOS 5, 6             LMDE                    PCLinuxOS
> >       ClonOS                  Lubuntu 17              Peppermint
> >       Debian 7--10            LXLE                    Salix
> >       DesktopBSD              macOS                   ScientificLinux 6
> >       Devuan                  MidnightBSD             SlackEX
> >       DragonFlyBSD            Mint 18--20             Slackware
> >       ElementaryOS            MirBSD                  Solus
> >       FreeBSD 9--13           MXLinux 17, 18          T2
> >       FuryBSD                 NetBSD 6-1010           Trident
> >       Gecko                   NomadBSD                Trisquel
> >       Gentoo                  OmniOS                  TrueOS
> >       GhostBSD                OmniTribblix            Ubuntu 14--18
> >       GNU/Hurd                OpenBSD                 Xubuntu 18
> >       HardenedBSD             OpenMandriva            Zenwalk
> >       Helium                  openSUSE                Zorinos
> >
> > ------------------------------------------------------------------------
> >
> > Some names appear in both tables, indicating a transition from
> > separate directories to symlinked directories in more recent O/S
> > releases.
> >
> > Many of these system names are spelled in mixed lettercase, and if
> > I've botched some of them, I extend my apologies to their authors.
> >
> > Some of those systems run on multiple CPU architectures, and our test
> > farm exploits that; however, I found no instance of the CPU type
> > changing the separation or symbolic linking of /bin and /usr/bin.
> >
>
>
Solaris /bin was a symlink to /usr/bin as early as 2.5.1.  It's also worth
pointing out that NetBSD, in addition to having a separate /bin and
/usr/bin, has /rescue which has a large selection of statically linked
binaries.

-Henry

[-- Attachment #2: Type: text/html, Size: 5886 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-23 19:37 [TUHS] Abstractions Nelson H. F. Beebe
  2021-02-23 21:02 ` Charles H. Sauer
@ 2021-02-24  1:51 ` Greg A. Woods
  2021-02-24  2:23   ` Nelson H. F. Beebe
  1 sibling, 1 reply; 35+ messages in thread
From: Greg A. Woods @ 2021-02-24  1:51 UTC (permalink / raw)
  To: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 2568 bytes --]

At Tue, 23 Feb 2021 12:37:57 -0700, "Nelson H. F. Beebe" <beebe@math.utah.edu> wrote:
Subject: Re: [TUHS] Abstractions
>
> The recent discussions on the TUHS list of whether /bin and /usr/bin
> are different, or symlinked, brought to mind the limited disk and tape
> sizes of the 1970s and 1980s.  Especially the lower-cost tape
> technologies had issues with correct recognition of an end-of-tape
> condition, making it hard to span a dump across tape volumes, and
> strongly suggesting that directory tree sizes be limited to what could
> fit on a single tape.

Hmmmm... you may just be mixing up the names of the archive tools you
mean, but on the other hand maybe you don't know that "dump" does whole
filesystems, not just sub-directories.

That of course doesn't take anything away from what you were saying
about making sure you could do a full dump onto a single tape with some
types of less high-end and high-quality tape devices.  But that's a
"newer" problem.

Original Unix dump(1m) had no trouble asking for additional tapes to be
mounted when the filesystem required multiple tapes.

So it has nothing to do with legacy of the original root and /usr split.


> I made an experiment today across a broad range of operating systems
> (many with multiple versions in our test farm), and produced these two
> tables, where version numbers are included only if the O/S changed
> practices:

An interesting compilation, but sadly (to me at least) it is mostly a
mess of GNU/Linux which, rightly or wrongly, I categorize all under one
(extremely opaque) umbrella.

BTW, when I said "long ago" for Solaris, I meant a REALLY long time ago.
/bin has been a symlink to /usr/bin since Solaris 2.0 and yet /usr
could/can still be a separate filesystem on a Solaris installation.
This is accomplished by putting everything necessary to boot the system
up to the point where other additional filesystems can be mounted using
just the programs found in /sbin.  Of course this wasn't done smoothly
and completely all in one go.  IIRC /sbin/sh didn't exist until
Solaris-9, and (also IIRC) it is just a copy of /usr/bin/sh.

So Sun pushed everything in /bin to /usr/bin, then copied a few things
back to /sbin as they found they needed them.  Kind of a half-assed hack
that wasn't well thought out, and had very poor motivations.

I'd forgot that IRIX was following Solaris on this track.

--
					Greg A. Woods <gwoods@acm.org>

Kelowna, BC     +1 250 762-7675           RoboHack <woods@robohack.ca>
Planix, Inc. <woods@planix.com>     Avoncote Farms <woods@avoncote.ca>

[-- Attachment #2: OpenPGP Digital Signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-24  1:51 ` Greg A. Woods
@ 2021-02-24  2:23   ` Nelson H. F. Beebe
  0 siblings, 0 replies; 35+ messages in thread
From: Nelson H. F. Beebe @ 2021-02-24  2:23 UTC (permalink / raw)
  To: The Unix Heritage Society mailing list

Greg Woods responds to my posting:

>> Hmmmm... you may just be mixing up the names of the archive tools you
>> mean, but on the other hand maybe you don't know that "dump" does whole
>> filesystems, not just sub-directories.

I meant "dump" as a generic verb, not specifically the Unix dump
utility.  Many sites also used tar to backup directory trees: after
all, tar means Tape ARchiver.

>> Original Unix dump(1m) had no trouble asking for additional tapes ...

That was, however, contingent on a reliable signal from the tape unit,
and my strong recollection is that when we moved to various types of
cheap cassette tapes, the end-of-tape indicator was unreliable.  Thus,
we paid attention to both disk and tape sizes.

Today, with 10TB+ on LTO-8 tapes, it isn't an issue for us, and we
also tend to have many different ZFS volumes representing various
parts of the filesystem, allowing different backup and snapshotting
policies.  Besides tapes and snapshots, we also have a live SAN
mirror, and a remote snapshot server, giving plenty of data
replication, and the warm fuzzy feelings from that.  After 20 years of
ZFS, I don't recall us ever losing data.  We have also gone through
two generations of major fileserver upgrades and complete data
migrations without service interruptions (except for a brief interval
for each user account to synchronize data on old and new servers).

-------------------------------------------------------------------------------
- Nelson H. F. Beebe                    Tel: +1 801 581 5254                  -
- University of Utah                    FAX: +1 801 581 4148                  -
- Department of Mathematics, 110 LCB    Internet e-mail: beebe@math.utah.edu  -
- 155 S 1400 E RM 233                       beebe@acm.org  beebe@computer.org -
- Salt Lake City, UT 84112-0090, USA    URL: http://www.math.utah.edu/~beebe/ -
-------------------------------------------------------------------------------

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-23 21:15   ` Henry Bent
@ 2021-02-24  2:47     ` Greg A. Woods
  2021-02-24  3:20       ` Warner Losh
  0 siblings, 1 reply; 35+ messages in thread
From: Greg A. Woods @ 2021-02-24  2:47 UTC (permalink / raw)
  To: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 1911 bytes --]

At Tue, 23 Feb 2021 16:15:52 -0500, Henry Bent <henry.r.bent@gmail.com> wrote:
Subject: Re: [TUHS] Abstractions
>
> It's also worth
> pointing out that NetBSD, in addition to having a separate /bin and
> /usr/bin, has /rescue which has a large selection of statically linked
> binaries.

Indeed.  However /rescue is really just a hack to avoid the problems
that occur when basic tools are dynamic-linked.

My vastly preferred alternative is to static-link everything.

Of course with C libraries these days that means the binaries can be
rather large -- albiet still relatively small in comparison to modern
disks.

In any case I've also built NetBSD such that all of the base system
binaries are linked together into one binary (we call this "crunchgen",
but Linux usually calls it "Busybox(tm)").  I decided to put all the bin
directories together into one for the ultimate savings of space and time
and effort, but it would be trivial to keep the root and /usr split for
better managing application-specific embedded systems.

This hard-static-linking of everything into one binary results in a
surprisingly small, indeed very tiny, system.  For i386 (32-bit) it
could probably boot multiuser in about 16mb of RAM.

What I've got so far is a bootable image file of a "complete"
NetBSD-5/i386 systems that's just a tiny bit over 7Mb.  It contains a
kernel and a ramdisk image with a 12Mb filesystem containing a crunchgen
binary with almost everything in it (247 system programs, including all
the networking tools, but no named, and no toolchain, no mailer, and no
manual pages -- not atypical of what was delivered with some commercial
unix systems of days gone by, but of course updated with modern things
like ssh, etc..)

--
					Greg A. Woods <gwoods@acm.org>

Kelowna, BC     +1 250 762-7675           RoboHack <woods@robohack.ca>
Planix, Inc. <woods@planix.com>     Avoncote Farms <woods@avoncote.ca>

[-- Attachment #2: OpenPGP Digital Signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-24  2:47     ` Greg A. Woods
@ 2021-02-24  3:20       ` Warner Losh
  2021-02-24 20:05         ` Greg A. Woods
  0 siblings, 1 reply; 35+ messages in thread
From: Warner Losh @ 2021-02-24  3:20 UTC (permalink / raw)
  To: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 2488 bytes --]

On Tue, Feb 23, 2021, 7:47 PM Greg A. Woods <woods@robohack.ca> wrote:

> At Tue, 23 Feb 2021 16:15:52 -0500, Henry Bent <henry.r.bent@gmail.com>
> wrote:
> Subject: Re: [TUHS] Abstractions
> >
> > It's also worth
> > pointing out that NetBSD, in addition to having a separate /bin and
> > /usr/bin, has /rescue which has a large selection of statically linked
> > binaries.
>
> Indeed.  However /rescue is really just a hack to avoid the problems
> that occur when basic tools are dynamic-linked.
>
> My vastly preferred alternative is to static-link everything.
>
> Of course with C libraries these days that means the binaries can be
> rather large -- albiet still relatively small in comparison to modern
> disks.
>
> In any case I've also built NetBSD such that all of the base system
> binaries are linked together into one binary (we call this "crunchgen",
> but Linux usually calls it "Busybox(tm)").  I decided to put all the bin
> directories together into one for the ultimate savings of space and time
> and effort, but it would be trivial to keep the root and /usr split for
> better managing application-specific embedded systems.
>
> This hard-static-linking of everything into one binary results in a
> surprisingly small, indeed very tiny, system.  For i386 (32-bit) it
> could probably boot multiuser in about 16mb of RAM.
>

I booted a FreeBSD/i386 4 system, sans compilers and a few other things,
off 16MB CF card in the early 2000s. I did both static (one binary) and
dynamic and found dynamic worked a lot better for the embedded system...

I also did a 8MB PoC router and data logger image that was stripped to the
bone.

PicoBSD fit onto a 1.44MB floppy as lat as FreeBSD 4 and made a good
firewall...

Warner

What I've got so far is a bootable image file of a "complete"
> NetBSD-5/i386 systems that's just a tiny bit over 7Mb.  It contains a
> kernel and a ramdisk image with a 12Mb filesystem containing a crunchgen
> binary with almost everything in it (247 system programs, including all
> the networking tools, but no named, and no toolchain, no mailer, and no
> manual pages -- not atypical of what was delivered with some commercial
> unix systems of days gone by, but of course updated with modern things
> like ssh, etc..)
>
> --
>                                         Greg A. Woods <gwoods@acm.org>
>
> Kelowna, BC     +1 250 762-7675           RoboHack <woods@robohack.ca>
> Planix, Inc. <woods@planix.com>     Avoncote Farms <woods@avoncote.ca>
>

[-- Attachment #2: Type: text/html, Size: 3755 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-24  3:20       ` Warner Losh
@ 2021-02-24 20:05         ` Greg A. Woods
  0 siblings, 0 replies; 35+ messages in thread
From: Greg A. Woods @ 2021-02-24 20:05 UTC (permalink / raw)
  To: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 4028 bytes --]

At Tue, 23 Feb 2021 20:20:55 -0700, Warner Losh <imp@bsdimp.com> wrote:
Subject: Re: [TUHS] Abstractions
>
> I booted a FreeBSD/i386 4 system, sans compilers and a few other things,
> off 16MB CF card in the early 2000s. I did both static (one binary) and
> dynamic and found dynamic worked a lot better for the embedded system...

I guess it may depend on your measure of "better"?

With a single static-linked binary on a modern demand paged system with
shared text pages, the effect is that almost all instructions for any
and all programs (and of course all libraries) are almost always paged
in at any given time.  The result is that program startup requires so
few page-in faults that it appears to happen instantaneously.

My little i386 image feels faster at the command line (e.g. running on
an old Soekris box, even when the root filesystem is on a rather slow
flash drive) than on any of the fastest non-static-linked systems I've
ever used because of this -- that is of course until it is asked to do
any actual computing or other I/O operations.  :-)

So, in an embedded system there will be many influencing factors,
including such as how many exec()s there are during normal operations.

For machines with oodles of memory and very fast and large SSDs (and
using any kernel with a decently tuneable paging system) one can simply
static-link all binaries separately and achieve similar results, at
least for programs that are run relatively often.

For example the build times of a full system build of, e.g. NetBSD, with
a fully static-linked host system and toolchain are remarkably lower
than on a fully dynamic-linked system since all the extra processing
(and esp. any extra I/Os) done by the "stupid" dynamic linker (i.e. the
one that's ubiquitous in modern unixy systems) are completely and
forever eliminated.  I haven't even measured the difference in years now
because I find fully dynamic-linked systems too painful to use for
intensive development of large systems.

Taking this to the opposite extreme one need only use modern macOS on a
machine with an older spinning-rust hard drive that has a loud seek arm
to hear and feel how incredibly slow even the simplest tasks can be,
e.g. typing "man man" after a reboot or a few days of not running "man".
This is because on top of the "stupid" dynamic linker that's needed to
start the "man" program, there's also a huge stinking pile of additional
wrappers has been added to all of the toolchain command-line tools that
require doing even more gratuitous I/O operations (as well as running
perhaps millions more gratuitous instructions) for infrequent
invocations (luckily these wrappers seem to cache some of the most
expensive overhead).  (note: "man" is not in the same boat as, e.g. the
toolchain progs, and I'm not quite sure why it churns so much on first
invocations)

My little static-linked i386 system can run "man man" several (many?)
thousand times before my old iMac can display even the first line of
output.  And that's for a simple small program -- just imagine the
immense atrocities necessary to run a program that links to several
dozen libraries (e.g. the typical GUI application like a web browser,
with the saving grace that we don't usually restart browsers in a loop
like we restart compilers; but, e.g. /usr/bin/php on macos links to 21
libraries, and even the linker (ld) needs 7 dynamic libraries).

BTW, a non-stupid dynamic linker would work the way Multics did (and to
some extent I think that's more how dynamic linking worked in AT&T UNIX
(SysVr3.2) on the 3B2s), but such things are so much more complicated in
a flat address space.  Pre-binding, such as I think macOS and IRIX do
(and maybe can be done with the most modern binutils), are somewhat like
Multics "bound segments" (though still less flexible and perhaps less
performant).

--
					Greg A. Woods <gwoods@acm.org>

Kelowna, BC     +1 250 762-7675           RoboHack <woods@robohack.ca>
Planix, Inc. <woods@planix.com>     Avoncote Farms <woods@avoncote.ca>

[-- Attachment #2: OpenPGP Digital Signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-22  0:13   ` Warren Toomey
@ 2021-02-27  2:47     ` Dave Horsfall
  0 siblings, 0 replies; 35+ messages in thread
From: Dave Horsfall @ 2021-02-27  2:47 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On Mon, 22 Feb 2021, Warren Toomey wrote:

>> That's how we ran our RK-05 11/40s since Ed 5...  Good fun writing a 
>> DJ-11 driver from the DH-11 source; even more fun when I wrote a UT-200 
>> driver from the manual alone (I'm sure that "ei.c" is Out There 
>> Somewhere), junking IanJ's driver.
>
> https://minnie.tuhs.org/cgi-bin/utree.pl?file=AUSAM/sys/dmr/ei.c

Nah; that's the IanJ rubbish.  Mine was written from scratch (and actually 
worked) but I don't think that it ever left UNSW.

-- Dave

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
@ 2021-02-24  4:18 Rudi Blom
  0 siblings, 0 replies; 35+ messages in thread
From: Rudi Blom @ 2021-02-24  4:18 UTC (permalink / raw)
  To: tuhs

Some additions:

Systems with /bin a symlink to /usr/bin
Digital UNIX 4.0
Tru64 UNIX 5.0 to 5.1B
HP-UX 11i 11.23 and 11.31

Systems with separate /bin and /usr/bin
SCO UNIX 3.2 V4.0 to V4.2

-- 
The more I learn the better I understand I know nothing.

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-21 23:01   ` Steve Nickolas
@ 2021-02-23  3:31     ` Andrew Warkentin
  0 siblings, 0 replies; 35+ messages in thread
From: Andrew Warkentin @ 2021-02-23  3:31 UTC (permalink / raw)
  To: tuhs

On 2/21/21, Steve Nickolas <usotsuki@buric.co> wrote:
>
> While I've been stuck regarding bringing up a kernel, C compiler and libc
> all together, (keeping in mind my desire to avoid gcc and glibc for the
> project) the conceptual distribution I've been working on for some time
> uses more or less the same abstraction as the BSDs, with distinct /bin and
> /sbin vs. /usr/bin and /usr/sbin as I personally believe it should be,
> that the stuff in /bin should be enough to bring up and/or run diagnostics
> on a system, and everything else go in /usr.
>
I don't see much of a point in maintaining the separation these days.
/bin and /usr/bin were originally separated because it wasn't possible
to fit everything on one disk, and (AFAIK) the separation was mostly
maintained after that to reduce the chance of filesystem corruption
rendering the system unbootable (which is much less of a problem
nowadays because of journalled and log-structured filesystems).

Under UX/RT, the OS I'm writing, all commands (administrative or
otherwise) will appear to be in /bin, and all daemons will appear to
be in /sbin (with corresponding symlinks in /usr). The separation into
administrative and regular commands will be meaningless since the
traditional root/non-root security model will be completely eliminated
in favor of role-based access control. The / and /usr separation will
be useless since it will be impossible to have a separate /usr
partition (the contents of the root will be dynamically bound from a
collection of individual package directories, and won't correspond to
the root of the system volume).

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-23  2:50       ` Theodore Ts'o
@ 2021-02-23  3:19         ` Warner Losh
  0 siblings, 0 replies; 35+ messages in thread
From: Warner Losh @ 2021-02-23  3:19 UTC (permalink / raw)
  To: Theodore Ts'o; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 1101 bytes --]

On Mon, Feb 22, 2021, 7:50 PM Theodore Ts'o <tytso@mit.edu> wrote:

> On Mon, Feb 22, 2021 at 07:38:21PM -0500, Steve Nickolas wrote:
> > On Tue, 23 Feb 2021, Wesley Parish wrote:
> >
> > > I've just checked Slackware 14.* and it's still got a few binaries in
> > > /bin, unlike the RedHat* group which has indeed sent them all to
> > > /usr/bin. I don't know about the Debian* group, or if the Mandrake*
> > > group have gone with the RedHat* or not. Let alone all the other
> > > distros.
> >
> > Debian links /bin to /usr/bin.
>
> New installs of Debian will use a /usr merged configuration.  However,
> for pre-existing installations, we are not yet forcing, or even
> strongly recommending, system administrators to install the usrmerge
> package which will transition an legacy directory hierarchy to be /usr
> merged.  So at the moment, Debian packages need to support both merged
> and non-merged configurations, which is not ideal
>

I anticipate needing a /usr/bin/bash soon on my FreeBSD system for the same
reason I have a /bin/bash pointing at /usr/local/bin/bash. Progress :)

Warner

>

[-- Attachment #2: Type: text/html, Size: 1744 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-23  0:38     ` Steve Nickolas
@ 2021-02-23  2:50       ` Theodore Ts'o
  2021-02-23  3:19         ` Warner Losh
  0 siblings, 1 reply; 35+ messages in thread
From: Theodore Ts'o @ 2021-02-23  2:50 UTC (permalink / raw)
  To: Steve Nickolas; +Cc: The Eunuchs Hysterical Society

On Mon, Feb 22, 2021 at 07:38:21PM -0500, Steve Nickolas wrote:
> On Tue, 23 Feb 2021, Wesley Parish wrote:
> 
> > I've just checked Slackware 14.* and it's still got a few binaries in
> > /bin, unlike the RedHat* group which has indeed sent them all to
> > /usr/bin. I don't know about the Debian* group, or if the Mandrake*
> > group have gone with the RedHat* or not. Let alone all the other
> > distros.
> 
> Debian links /bin to /usr/bin.

New installs of Debian will use a /usr merged configuration.  However,
for pre-existing installations, we are not yet forcing, or even
strongly recommending, system administrators to install the usrmerge
package which will transition an legacy directory hierarchy to be /usr
merged.  So at the moment, Debian packages need to support both merged
and non-merged configurations, which is not ideal from a pacakge
maintainer's POV.

						- Ted

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-23  0:25   ` Wesley Parish
  2021-02-23  0:38     ` Steve Nickolas
@ 2021-02-23  1:47     ` Theodore Ts'o
  1 sibling, 0 replies; 35+ messages in thread
From: Theodore Ts'o @ 2021-02-23  1:47 UTC (permalink / raw)
  To: Wesley Parish; +Cc: The Eunuchs Hysterical Society

On Tue, Feb 23, 2021 at 01:25:51PM +1300, Wesley Parish wrote:
> I've just checked Slackware 14.* and it's still got a few binaries in
> /bin, unlike the RedHat* group which has indeed sent them all to
> /usr/bin. I don't know about the Debian* group, or if the Mandrake*
> group have gone with the RedHat* or not. Let alone all the other
> distros.

More information about the /usr migration, can be found at:

* https://wiki.debian.org/UsrMerge
* https://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge/

One of the interesting points made in the above is that merging /bin
and /usr/bin, et. al., was first done by Solaris 11 (ten years ago)
and so one of the arguments for Linux distributions for proceeding
with the /usr merge was to improve cross compatibility with legacy
commercial Unix systems.

So obviously, like so many other things, it's all Oracle's fault.  :-)

   	      	      	   	 	 - Ted

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-23  0:25   ` Wesley Parish
@ 2021-02-23  0:38     ` Steve Nickolas
  2021-02-23  2:50       ` Theodore Ts'o
  2021-02-23  1:47     ` Theodore Ts'o
  1 sibling, 1 reply; 35+ messages in thread
From: Steve Nickolas @ 2021-02-23  0:38 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On Tue, 23 Feb 2021, Wesley Parish wrote:

> I've just checked Slackware 14.* and it's still got a few binaries in
> /bin, unlike the RedHat* group which has indeed sent them all to
> /usr/bin. I don't know about the Debian* group, or if the Mandrake*
> group have gone with the RedHat* or not. Let alone all the other
> distros.

Debian links /bin to /usr/bin.

-uso.

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-21 22:40 ` Dave Horsfall
  2021-02-21 23:01   ` Steve Nickolas
  2021-02-22  0:13   ` Warren Toomey
@ 2021-02-23  0:25   ` Wesley Parish
  2021-02-23  0:38     ` Steve Nickolas
  2021-02-23  1:47     ` Theodore Ts'o
  2 siblings, 2 replies; 35+ messages in thread
From: Wesley Parish @ 2021-02-23  0:25 UTC (permalink / raw)
  To: Dave Horsfall; +Cc: The Eunuchs Hysterical Society

I've just checked Slackware 14.* and it's still got a few binaries in
/bin, unlike the RedHat* group which has indeed sent them all to
/usr/bin. I don't know about the Debian* group, or if the Mandrake*
group have gone with the RedHat* or not. Let alone all the other
distros.

Wesley Parish

On 2/22/21, Dave Horsfall <dave@horsfall.org> wrote:
> On Sat, 20 Feb 2021, M Douglas McIlroy wrote:
>
>>>  - separation of code and data using read-only and read/write file
>>>  systems
>>
>> I'll bite. How do you install code in a read-only file system? And where
>> does a.out go?
>
> I once worked for a place who reckoned that /bin and /lib etc ought to be
> in an EEPROM; I reckon that he was right (Penguin/OS dumps everything
> under /usr/bin, for example).
>
>> My guess is that /bin is in a file system of its own. Executables from
>> /letc and /lib are probably there too. On the other hand, I guess users'
>> personal code is still read/write.
>
> That's how we ran our RK-05 11/40s since Ed 5...  Good fun writing a DJ-11
> driver from the DH-11 source; even more fun when I wrote a UT-200 driver
> from the manual alone (I'm sure that "ei.c" is Out There Somewhere),
> junking IanJ's driver.
>
> The war stories that I could tell...
>
>> I agree that such an arrangement is prudent. I don't see a way, though,
>> to update bin without disrupting most running programs.
>
> Change is inevitable; the trick is to minimise the disruption.
>
> -- Dave, who carried RK-05s all over the UNSW campus
>

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-21 22:40 ` Dave Horsfall
  2021-02-21 23:01   ` Steve Nickolas
@ 2021-02-22  0:13   ` Warren Toomey
  2021-02-27  2:47     ` Dave Horsfall
  2021-02-23  0:25   ` Wesley Parish
  2 siblings, 1 reply; 35+ messages in thread
From: Warren Toomey @ 2021-02-22  0:13 UTC (permalink / raw)
  To: Dave Horsfall; +Cc: The Eunuchs Hysterical Society

On Mon, Feb 22, 2021 at 09:40:57AM +1100, Dave Horsfall wrote:
> That's how we ran our RK-05 11/40s since Ed 5...  Good fun writing a DJ-11
> driver from the DH-11 source; even more fun when I wrote a UT-200 driver
> from the manual alone (I'm sure that "ei.c" is Out There Somewhere), junking
> IanJ's driver.

https://minnie.tuhs.org/cgi-bin/utree.pl?file=AUSAM/sys/dmr/ei.c

Cheers, Warren

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-21 22:40 ` Dave Horsfall
@ 2021-02-21 23:01   ` Steve Nickolas
  2021-02-23  3:31     ` Andrew Warkentin
  2021-02-22  0:13   ` Warren Toomey
  2021-02-23  0:25   ` Wesley Parish
  2 siblings, 1 reply; 35+ messages in thread
From: Steve Nickolas @ 2021-02-21 23:01 UTC (permalink / raw)
  To: Dave Horsfall; +Cc: The Eunuchs Hysterical Society

On Mon, 22 Feb 2021, Dave Horsfall wrote:

> I once worked for a place who reckoned that /bin and /lib etc ought to be in 
> an EEPROM; I reckon that he was right (Penguin/OS dumps everything under 
> /usr/bin, for example).

I have used distributions in the past that maintained the traditional 
distinction.

While I've been stuck regarding bringing up a kernel, C compiler and libc 
all together, (keeping in mind my desire to avoid gcc and glibc for the 
project) the conceptual distribution I've been working on for some time 
uses more or less the same abstraction as the BSDs, with distinct /bin and 
/sbin vs. /usr/bin and /usr/sbin as I personally believe it should be, 
that the stuff in /bin should be enough to bring up and/or run diagnostics 
on a system, and everything else go in /usr.

-uso.

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-20 23:09 M Douglas McIlroy
                   ` (2 preceding siblings ...)
  2021-02-21 22:40 ` Dave Horsfall
@ 2021-02-21 22:54 ` Clem Cole
  3 siblings, 0 replies; 35+ messages in thread
From: Clem Cole @ 2021-02-21 22:54 UTC (permalink / raw)
  To: M Douglas McIlroy; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 819 bytes --]

On Sat, Feb 20, 2021 at 6:10 PM M Douglas McIlroy <
m.douglas.mcilroy@dartmouth.edu> wrote:

> >  - separation of code and data using read-only and read/write file
> systems
>
> I'll bite. How do you install code in a read-only file system? And
> where does a.out go?
>
The best way I have seen this done is with overlay and union file system
support.   The 'writeable' versions are the file in /bin are overlayed as
needed.   To do this properly you need the stackable file system stuff we
worked on at LCC and Sun.  If you can interpose at the inode level it's
very cool and flexible (Sun played with - but makes the Sun symlink
nightmare seem like an easy night at the movies), at the filesystem switch
layer (Locus and UCLA - scheme that was in BSD at one point - easier to
manage/admin).


ᐧ

[-- Attachment #2: Type: text/html, Size: 1892 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-20 23:09 M Douglas McIlroy
  2021-02-21  8:15 ` Otto Moerbeek
  2021-02-21 11:08 ` Rich Morin
@ 2021-02-21 22:40 ` Dave Horsfall
  2021-02-21 23:01   ` Steve Nickolas
                     ` (2 more replies)
  2021-02-21 22:54 ` Clem Cole
  3 siblings, 3 replies; 35+ messages in thread
From: Dave Horsfall @ 2021-02-21 22:40 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On Sat, 20 Feb 2021, M Douglas McIlroy wrote:

>>  - separation of code and data using read-only and read/write file
>>  systems
>
> I'll bite. How do you install code in a read-only file system? And where 
> does a.out go?

I once worked for a place who reckoned that /bin and /lib etc ought to be 
in an EEPROM; I reckon that he was right (Penguin/OS dumps everything 
under /usr/bin, for example).

> My guess is that /bin is in a file system of its own. Executables from 
> /letc and /lib are probably there too. On the other hand, I guess users' 
> personal code is still read/write.

That's how we ran our RK-05 11/40s since Ed 5...  Good fun writing a DJ-11 
driver from the DH-11 source; even more fun when I wrote a UT-200 driver 
from the manual alone (I'm sure that "ei.c" is Out There Somewhere), 
junking IanJ's driver.

The war stories that I could tell...

> I agree that such an arrangement is prudent. I don't see a way, though, 
> to update bin without disrupting most running programs.

Change is inevitable; the trick is to minimise the disruption.

-- Dave, who carried RK-05s all over the UNSW campus

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-20 23:09 M Douglas McIlroy
  2021-02-21  8:15 ` Otto Moerbeek
@ 2021-02-21 11:08 ` Rich Morin
  2021-02-21 22:40 ` Dave Horsfall
  2021-02-21 22:54 ` Clem Cole
  3 siblings, 0 replies; 35+ messages in thread
From: Rich Morin @ 2021-02-21 11:08 UTC (permalink / raw)
  To: TUHS main list

> On Feb 20, 2021, at 15:09, M Douglas McIlroy <m.douglas.mcilroy@dartmouth.edu> wrote:
> 
>> - separation of code and data using read-only and read/write file systems
> 
> I'll bite.  How do you install code in a read-only file system?

Disclaimer: I haven't actually used Nerves myself, just watched some presentations, read various web pages, etc.  So anything I say about it is quite unreliable.  And, although that item was (sort of) true, it was obviously rather misleading if interpreted too broadly.  So, I'll try to provide some context to explain what I meant by it.

As I understand it, Nerves is intended as a build and delivery mechanism for IoT system software.  It's supposed to be possible to upgrade a deployed device without blowing away its persistent saved state.  And, if the upgrade fails, to back down to the previous version.  Also, the running code on the device should not be able to trash the system software.

To support this, they use multiple file systems, with various updating attributes.  For example, they might have two file systems for the system software and a third one for the persistent saved state.  This lets a developer upload and boot a new copy of the system software, but fall back to the old version if something goes wrong.

-r


^ permalink raw reply	[flat|nested] 35+ messages in thread

* [TUHS] Abstractions
@ 2021-02-21 10:47 Paul Ruizendaal
  0 siblings, 0 replies; 35+ messages in thread
From: Paul Ruizendaal @ 2021-02-21 10:47 UTC (permalink / raw)
  To: TUHS main list

To quote from Jon’s post:

> There have been heated discussions on this list about kernel API bloat.  In my
> opinion, these discussions have mainly been people grumbling about what they
> don't like.  I'd like to flip the discussion around to what we would like.


> Ken and Dennis did a great job with initial abstractions.  Some on this list
> have claimed that these abstractions weren't sufficient for modern times.
> Now that we have new information from modern use cases, how would we rethink
> the basic abstractions?

I’d like to add the constraint of things that would have been implementable
on the hardware of the late 1970’s, let’s say a PDP11/70 with Datakit or
3Mbps Ethernet or Arpanet; maybe also Apple 2 class bitmap graphics.

And quote some other posts:

> Because it's easy pickings, I would claim that the socket system call is out
> of line with the UNIX abstractions; it exists because of practical political
> considerations, not because it's needed.  I think that it would have fit
> better folded into the open system call.

>> 
>> Somebody once suggested a filesystem interface (it certainly fits the Unix 
>> philosophy); I don't recall the exact details.
> 
> And it was done, over 30 years ago; see Plan 9 from Bell Labs....

I would argue that quite a bit of that was implementable as early as 6th
Edition. I was researching that very topic last Spring [1] and back ported
Peter Weinberger’s File System Switch (FSS) from 8th to 6th Edition; the
switch itself bloats the kernel by about half a kilobyte. I think it may be
one of the few imaginable extensions that do not dilute the incredible
bang/buck ratio of the V6 kernel.

With that change in place a lot of other things become possible:
- a Kilian style procfs
- a Weinberger style network FS
- a text/file based ioctl
- a clean approach to named pipes
- a different starting point to sockets

Each of these would add to kernel size of course, hence I’m thinking about
a split I/D kernel.

To some extent it is surprising that the FSS did not happen around 1975, as
many ideas around it were 'in the air' at the time (Heinz Lycklama’s peripheral
Unix, the Spider network Filestore, Rand ports, Arpanet Unix, etc). With the
benefit of hindsight, it isn’t a great code leap from the cdev switch to the
FSS - but probably the ex ante conceptual leap was just too big at the time.

Paul

[1] Code diffs here:
https://1587660.websites.xs4all.nl/cgi-bin/9995/vdiff?from=fab15b88a6a0f36bdb41f24f0b828a67c5f9fe03&to=b95342aaa826bb3c422963108c76d09969b1de93&sbs=1



^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-20 23:09 M Douglas McIlroy
@ 2021-02-21  8:15 ` Otto Moerbeek
  2021-02-21 11:08 ` Rich Morin
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 35+ messages in thread
From: Otto Moerbeek @ 2021-02-21  8:15 UTC (permalink / raw)
  To: M Douglas McIlroy; +Cc: tuhs

On Sat, Feb 20, 2021 at 06:09:42PM -0500, M Douglas McIlroy wrote:

> >  - separation of code and data using read-only and read/write file systems
> 
> I'll bite. How do you install code in a read-only file system? And
> where does a.out go?
> 
> My guess is that /bin is in a file system of its own. Executables from
>  /letc and /lib are probably there too. On the other hand, I guess
> users' personal code is still read/write.
> 
> I agree that such an arrangement is prudent. I don't see a way,
> though, to update bin without disrupting most running programs.
> 
> Doug

I always wonder how to distunguish data and programs when people want
to separate them. One person's data is another person's program and
vice versa. Think scripting, config files, grammar definitions,
postscript files, exectuables to be fed to emulators, compilers,
linkers, code analysis tools, the examples are endless.

Turing already saw that form the theoretical point of view, others
(like Von Neumann) more from the practical persppective.

Data = Programs.

	-Otto

^ permalink raw reply	[flat|nested] 35+ messages in thread

* [TUHS] Abstractions
@ 2021-02-20 23:09 M Douglas McIlroy
  2021-02-21  8:15 ` Otto Moerbeek
                   ` (3 more replies)
  0 siblings, 4 replies; 35+ messages in thread
From: M Douglas McIlroy @ 2021-02-20 23:09 UTC (permalink / raw)
  To: tuhs

>  - separation of code and data using read-only and read/write file systems

I'll bite. How do you install code in a read-only file system? And
where does a.out go?

My guess is that /bin is in a file system of its own. Executables from
 /letc and /lib are probably there too. On the other hand, I guess
users' personal code is still read/write.

I agree that such an arrangement is prudent. I don't see a way,
though, to update bin without disrupting most running programs.

Doug

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-15 19:56 Jon Steinhart
                   ` (3 preceding siblings ...)
  2021-02-16 22:46 ` Steffen Nurpmeso
@ 2021-02-17 12:09 ` David Arnold
  4 siblings, 0 replies; 35+ messages in thread
From: David Arnold @ 2021-02-17 12:09 UTC (permalink / raw)
  To: Jon Steinhart; +Cc: TUHS main list

> On 16 Feb 2021, at 06:56, Jon Steinhart <jon@fourwinds.com> wrote:
> 
> Was thinking about our recent discussion about system call bloat and such.
> Seemed to me that there was some argument that it was needed in order to
> support modern needs.  As I tried to say, I think that a good part of the
> bloat stemmed from we-need-to-add-this-to-support-that thinking instead
> of what's-the-best-way-to-extend-the-system-to-support-this-need thinking.
> 
> So if y'all are up for it, I'd like to have a discussion on what abstractions
> would be appropriate in order to meet modern needs.  Any takers?

Plan9 showed that it’s possible to evolve the Unix model to encompass new needs without compromising the abstraction, although to be fair, it basically addressed only the first 15-20 years of changes since V7.  Freedom to break backward compatibility is obviously a key enabler, and difficult to manage for a commercial system. 

Despite its various issues, I think the Mach abstractions also stand up well as an insightful effort for their time.

One area that has continued to evolve in Unix, with a trail of (mostly) still-supported-but-no-longer-recommended APIs, is asynchronous event handling.  mpx, select, poll, kevents, AIO, /dev/poll, epoll, port_create, inotify, dnotify, FEN, etc.  What a mess!

Containers, jails, zones, namespaces, etc, is another area with diverse solutions, none of which have been sufficiently the Right Thing to be adopted by everyone else.

For today’s uses and hardware, the Unix API does too much: rich, stateful APIs copying everything from userland to kernel and back again — the context switching and data copying time is prohibitive, and so the kernel ends up being bypassed once it’s checked the permissions and allocated the hardware resources.  I hesitate to call it a micro-kernel model, but the kernel is used less, and libraries and services take on more of the work.





d


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-16 19:59   ` Jon Steinhart
  2021-02-17  4:01     ` Theodore Ts'o
@ 2021-02-17  6:50     ` Chris Hanson
  1 sibling, 0 replies; 35+ messages in thread
From: Chris Hanson @ 2021-02-17  6:50 UTC (permalink / raw)
  To: Jon Steinhart; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 1626 bytes --]

On Feb 16, 2021, at 11:59 AM, Jon Steinhart <jon@fourwinds.com> wrote:
> 
> 	The original Apple Macintosh API was published in 1985 in a three-­
> 	volume set of books called Inside Macintosh (Addison-­Wesley). The
> 	set was over 1,200 pages long. It’s completely obsolete; modern
> 	(UNIX-based) Macs don’t use any of it. Why didn’t this API
> 	design last?

I think this is a little bit of a red herring; most of the original Macintosh Toolbox APIs would not be considered "system calls" then or now. The Macintosh Operating System APIs were a much more tightly-scoped set on top of which was the Toolbox.

For example, in the original filesystem and device driver interfaces, you had _PBOpen, _PBClose, _PBRead, _PBWrite, and _PBControl. Sound familiar? One major difference is that these took a struct full of arguments (a parameter block in Macintosh API terminology) and could be used either synchronously or asynchronously with a callback, unlike the core UNIX filesystem calls.

A more oranges-to-oranges comparison would be to look at the Macintosh Operating System and Toolbox API surface compared with, say, the SunOS and SunWindows API surface…

And then, of course, there's the question of how long the design lasted: The Carbon API set is a direct descendant of the original Macintosh Operating System and Toolbox API set, and was supported for the entire lifetime of 32-bit executables on the Mac. I ported plenty of OS & Toolbox code to Carbon and it was mostly a matter of updating UI metrics and replacing direct structure accesses with equivalent function calls.

  -- Chris



[-- Attachment #2: Type: text/html, Size: 7762 bytes --]

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-16 19:59   ` Jon Steinhart
@ 2021-02-17  4:01     ` Theodore Ts'o
  2021-02-17  6:50     ` Chris Hanson
  1 sibling, 0 replies; 35+ messages in thread
From: Theodore Ts'o @ 2021-02-17  4:01 UTC (permalink / raw)
  To: Jon Steinhart; +Cc: TUHS main list

It's always useful to talk about requirements as the first part of the
design process.

At the high level, how important is backwards compatibility?  Is the
problem of how support existing application in scope, or not?  Or is
the assumption that emulation libraries will always be sufficient.

How about performance, either of applications using the new API, or
applcications using the legacy API's?  And what are the hardware
platforms that this new set of abstractions going to target?  Is the
goal only to target small embedded systems?  Mobile handsets?  Desktop
systems?  Is it supposed to be able to scale to super computers?  Are
web front-ends that need to be able to accept thousands of incoming
TCP connections per second, and then redirect those connections to
application logic servers in scope?

Solutions that involve being able to support intpret general SQL
queries may not scale in terms of performance and the ability to
support thousands of file descriptors in a single process.

Backwards compatibility is why we have multiple asynchronous I/O
interfaces --- from select, poll, epoll, kqueue, and io_uring.  And
the reason why we've had multiple asynchronus I/O interfaces over the
decades is because the performance requirements have changed, and the
capability of hardware interfaces for high performance I/O has
changed; it's no longer about I/O ports and interrupts, but instead,
having multiple request and response queues through memory mapped I/O,
and the need to be able to use multiple CPU's and multiplexing
multiple network or storage transactions across a single doorbell or
system call.

If all of this is out of scope, then the design process will be much
simpler, and perhaps more elegant; but the resulting design will not
be useful for many of the use cases where Linux is used today.  And
perhaps that's OK.  On the other hand, one person's simple, elegant
design is another person's toy that isn't fit for their purpose.

IBM once said that part of Linux's power is that it scales from wrist
watches to super computers.  Is that in scope for this theoretical
design question?

						- Ted

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-15 19:56 Jon Steinhart
                   ` (2 preceding siblings ...)
  2021-02-16 12:26 ` Rich Morin
@ 2021-02-16 22:46 ` Steffen Nurpmeso
  2021-02-17 12:09 ` David Arnold
  4 siblings, 0 replies; 35+ messages in thread
From: Steffen Nurpmeso @ 2021-02-16 22:46 UTC (permalink / raw)
  To: Jon Steinhart; +Cc: TUHS main list

Jon Steinhart wrote in
 <202102151956.11FJuRIh3079869@darkstar.fourwinds.com>:
 |Was thinking about our recent discussion about system call bloat and such.
 |Seemed to me that there was some argument that it was needed in order to
 |support modern needs.  As I tried to say, I think that a good part of the
 |bloat stemmed from we-need-to-add-this-to-support-that thinking instead
 |of what's-the-best-way-to-extend-the-system-to-support-this-need thinking.
 |
 |So if y'all are up for it, I'd like to have a discussion on what abstrac\
 |tions
 |would be appropriate in order to meet modern needs.  Any takers?

Proper program exit integer status codes.
Now that "set -o pipefail" is a standardized feature of POSIX
shells all that is needed are programs which properly handle
errors and also report that to the outside.  This is very hard,
especially when put over existing codebases.

But also new code.  For example i use BTRFS (with a long term
perspective to switch to ZFS, because of restartable snapshot
sends, and also because of ZFS encrypted partitions to replace my
several encfs-encrypted on-demand storages, these now can even be
shared in between FreeBSD and Linux), (i use it at all because it
ships with the Linux kernel, can be compiled-in, is
copyright-compatible, that is i wanted to test that coming from
over two decades of ext2/3/4 on Linux and of course the default of
FreeBSD, and i really drive the entire thing with subvolumes, only
the EFI boot partition is truly separate), anyhow, receiving
snapshots can fail but the snapshot counts as having been properly
received, and no exit status whatsoever will report the failure.
(At least in my practical experiences.)
Easy scriptability with proper (also meaning automatically
interpretable) error reports.

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-16  8:15 ` Tom Ivar Helbekkmo via TUHS
  2021-02-16 10:04   ` Wesley Parish
@ 2021-02-16 19:59   ` Jon Steinhart
  2021-02-17  4:01     ` Theodore Ts'o
  2021-02-17  6:50     ` Chris Hanson
  1 sibling, 2 replies; 35+ messages in thread
From: Jon Steinhart @ 2021-02-16 19:59 UTC (permalink / raw)
  To: TUHS main list

Tom Ivar Helbekkmo writes:
> Jon Steinhart <jon@fourwinds.com> writes:
>
> > So if y'all are up for it, I'd like to have a discussion on what
> > abstractions would be appropriate in order to meet modern needs.  Any
> > takers?
>
> A late friend of mine felt strongly that Unix needed an SQL interface to
> the kernel.  With all information and configuration in a well designed
> schema, system administration could be greatly enhanced, he felt, and
> could have standard interaction patterns across components -- instead of
> all the quirky command line interfaces we have today, and their user
> oriented output formats that you need to parse to use the data.
>
> sysctl done right, so to speak.

OK, that's interesting and makes my brain a bit crazy.  Are we talking

 select file_descriptor from file_table where file_name='foo' && flags='O_EXCL';
 delete from process_table where process_id=pid;

and so on?  Lots of possibilities for weird joins.

But, this wasn't exactly what I was looking for in my original post which was
maybe too terse.

There have been heated discussions on this list about kernel API bloat.  In my
opinion, these discussions have mainly been people grumbling about what they
don't like.  I'd like to flip the discussion around to what we would like.

Ken and Dennis did a great job with initial abstractions.  Some on this list
have claimed that these abstractions weren't sufficient for modern times.
Now that we have new information from modern use cases, how would we rethink
the basic abstractions?

Quoting from something that I wrote a few years ago:

	The original Apple Macintosh API was published in 1985 in a three-­
	volume set of books called Inside Macintosh (Addison-­Wesley). The
	set was over 1,200 pages long. It’s completely obsolete; modern
	(UNIX-based) Macs don’t use any of it. Why didn’t this API
	design last?

	By contrast, version 6 of the UNIX operating system was released 10
	years earlier in 1975, with a 321-page manual. It embodied a completely
	different approach that sported a narrow and deep API.

	Both the UNIX API and a large number of the original applications are
	still in widespread use today, more than 40 years later, which is a
	testament to the quality of the design. Not only that, but a large
	number of the libraries are still in use and essentially unchanged,
	though their functionality has been copied into many other systems.

While I don't have a count of the number of entries in the original Mac API,
I'm guessing that number of Linux system calls is getting closer to that
number.

Is there any way that the abstractions can be rethought to get us back to an
API that more concise and flexible?  By flexible I mean the ability to support
new functionality without adding more system calls?

While the SQL interface notion is interesting, to me it's more in line with
using a different language to access the API.  But it would be interesting to
see it fleshed out because maybe the abstractions provided by various tables
would be different.

Because it's easy pickings, I would claim that the socket system call is out
of line with the UNIX abstractions; it exists because of practical political
considerations, not because it's needed.  I think that it would have fit
better folded into the open system call.

Something else added along with the networking was readv/writev.  In this case,
I would claim that those are the correct modern abstraction and that read/write
are a subset.

Hope that clarifies the discussion that I'm trying to kick off.

Jon

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-15 19:56 Jon Steinhart
  2021-02-15 21:52 ` Dave Horsfall
  2021-02-16  8:15 ` Tom Ivar Helbekkmo via TUHS
@ 2021-02-16 12:26 ` Rich Morin
  2021-02-16 22:46 ` Steffen Nurpmeso
  2021-02-17 12:09 ` David Arnold
  4 siblings, 0 replies; 35+ messages in thread
From: Rich Morin @ 2021-02-16 12:26 UTC (permalink / raw)
  To: TUHS main list

> On Feb 15, 2021, at 11:56, Jon Steinhart <jon@fourwinds.com> wrote:
> 
> Was thinking about our recent discussion about system call bloat and such.
> Seemed to me that there was some argument that it was needed in order to
> support modern needs.  As I tried to say, I think that a good part of the
> bloat stemmed from we-need-to-add-this-to-support-that thinking instead
> of what's-the-best-way-to-extend-the-system-to-support-this-need thinking.
> 
> So if y'all are up for it, I'd like to have a discussion on what abstractions
> would be appropriate in order to meet modern needs.  Any takers?

The folks behind the Nerves Project (https://www.nerves-project.org) have done some serious thinking about this question, albeit mostly confined to the IoT space.  They have also written (and distribute) some nifty implementation code.

I won't try to cover all of their work here, but some high points include:

- automated build and cross-compilation of entire Linux-based systems
- automated distribution of (and fallbacks for) updated system code
- separation of code and data using read-only and read/write file systems
- support for multiple target platforms (e.g., processors, boards)

- Erlang-style supervision trees (via Elixir) for critical services, etc.
- extremely rapid boot times for the resulting (Linux-based) systems

For more information, check out their web site, watch some presentations,
and/or (gasp!) try out the code...

-r
   

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-16  8:15 ` Tom Ivar Helbekkmo via TUHS
@ 2021-02-16 10:04   ` Wesley Parish
  2021-02-16 19:59   ` Jon Steinhart
  1 sibling, 0 replies; 35+ messages in thread
From: Wesley Parish @ 2021-02-16 10:04 UTC (permalink / raw)
  To: Tom Ivar Helbekkmo, Jon Steinhart; +Cc: TUHS main list

Now that is an interesting idea. Did he ever get around to developing 
it? Any documents? Any experimental results? (Mind you, he'd've run into 
CJ Date's reservations on the incompleteness of SQL as a language stuck 
between relational algebra and relational calculus ... :) )

Wesley Parish

On 16/02/21 9:15 pm, Tom Ivar Helbekkmo via TUHS wrote:
> Jon Steinhart <jon@fourwinds.com> writes:
>
>> So if y'all are up for it, I'd like to have a discussion on what
>> abstractions would be appropriate in order to meet modern needs.  Any
>> takers?
> A late friend of mine felt strongly that Unix needed an SQL interface to
> the kernel.  With all information and configuration in a well designed
> schema, system administration could be greatly enhanced, he felt, and
> could have standard interaction patterns across components -- instead of
> all the quirky command line interfaces we have today, and their user
> oriented output formats that you need to parse to use the data.
>
> sysctl done right, so to speak.
>
> -tih

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-15 19:56 Jon Steinhart
  2021-02-15 21:52 ` Dave Horsfall
@ 2021-02-16  8:15 ` Tom Ivar Helbekkmo via TUHS
  2021-02-16 10:04   ` Wesley Parish
  2021-02-16 19:59   ` Jon Steinhart
  2021-02-16 12:26 ` Rich Morin
                   ` (2 subsequent siblings)
  4 siblings, 2 replies; 35+ messages in thread
From: Tom Ivar Helbekkmo via TUHS @ 2021-02-16  8:15 UTC (permalink / raw)
  To: Jon Steinhart; +Cc: TUHS main list

Jon Steinhart <jon@fourwinds.com> writes:

> So if y'all are up for it, I'd like to have a discussion on what
> abstractions would be appropriate in order to meet modern needs.  Any
> takers?

A late friend of mine felt strongly that Unix needed an SQL interface to
the kernel.  With all information and configuration in a well designed
schema, system administration could be greatly enhanced, he felt, and
could have standard interaction patterns across components -- instead of
all the quirky command line interfaces we have today, and their user
oriented output formats that you need to parse to use the data.

sysctl done right, so to speak.

-tih
-- 
Most people who graduate with CS degrees don't understand the significance
of Lisp.  Lisp is the most important idea in computer science.  --Alan Kay

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-15 21:52 ` Dave Horsfall
@ 2021-02-16  7:13   ` arnold
  0 siblings, 0 replies; 35+ messages in thread
From: arnold @ 2021-02-16  7:13 UTC (permalink / raw)
  To: tuhs, dave

Dave Horsfall <dave@horsfall.org> wrote:

> On Mon, 15 Feb 2021, Jon Steinhart wrote:
>
> [...]
>
> > So if y'all are up for it, I'd like to have a discussion on what 
> > abstractions would be appropriate in order to meet modern needs.  Any 
> > takers?
>
> Somebody once suggested a filesystem interface (it certainly fits the Unix 
> philosophy); I don't recall the exact details.
>
> -- Dave

And it was done, over 30 years ago; see Plan 9 from Bell Labs....

Arnold

^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: [TUHS] Abstractions
  2021-02-15 19:56 Jon Steinhart
@ 2021-02-15 21:52 ` Dave Horsfall
  2021-02-16  7:13   ` arnold
  2021-02-16  8:15 ` Tom Ivar Helbekkmo via TUHS
                   ` (3 subsequent siblings)
  4 siblings, 1 reply; 35+ messages in thread
From: Dave Horsfall @ 2021-02-15 21:52 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On Mon, 15 Feb 2021, Jon Steinhart wrote:

[...]

> So if y'all are up for it, I'd like to have a discussion on what 
> abstractions would be appropriate in order to meet modern needs.  Any 
> takers?

Somebody once suggested a filesystem interface (it certainly fits the Unix 
philosophy); I don't recall the exact details.

-- Dave

^ permalink raw reply	[flat|nested] 35+ messages in thread

* [TUHS] Abstractions
@ 2021-02-15 19:56 Jon Steinhart
  2021-02-15 21:52 ` Dave Horsfall
                   ` (4 more replies)
  0 siblings, 5 replies; 35+ messages in thread
From: Jon Steinhart @ 2021-02-15 19:56 UTC (permalink / raw)
  To: TUHS main list

Was thinking about our recent discussion about system call bloat and such.
Seemed to me that there was some argument that it was needed in order to
support modern needs.  As I tried to say, I think that a good part of the
bloat stemmed from we-need-to-add-this-to-support-that thinking instead
of what's-the-best-way-to-extend-the-system-to-support-this-need thinking.

So if y'all are up for it, I'd like to have a discussion on what abstractions
would be appropriate in order to meet modern needs.  Any takers?

Jon

^ permalink raw reply	[flat|nested] 35+ messages in thread

end of thread, other threads:[~2021-02-27  2:48 UTC | newest]

Thread overview: 35+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-02-23 19:37 [TUHS] Abstractions Nelson H. F. Beebe
2021-02-23 21:02 ` Charles H. Sauer
2021-02-23 21:15   ` Henry Bent
2021-02-24  2:47     ` Greg A. Woods
2021-02-24  3:20       ` Warner Losh
2021-02-24 20:05         ` Greg A. Woods
2021-02-24  1:51 ` Greg A. Woods
2021-02-24  2:23   ` Nelson H. F. Beebe
  -- strict thread matches above, loose matches on Subject: below --
2021-02-24  4:18 Rudi Blom
2021-02-21 10:47 Paul Ruizendaal
2021-02-20 23:09 M Douglas McIlroy
2021-02-21  8:15 ` Otto Moerbeek
2021-02-21 11:08 ` Rich Morin
2021-02-21 22:40 ` Dave Horsfall
2021-02-21 23:01   ` Steve Nickolas
2021-02-23  3:31     ` Andrew Warkentin
2021-02-22  0:13   ` Warren Toomey
2021-02-27  2:47     ` Dave Horsfall
2021-02-23  0:25   ` Wesley Parish
2021-02-23  0:38     ` Steve Nickolas
2021-02-23  2:50       ` Theodore Ts'o
2021-02-23  3:19         ` Warner Losh
2021-02-23  1:47     ` Theodore Ts'o
2021-02-21 22:54 ` Clem Cole
2021-02-15 19:56 Jon Steinhart
2021-02-15 21:52 ` Dave Horsfall
2021-02-16  7:13   ` arnold
2021-02-16  8:15 ` Tom Ivar Helbekkmo via TUHS
2021-02-16 10:04   ` Wesley Parish
2021-02-16 19:59   ` Jon Steinhart
2021-02-17  4:01     ` Theodore Ts'o
2021-02-17  6:50     ` Chris Hanson
2021-02-16 12:26 ` Rich Morin
2021-02-16 22:46 ` Steffen Nurpmeso
2021-02-17 12:09 ` David Arnold

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).