The Unix Heritage Society mailing list
 help / color / mirror / Atom feed
* [TUHS] [tuhs] The Unix shell: a 50-year view
@ 2021-07-02 21:24 Nelson H. F. Beebe
  2021-07-02 21:36 ` Larry McVoy
                   ` (2 more replies)
  0 siblings, 3 replies; 109+ messages in thread
From: Nelson H. F. Beebe @ 2021-07-02 21:24 UTC (permalink / raw)
  To: The Unix Heritage Society mailing list

In this week's BSDNow.tv podcast, available at

        https://www.bsdnow.tv/409

there is a story about a new conference paper on the Unix shell.  The
paper is available at

	Unix shell programming: the next 50 years
	HotOS '21: Workshop on Hot Topics in Operating Systems, Ann
	Arbor, Michigan, 1 June, 2021--3 June, 2021
	https://doi.org/10.1145/3458336.3465294

The tone is overall negative, though they do say nice things about
Doug McIlroy and Steve Johnson, and they offer ideas about
improvements.

List readers will have their own views of the paper.  My own is that,
despite its dark corners, the Bourne shell has served us
extraordinarily well, and I have been writing in it daily for decades
without being particularly bothered by the many issues raised by the
paper's authors.  Having dealt with so-called command shells on
numerous other operating systems, at least the Unix shells rarely get
in my way.

-------------------------------------------------------------------------------
- Nelson H. F. Beebe                    Tel: +1 801 581 5254                  -
- University of Utah                    FAX: +1 801 581 4148                  -
- Department of Mathematics, 110 LCB    Internet e-mail: beebe@math.utah.edu  -
- 155 S 1400 E RM 233                       beebe@acm.org  beebe@computer.org -
- Salt Lake City, UT 84112-0090, USA    URL: http://www.math.utah.edu/~beebe/ -
-------------------------------------------------------------------------------

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-02 21:24 [TUHS] [tuhs] The Unix shell: a 50-year view Nelson H. F. Beebe
@ 2021-07-02 21:36 ` Larry McVoy
  2021-07-02 21:56   ` Henry Bent
                     ` (2 more replies)
  2021-07-02 22:27 ` [TUHS] [tuhs] The Unix shell: a 50-year view Chet Ramey
  2021-07-02 23:09 ` Steve Nickolas
  2 siblings, 3 replies; 109+ messages in thread
From: Larry McVoy @ 2021-07-02 21:36 UTC (permalink / raw)
  To: Nelson H. F. Beebe; +Cc: The Unix Heritage Society mailing list

As I started reading it I found plenty to disagree with in the first few
paragraphs but they completely lost me at "After all, moving from System
V init scripts to systemd has arguably improvedthe Linux boot sequence."

Um, no, just, no.


On Fri, Jul 02, 2021 at 03:24:20PM -0600, Nelson H. F. Beebe wrote:
> In this week's BSDNow.tv podcast, available at
> 
>         https://www.bsdnow.tv/409
> 
> there is a story about a new conference paper on the Unix shell.  The
> paper is available at
> 
> 	Unix shell programming: the next 50 years
> 	HotOS '21: Workshop on Hot Topics in Operating Systems, Ann
> 	Arbor, Michigan, 1 June, 2021--3 June, 2021
> 	https://doi.org/10.1145/3458336.3465294
> 
> The tone is overall negative, though they do say nice things about
> Doug McIlroy and Steve Johnson, and they offer ideas about
> improvements.
> 
> List readers will have their own views of the paper.  My own is that,
> despite its dark corners, the Bourne shell has served us
> extraordinarily well, and I have been writing in it daily for decades
> without being particularly bothered by the many issues raised by the
> paper's authors.  Having dealt with so-called command shells on
> numerous other operating systems, at least the Unix shells rarely get
> in my way.
> 
> -------------------------------------------------------------------------------
> - Nelson H. F. Beebe                    Tel: +1 801 581 5254                  -
> - University of Utah                    FAX: +1 801 581 4148                  -
> - Department of Mathematics, 110 LCB    Internet e-mail: beebe@math.utah.edu  -
> - 155 S 1400 E RM 233                       beebe@acm.org  beebe@computer.org -
> - Salt Lake City, UT 84112-0090, USA    URL: http://www.math.utah.edu/~beebe/ -
> -------------------------------------------------------------------------------

-- 
---
Larry McVoy            	     lm at mcvoy.com             http://www.mcvoy.com/lm 

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-02 21:36 ` Larry McVoy
@ 2021-07-02 21:56   ` Henry Bent
  2021-07-02 23:12     ` Steve Nickolas
  2021-07-04 18:17     ` John Dow via TUHS
  2021-07-03  0:09   ` Andrew Warkentin
  2021-07-03 15:49   ` Andy Kosela
  2 siblings, 2 replies; 109+ messages in thread
From: Henry Bent @ 2021-07-02 21:56 UTC (permalink / raw)
  To: Larry McVoy; +Cc: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 2284 bytes --]

After rattling around in the back of a train car with nothing but my
thoughts, I emerged and said:

"systemd is a pox on our way of life"

and then promptly rolled over and went back to sleep.

-Henry

On Fri, 2 Jul 2021 at 17:38, Larry McVoy <lm@mcvoy.com> wrote:

> As I started reading it I found plenty to disagree with in the first few
> paragraphs but they completely lost me at "After all, moving from System
> V init scripts to systemd has arguably improvedthe Linux boot sequence."
>
> Um, no, just, no.
>
>
> On Fri, Jul 02, 2021 at 03:24:20PM -0600, Nelson H. F. Beebe wrote:
> > In this week's BSDNow.tv podcast, available at
> >
> >         https://www.bsdnow.tv/409
> >
> > there is a story about a new conference paper on the Unix shell.  The
> > paper is available at
> >
> >       Unix shell programming: the next 50 years
> >       HotOS '21: Workshop on Hot Topics in Operating Systems, Ann
> >       Arbor, Michigan, 1 June, 2021--3 June, 2021
> >       https://doi.org/10.1145/3458336.3465294
> >
> > The tone is overall negative, though they do say nice things about
> > Doug McIlroy and Steve Johnson, and they offer ideas about
> > improvements.
> >
> > List readers will have their own views of the paper.  My own is that,
> > despite its dark corners, the Bourne shell has served us
> > extraordinarily well, and I have been writing in it daily for decades
> > without being particularly bothered by the many issues raised by the
> > paper's authors.  Having dealt with so-called command shells on
> > numerous other operating systems, at least the Unix shells rarely get
> > in my way.
> >
> >
> -------------------------------------------------------------------------------
> > - Nelson H. F. Beebe                    Tel: +1 801 581 5254
>       -
> > - University of Utah                    FAX: +1 801 581 4148
>       -
> > - Department of Mathematics, 110 LCB    Internet e-mail:
> beebe@math.utah.edu  -
> > - 155 S 1400 E RM 233                       beebe@acm.org
> beebe@computer.org -
> > - Salt Lake City, UT 84112-0090, USA    URL:
> http://www.math.utah.edu/~beebe/ -
> >
> -------------------------------------------------------------------------------
>
> --
> ---
> Larry McVoy                  lm at mcvoy.com
> http://www.mcvoy.com/lm
>

[-- Attachment #2: Type: text/html, Size: 3561 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-02 21:24 [TUHS] [tuhs] The Unix shell: a 50-year view Nelson H. F. Beebe
  2021-07-02 21:36 ` Larry McVoy
@ 2021-07-02 22:27 ` Chet Ramey
  2021-07-02 23:09 ` Steve Nickolas
  2 siblings, 0 replies; 109+ messages in thread
From: Chet Ramey @ 2021-07-02 22:27 UTC (permalink / raw)
  To: Nelson H. F. Beebe, The Unix Heritage Society mailing list

On 7/2/21 5:24 PM, Nelson H. F. Beebe wrote:
> In this week's BSDNow.tv podcast, available at
> 
>          https://www.bsdnow.tv/409
> 
> there is a story about a new conference paper on the Unix shell.  The
> paper is available at
> 
> 	Unix shell programming: the next 50 years
> 	HotOS '21: Workshop on Hot Topics in Operating Systems, Ann
> 	Arbor, Michigan, 1 June, 2021--3 June, 2021
> 	https://doi.org/10.1145/3458336.3465294
> 
> The tone is overall negative

Perhaps, though they do say

"The shell is a useful abstraction deserving of our attention despite its
imperfections (Section 2)."

and

"The Unix shell hits a sweet spot between succinctness, expressive power,
and performance."

I participated in the accompanying panel. Everyone seems to hate the
shell's quoting rules and word splitting, and those two things alone are
sufficient to give the shell a bad rap, yet one thing that became clear
is that people are trying to use the shell (or not) without understanding
the underlying Unix abstractions that we take for granted (such as the file
system). And what's worse: they don't want to and nobody is teaching them.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
		 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet@case.edu    http://tiswww.cwru.edu/~chet/

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-02 21:24 [TUHS] [tuhs] The Unix shell: a 50-year view Nelson H. F. Beebe
  2021-07-02 21:36 ` Larry McVoy
  2021-07-02 22:27 ` [TUHS] [tuhs] The Unix shell: a 50-year view Chet Ramey
@ 2021-07-02 23:09 ` Steve Nickolas
  2 siblings, 0 replies; 109+ messages in thread
From: Steve Nickolas @ 2021-07-02 23:09 UTC (permalink / raw)
  To: The Unix Heritage Society mailing list

On Fri, 2 Jul 2021, Nelson H. F. Beebe wrote:

> List readers will have their own views of the paper.  My own is that,
> despite its dark corners, the Bourne shell has served us
> extraordinarily well, and I have been writing in it daily for decades
> without being particularly bothered by the many issues raised by the
> paper's authors.  Having dealt with so-called command shells on
> numerous other operating systems, at least the Unix shells rarely get
> in my way.

I still use the descendent Korn shell most of the time (often in the form 
of bash) on modern systems, and I don't just mean those that are 
Unix-like.  I've found that a lot of times I can build a quick tool using 
bash or ksh93, or use a bash one-liner, and do something that's impossible 
with command.com on DOS or cmd.exe on NT and OS/2.

-uso.

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-02 21:56   ` Henry Bent
@ 2021-07-02 23:12     ` Steve Nickolas
  2021-07-02 23:49       ` Steffen Nurpmeso
  2021-07-03 12:04       ` Thomas Paulsen
  2021-07-04 18:17     ` John Dow via TUHS
  1 sibling, 2 replies; 109+ messages in thread
From: Steve Nickolas @ 2021-07-02 23:12 UTC (permalink / raw)
  To: The Unix Heritage Society mailing list

On Fri, 2 Jul 2021, Henry Bent wrote:
> On Fri, 2 Jul 2021 at 17:38, Larry McVoy <lm@mcvoy.com> wrote:
>
>> As I started reading it I found plenty to disagree with in the first few
>> paragraphs but they completely lost me at "After all, moving from System
>> V init scripts to systemd has arguably improvedthe Linux boot sequence."
>>
>> Um, no, just, no.
>
> After rattling around in the back of a train car with nothing but my
> thoughts, I emerged and said:
>
> "systemd is a pox on our way of life"
>
> and then promptly rolled over and went back to sleep.
>
> -Henry
>

The Linux distro I use does use systemd but I can ignore it and go on with 
my life as if it were still running sysvinit.  So it's not that big a deal 
to me.

I'd prefer that they kept sysvinit, but eh.  What's that saying?  "Those 
who fail to understand Unix are condemned to recreate it, poorly" ?

-uso.

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-02 23:12     ` Steve Nickolas
@ 2021-07-02 23:49       ` Steffen Nurpmeso
  2021-07-03 13:34         ` Steffen Nurpmeso
  2021-07-03 12:04       ` Thomas Paulsen
  1 sibling, 1 reply; 109+ messages in thread
From: Steffen Nurpmeso @ 2021-07-02 23:49 UTC (permalink / raw)
  To: Steve Nickolas; +Cc: The Unix Heritage Society mailing list

Steve Nickolas wrote in
 <alpine.DEB.2.21.2107021910240.15013@sd-119843.dedibox.fr>:
 |On Fri, 2 Jul 2021, Henry Bent wrote:
 |> On Fri, 2 Jul 2021 at 17:38, Larry McVoy <lm@mcvoy.com> wrote:
 |>
 |>> As I started reading it I found plenty to disagree with in the first few
 |>> paragraphs but they completely lost me at "After all, moving from System
 |>> V init scripts to systemd has arguably improvedthe Linux boot sequence."
 |>>
 |>> Um, no, just, no.
 |>
 |> After rattling around in the back of a train car with nothing but my
 |> thoughts, I emerged and said:
 |>
 |> "systemd is a pox on our way of life"
 |>
 |> and then promptly rolled over and went back to sleep.
 |>
 |> -Henry
 |>
 |
 |The Linux distro I use does use systemd but I can ignore it and go on with 
 |my life as if it were still running sysvinit.  So it's not that big a deal 
 |to me.
 |
 |I'd prefer that they kept sysvinit, but eh.  What's that saying?  "Those 
 |who fail to understand Unix are condemned to recreate it, poorly" ?

Well iwd(8) for example that i happen to have installed as
a replacement for wpa_supplicant just today (because
.. i installed my beloved sysvinit/BSD rc based CRUX-Linux on
a new spare laptop, a 18 month old but totally new IdeaPad S145,
and since i do not have initrd kernels but compile an all-builtin
one i use a generic kernel in order to lsmod(8) in order to be
able to create a custom kernel with a modified "make localyesmod"
that Linux supports, here ArchLinux 2020.12, because AlpineLinux
did not do it, somehow, and whereas i hate that ArchLinux now that
all old developers fled does not even ship install instructions it
actually does come with iwd/iwctl, and it happened that iwctl
looked so much better to me than wpa_cli of wpa_supplicant, that
i took the time and replaced there here, iwctl has at least one
bug though, iwd ditto, and i must say that whereas iwd is much
smaller is has a larger memory footprint than wpa_supplicant, and
.. whatever), and note that iwd neither supports a normal syslog
nor command line parameters to configure paths etc., the latter
have to come in via environment if they shall, and the manual says
these are "normally provided by systemd" (which is not true .. for
me).  And no syslog at all!  No PID file handling.  And bugs even
though Intel.com paid for it.  I mean "get up, stand up", they do
a thousand things and not seldom i wonder.

@Tony Finch: wonderful reading of the first two pages, thanks for
this link!  It seems Knuth always had "a touch" for writing good,
and for good writing.

P.S.: not that do-one-thing's like sysklogd are bug free, for
example the sysklogd CRUX now uses/offers introduced a new bug
that causes /var/log/kernel to print monotonic clock times or
what, mind you, just now it is "Jun 30 13:13:16" (for "kent").

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-02 21:36 ` Larry McVoy
  2021-07-02 21:56   ` Henry Bent
@ 2021-07-03  0:09   ` Andrew Warkentin
  2021-07-03 15:49   ` Andy Kosela
  2 siblings, 0 replies; 109+ messages in thread
From: Andrew Warkentin @ 2021-07-03  0:09 UTC (permalink / raw)
  To: tuhs

On 02/07/2021, Larry McVoy <lm@mcvoy.com> wrote:
> As I started reading it I found plenty to disagree with in the first few
> paragraphs but they completely lost me at "After all, moving from System
> V init scripts to systemd has arguably improvedthe Linux boot sequence."
>
> Um, no, just, no.
>
>
I'm definitely not a fan of systemd, but I think legacy init systems
leave a lot to be desired as well. UX/RT, the OS I'm writing, will
have an init system that uses declarative unit files for daemons and
manages them statefully, but unlike systemd will be designed to be
highly extensible.

Basically all policy, including startup and shutdown, will be in
scripts. Startup and shutdown will mostly be implemented with one
script per phase (like on some older SysV and Linux systems). It will
also be possible to add support for new options to unit files by
adding hook scripts.

In addition, instead of having a single daemon managing all unit
types, there will be separate delegated restarters for all unit types
other than one-shots and unconditional daemons (it will be kind of
like SMF minus the databases and XML).

I'm also probably going to write my own shell at some point. Most
likely it will be Bourne-like with some object-oriented features,
including support for piping objects (unlike PowerShell this will work
with external commands and will be implemented by serializing objects
to text-based formats using hook functions). All UI-related features
like history, editing, and completion will be implemented in a
separate listener process that functions as a pseudo-terminal driver.

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-02 23:12     ` Steve Nickolas
  2021-07-02 23:49       ` Steffen Nurpmeso
@ 2021-07-03 12:04       ` Thomas Paulsen
  2021-07-03 13:20         ` Dan Cross
  1 sibling, 1 reply; 109+ messages in thread
From: Thomas Paulsen @ 2021-07-03 12:04 UTC (permalink / raw)
  To: Steve Nickolas; +Cc: tuhs

>The Linux distro I use does use systemd but I can ignore it and go on with my life as if it were still running sysvinit.  So it's not that big a deal to me.
I'm running red hat fedora which switched 100% to systemd a couple of years ago. It's fine, a good piece of software.



^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-03 12:04       ` Thomas Paulsen
@ 2021-07-03 13:20         ` Dan Cross
  2021-07-03 17:37           ` Theodore Ts'o
                             ` (3 more replies)
  0 siblings, 4 replies; 109+ messages in thread
From: Dan Cross @ 2021-07-03 13:20 UTC (permalink / raw)
  To: Thomas Paulsen; +Cc: tuhs

[-- Attachment #1: Type: text/plain, Size: 5028 bytes --]

On Sat, Jul 3, 2021 at 8:36 AM Thomas Paulsen <thomas.paulsen@firemail.de>
wrote:

> >The Linux distro I use does use systemd but I can ignore it and go on
> with my life as if it were still running sysvinit.  So it's not that big a
> deal to me.
> I'm running red hat fedora which switched 100% to systemd a couple of
> years ago. It's fine, a good piece of software.
>

Systemd is both good and bad.

The good part is that it allows for parallel service startup, which means
that your system gets up and running (and serving useful work) faster. It
understands the service dependency graph and thus the service start order
and what can be done simultaneously. It also, in theory anyway, makes life
a little easier for the authors of what I'll call "service software" (read:
daemons and servers): need a privileged resource? Describe it to systemd,
that will then acquire it for you and hand it to your unprivileged service
software. That's nifty, aids privilege separation, namespace sandboxing a
la containers, etc.

But systemd is also terrible. In an extremely un-Unix-like manner, it
consolidates an enormous number of disparate functions into a single
monolithic piece of software. It uses ersatz data formats that are abjectly
horrible. It uses binary logging, which means that one cannot use the full
complement of Unix text filters to inspect logs. It does all sorts of stuff
behind the scenes (manipulating filesystems, for example) in ways that are
obscured and opaque to system administrators, let alone programmers; this
can have surprising results (ask Ron Minnich about systemd-related data
loss). It has complete disregard, and indeed, contempt for the Unix
philosophy.

What's interesting (to me, anyway), is how few people care about the
deficiencies. This suggests a pretty fundamental shift in how people use
Unix-like systems, and Linux in particular: for _most_ users, it has become
a commoditized vessel for running other interesting software, but not what
we historically think of as "Unix". Sure, a lot of people still live at the
command line, but that's no longer the primary focus of interaction. Our
machines are now either all-singing, all-dancing multimedia workstations
where the sole user is in a web browser most of the time, in which case the
underlying system is just a detail, they're server-class machines that are
running some workload, but not something that's interactive, or embedded
devices that are black boxes.

Much of Unix's early evolution and thus architecture and philosophy, came
from addressing a set of problems that people had in a historical context
that, one could argue, aren't that relevant anymore. A hierarchical
filesystem in a global namespace, pipelines facilitating chaining of
filters for interactive use, a simple but weak permissions model,
unstructured text as a universal interchange format, terminal-oriented text
editors.... All of these were fine on shared multiuser interactive
machines, but do they matter as much now? If the nexus of interaction is a
web browser, who cares whether I can grep a log file? If I just want my
bluetooth headphones and USB camera to work for an online meeting, how that
stuff is put together under the hood isn't that interesting to me. In
short, the Unix philosophy is becoming less and less relevant as a new
generation of users and programmers put different demands on systems. Linux
is _probably_ the most prevalent kernel in the world, but most places its
used it's hidden from view. In that context, something like systemd
actually makes some amount of sense.

Perhaps I've mentioned this story before: a former colleague at Google was
writing a shell script to figure something out. It wasn't working. Really,
what he wanted was basically a `grep -q` and an `if` statement, but he'd
written a complicated mess with shell functions that tried to "return" the
exit status of the processes they ran: his shell script was written more
like a Python program. I rewrote it for him in half-a-dozen or so lines
(down from 30 or 40) and talked about Unix for a minute, the philosophy,
etc. At the end of my monologue extolling the virtues of our style of
computing, he looked at me blankly and said something along the lines of,
"yeah. I think that time has passed and people just don't conceptualize
problems that way anymore." I was sad, but is it any wonder that the kids
reach for Python before a shell script these days? We aren't teaching, yes,
but moreover they don't want to learn because the problems they're trying
to solve are different. We need to accept and internalize that and think
hard about how the tools we use can be brought to bear on the problems that
are relevant _now_, not 30 years ago.

It does beg the question: is a Unix-style kernel still the appropriate
foundation for this style of computing? Why are we still using this system:
is it because of its intrinsic power and versatility, or because of
inertia? Systemd, containers, etc, all suggest that our models are
inadequate.

        - Dan C.

[-- Attachment #2: Type: text/html, Size: 5657 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-02 23:49       ` Steffen Nurpmeso
@ 2021-07-03 13:34         ` Steffen Nurpmeso
  2021-07-03 13:56           ` Richard Salz
  0 siblings, 1 reply; 109+ messages in thread
From: Steffen Nurpmeso @ 2021-07-03 13:34 UTC (permalink / raw)
  To: Steve Nickolas; +Cc: The Unix Heritage Society mailing list

Steffen Nurpmeso wrote in
 <20210702234941.MjMt0%steffen@sdaoden.eu>:
 |Steve Nickolas wrote in
 | <alpine.DEB.2.21.2107021910240.15013@sd-119843.dedibox.fr>:
 ||On Fri, 2 Jul 2021, Henry Bent wrote:
 ||> On Fri, 2 Jul 2021 at 17:38, Larry McVoy <lm@mcvoy.com> wrote:
 ||>> As I started reading it I found plenty to disagree with in the \
 ||>> first few
 ||>> paragraphs but they completely lost me at "After all, moving from \
 ||>> System
 ||>> V init scripts to systemd has arguably improvedthe Linux boot sequence.\
 ||>> "
 ||>>
 ||>> Um, no, just, no.
 ...
 ||The Linux distro I use does use systemd but I can ignore it and go \
 ||on with 
 ||my life as if it were still running sysvinit.  So it's not that big \
 ||a deal 
 ||to me.
 ||
 ||I'd prefer that they kept sysvinit, but eh.  What's that saying?  "Those 
 ||who fail to understand Unix are condemned to recreate it, poorly" ?
 |
 |Well iwd(8) for example that i happen to have installed as
 ...
 |.. whatever), and note that iwd neither supports a normal syslog
 |nor command line parameters to configure paths etc., the latter
 |have to come in via environment if they shall, and the manual says
 |these are "normally provided by systemd" (which is not true .. for
 |me).  And no syslog at all!  No PID file handling.  And bugs even
 |though Intel.com paid for it.  I mean "get up, stand up", they do
 |a thousand things and not seldom i wonder.

And not to mention that there is a builtin DHCP server, but it
only can be integrated nicely "back" into systemd, or resolvconf.
You know, so much logic, and then not even the possibility to
simply say "hook" to invoke a simply shell script.  So i need
dhcpcd in addition, just for that hook that i use, for example, to
setup dnsmasq resolv, and to start a shell-based rdate(8) hook to
contact my ntpd-driving web VM via VPN.  I mean, what can happen,
address, gateway, and some more variables.  Carrier gained and
carrier lost.  Just some environment variables to pass along.

And mind you, it is getting worse it seems.  They write their
software more and more integrated specifically for systemd beyond
that iwd thing.  They do not generate manual pages when creating
release balls.  They require newest compiler additions for some
minor syntax sugar on less that one percent of their codebase.  It
seems some new "systems" even require an active internet
connection .. for compiling the software!  (Rust?)

I mean, that is not new.  Zawinski's complain on completely
rewriting Netscape Navigator in C++ for the rewrites sake can
still be read on the web.  In parts it may be normal progression,
it is just the dumb to which it appears as entanglement.
With a "Happy to be stupid",

Ciao from Germany,

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-03 13:34         ` Steffen Nurpmeso
@ 2021-07-03 13:56           ` Richard Salz
  0 siblings, 0 replies; 109+ messages in thread
From: Richard Salz @ 2021-07-03 13:56 UTC (permalink / raw)
  To: Steve Nickolas, The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 261 bytes --]

I remember at some Usenix panel, Steve Johnson said "what happens to the
Unix model when the primary mode of interaction isn't a set of bytes"  This
was 20+ years ago, IIRC. I think Rob Pike's ACME and Squeak tried to
provide an answer to a half-way situation.

[-- Attachment #2: Type: text/html, Size: 306 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-02 21:36 ` Larry McVoy
  2021-07-02 21:56   ` Henry Bent
  2021-07-03  0:09   ` Andrew Warkentin
@ 2021-07-03 15:49   ` Andy Kosela
  2021-07-04 23:24     ` [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view) Derek Fawcus
  2 siblings, 1 reply; 109+ messages in thread
From: Andy Kosela @ 2021-07-03 15:49 UTC (permalink / raw)
  To: Larry McVoy; +Cc: The Unix Heritage Society mailing list

On 7/2/21, Larry McVoy <lm@mcvoy.com> wrote:
> As I started reading it I found plenty to disagree with in the first few
> paragraphs but they completely lost me at "After all, moving from System
> V init scripts to systemd has arguably improvedthe Linux boot sequence."
>
> Um, no, just, no.

They come from the same "modern" camp of cloud fanatics that claim
"there was no automation before Ansible and no containerization before
Docker".  They also think that C is obsolete and your software stack
is useless if you don't include a dozen or so flashy bleeding edge
technologies and programming languages.

This is the kind of techno-cult fashion that Nikolai Bezroukov warned
about.  In the last few years we have seen more and more "papers" that
debunk and mock older stable Unix technologies because they want to
forcefully push their agenda and their new marketing terms.  We have
seen this with systemd which was forcefully pushed down on us; we have
seen this with Ansible, etc.

This vandalization of Unix already took place and I do not see a clear
way out of this situation.

--Andy

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-03 13:20         ` Dan Cross
@ 2021-07-03 17:37           ` Theodore Ts'o
  2021-07-03 17:57             ` Warner Losh
  2021-07-04  0:47           ` Tomasz Rola
                             ` (2 subsequent siblings)
  3 siblings, 1 reply; 109+ messages in thread
From: Theodore Ts'o @ 2021-07-03 17:37 UTC (permalink / raw)
  To: Dan Cross; +Cc: tuhs

On Sat, Jul 03, 2021 at 09:20:57AM -0400, Dan Cross wrote:
> 
> Systemd is both good and bad.

I agree with most of what you assess as "good" and "bad" with systemd.
However...

> It uses ersatz data formats that are abjectly
> horrible. It uses binary logging, which means that one cannot use the full
> complement of Unix text filters to inspect logs....

To be fair, systemd isn't the first.  BSD's sar program uses a binary
log file.

I'm not sure whether it's BSD's fault or only after it was pulled into
the systat package for Linux, but the binary format for the sar file
is (a) not self-describing, (b) not backwards compatible, and (c) not
stable between different versions, such that if you copy /var/log/sa/saNN
the files from one system and try to interpret it on another system, you
need to make sure that other system has the same version of sar installed.

This can be a headache if you are trying to debug an enterprise system
which is using a seven-year-old version of RHEL, and your laptop is
running something a bit more modern.

Things like Ingres also came out of Berkely, and modern databases all
use binary format files.  Is anything running a database, whether it
be Ingress, Postgres, Oracle Enterprise Database, DB2, no longer "Unix"?

And then there's AIX, which uses binary config files which are
manipulated using "smit", and mixes per-machine specific configs with
more general configs, all in a single config (or should I say,
"registry") file, so $DEITY help you if you try to copy the smit
database from one system to another if you are trying to do some kind
of large scale administration setup.  And granted there are many
people would dispute whether AIX is actually "Unix", it technically
speaking qualified to use the "Unix"(tm) trademark.

Similarly, Digital Equipment Corpartion's Ultrix used a binary log
file which you had to transmogrify using the uerf ("Ultrix Error
Report Formatter").  Like systemd, Ultrix uses a structured logging so
you can pull out specific types of logs, and Ultrix's uerf works much
systemd's journalctl.

So the concept of using binary logs and binary files predates systemd,
and there are more than a few examples that can be found in historical
systems that most people would have no trouble calling "Unix", and
even qualify for the Unix(tm) trademark.

					- Ted

P.S. For the most part, Linux systems don't qualify for the Unix(tm)
trademark, although it's not clear most people care about that these
days.  (A long time ago distributions paid $$$ to Posix Compliance
labs, probably because the government procurement requirements
required Unix(tm) systems, much like OSI stacks were simlarly required
by government procurement requirements.)



^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-03 17:37           ` Theodore Ts'o
@ 2021-07-03 17:57             ` Warner Losh
  2021-07-03 18:10               ` Theodore Ts'o
  0 siblings, 1 reply; 109+ messages in thread
From: Warner Losh @ 2021-07-03 17:57 UTC (permalink / raw)
  To: Theodore Ts'o; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 1308 bytes --]

On Sat, Jul 3, 2021 at 11:45 AM Theodore Ts'o <tytso@mit.edu> wrote:

> On Sat, Jul 03, 2021 at 09:20:57AM -0400, Dan Cross wrote:
> >
> > Systemd is both good and bad.
>
> I agree with most of what you assess as "good" and "bad" with systemd.
> However...
>
> > It uses ersatz data formats that are abjectly
> > horrible. It uses binary logging, which means that one cannot use the
> full
> > complement of Unix text filters to inspect logs....
>
> To be fair, systemd isn't the first.  BSD's sar program uses a binary
> log file.
>

You mean System V's sar program? BSD has never had a sar program.
It's not in any of the CSRG releases, nor in any of {Free,Net,Open}BSD
systems that are around today.

It first appeared in System Vr1, and was in each of the following releases
if the listings of files on the net are any indication.


> I'm not sure whether it's BSD's fault or only after it was pulled into
> the systat package for Linux, but the binary format for the sar file
> is (a) not self-describing, (b) not backwards compatible, and (c) not
> stable between different versions, such that if you copy /var/log/sa/saNN
> the files from one system and try to interpret it on another system, you
> need to make sure that other system has the same version of sar installed.
>

Sounds awful...

Warner

[-- Attachment #2: Type: text/html, Size: 2001 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-03 17:57             ` Warner Losh
@ 2021-07-03 18:10               ` Theodore Ts'o
  2021-07-03 20:02                 ` Dan Cross
  0 siblings, 1 reply; 109+ messages in thread
From: Theodore Ts'o @ 2021-07-03 18:10 UTC (permalink / raw)
  To: Warner Losh; +Cc: TUHS main list

On Sat, Jul 03, 2021 at 11:57:44AM -0600, Warner Losh wrote:
> 
> You mean System V's sar program? BSD has never had a sar program.
> It's not in any of the CSRG releases, nor in any of {Free,Net,Open}BSD
> systems that are around today.

Ah, I got confused because a web search turned up:

https://www.freebsd.org/cgi/man.cgi?query=sar&sektion=1M&apropos=0&manpath=SunOS+5.8

.... and I got mislead because FreeBSD hosts Solaris man pages.

Although it appears more recently FreeBSD has "bsdsar" in its Ports.

My larger point remains, though; is System V considered "Unix" by Dan
Cross's definition?   :-)

	    	    	 	  	      	       - Ted

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-03 18:10               ` Theodore Ts'o
@ 2021-07-03 20:02                 ` Dan Cross
  0 siblings, 0 replies; 109+ messages in thread
From: Dan Cross @ 2021-07-03 20:02 UTC (permalink / raw)
  To: Theodore Ts'o; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 3307 bytes --]

On Sat, Jul 3, 2021 at 2:10 PM Theodore Ts'o <tytso@mit.edu> wrote:

> On Sat, Jul 03, 2021 at 11:57:44AM -0600, Warner Losh wrote:
> >
> > You mean System V's sar program? BSD has never had a sar program.
> > It's not in any of the CSRG releases, nor in any of {Free,Net,Open}BSD
> > systems that are around today.
>
> Ah, I got confused because a web search turned up:
>
>
> https://www.freebsd.org/cgi/man.cgi?query=sar&sektion=1M&apropos=0&manpath=SunOS+5.8
>
> .... and I got mislead because FreeBSD hosts Solaris man pages.
>
> Although it appears more recently FreeBSD has "bsdsar" in its Ports.
>

Huh. I guess 'sa' wasn't good enough, though that also uses a format that
suffers from many of the same deficiencies you point out for 'sar'.

My larger point remains, though; is System V considered "Unix" by Dan
> Cross's definition?   :-)


Well, with the caveat that I'm pretty sure Dan Cross's definition of Unix
is of interest to an extremely small set of people (possibly the empty
set...), and that any answer is a bit of a meditation on the existential
philosophy of, "really, what _is_ Unix, anyway?" I'll go out on a limb and
say yes.

I don't think that the pervasive use of text formats for logs in much of
the system was ever meant to be to the exclusion of binary formats for
specific applications where that was warranted (usually for efficiency),
which have existed since the near the beginning. Off the top of my head,
representative examples of logging or logging-adjacent binary formats
common on Unix systems and extending into antiquity include utmp/wtmp,
lastlog, the quota database and acct; perhaps even crash dumps and core
files might fall under this overarching umbrella.

Not to mention all of the other ubiquitous binary file formats that've been
floating around seemingly forever: tar/tp/ar/UFS dump, dbm/ndbm, any number
of compressed file formats, and so on. Not to mention the on-disk data
structures describing filesystems and so on. But historically speaking,
logs, whether generated by explicit print statements in programs or by
something like calling `syslog()`, have been text. I suppose the point
being that, while individual use cases might use some binary format, by and
large most systems opted to use text and for many years that was just the
accepted and anticipated course of action. It was the default.

Systemd turns this on it's head. Text isn't just no longer the default,
it's just not an option. Sure, the aforementioned things are similar, but
this is a step beyond.

And note that I didn't say that Linux wasn't Unix because of systemd (or
perhaps more properly, that the Linux distributions that use systemd
weren't Unix), but rather, that systemd itself wasn't very Unixy. I think
that's accurate. Of course with respect for the other binary things you
mentioned (databases and so on) people write all kinds of whacky things for
systems, but those don't define the system. Systemd is much closer to the
"system" than, say Ingres of Oracle or PostgreSQL.

I do think this shift probably started with System V, but it still largely
follows the model of "Unix". And even AIX has syslog.

It's funny...I have no recollection of the mechanism you describe for
Ultrix, and a DECstation 5000/240 was my main machine for a good while.

        - Dan C.

[-- Attachment #2: Type: text/html, Size: 4490 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-03 13:20         ` Dan Cross
  2021-07-03 17:37           ` Theodore Ts'o
@ 2021-07-04  0:47           ` Tomasz Rola
  2021-07-04  4:36             ` Larry McVoy
  2021-07-04 12:48             ` Dan Cross
  2021-07-04 20:10           ` Tony Finch
  2021-07-05  3:52           ` Bakul Shah
  3 siblings, 2 replies; 109+ messages in thread
From: Tomasz Rola @ 2021-07-04  0:47 UTC (permalink / raw)
  To: tuhs

On Sat, Jul 03, 2021 at 09:20:57AM -0400, Dan Cross wrote:
[...]
> Much of Unix's early evolution and thus architecture and philosophy, came
> from addressing a set of problems that people had in a historical context
> that, one could argue, aren't that relevant anymore.

I mostly agree with you, but I think certain things should be
expressed more explicitly, even if I do not want to be picky. So, I
see the claim of Unix not being relevant anymore might be interpreted
in two ways, and both are not (quite) true. 

First, if it means "for all people it is not relevant anymore", such
thing never happened IMHO. For this, all those folks would need to
consider Unix as a tool for their problems and decide that it was
obsolete. But, most of humans were never so much interested in
computers to know about "some yooo-neex" and this state of things
lasts to these days. If they use computer (cellphone), it is most
probably because now the devices enable dating and other forms of
entertainment. Some of them even heard about "Leah nukes".

Another way of understanding your statement is "there was a group of
folks for whom Unix was relevant and lost its appeal". This, again,
not true. The group itself may have lost members and gained members,
but I believe it is more or less same size. Maybe even bigger than it
was, and ability to have anything "unix-like" for free (as in
"voluntary donation") plays huge role here.

> A hierarchical filesystem in a global namespace, pipelines
> facilitating chaining of filters for interactive use, a simple but
> weak permissions model, unstructured text as a universal interchange
> format, terminal-oriented text editors.... All of these were fine on
> shared multiuser interactive machines, but do they matter as much
> now?

Depends. In a land of the blind a one-eye should be very careful,
because he would not be a king. I certainly would not like to be
endlessly asked for help with installing Linux. I think majority of
computer users do not care, will not care, that they have some files
on their machines or even that what they have are the machines. So,
they might not appreciate your knowledge about such things. Perhaps it
is a good thing?

[...]
> Perhaps I've mentioned this story before: a former colleague at Google was
> writing a shell script to figure something out. It wasn't working. Really,
> what he wanted was basically a `grep -q` and an `if` statement, but he'd
> written a complicated mess with shell functions that tried to "return" the
> exit status of the processes they ran: his shell script was written more
> like a Python program. I rewrote it for him in half-a-dozen or so lines
> (down from 30 or 40) and talked about Unix for a minute, the philosophy,
> etc. At the end of my monologue extolling the virtues of our style of
> computing, he looked at me blankly and said something along the lines of,
> "yeah. I think that time has passed and people just don't conceptualize
> problems that way anymore."

I think what he really meant (knowlingly or not) was "the money walks
somewhere else". As far as I can say, the kids are just following
money. This kind of behaviour is called a tropism. Since this is how
the world goes, I am not sure it can be described as bad or good. This
is very much like evolution, thanks to which we are living in and are
the result of living in, a huge slaughterhouse. The 30cm long
centipede hunts bats which evolved long after centipede and long after
insects were dominant life forms on a planet. /bin/sh will be used to
solve problems in a future, too. Innumerable Unix servers will be sent
to the landfill. systemd will grow a hairy-sticky ecosystem of its
own, developing pseudo-kernels like pseudo-brains in dinosaur belly.

-- 
Regards,
Tomasz Rola

--
** A C programmer asked whether computer had Buddha's nature.      **
** As the answer, master did "rm -rif" on the programmer's home    **
** directory. And then the C programmer became enlightened...      **
**                                                                 **
** Tomasz Rola          mailto:tomasz_rola@bigfoot.com             **

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-04  0:47           ` Tomasz Rola
@ 2021-07-04  4:36             ` Larry McVoy
  2021-07-04 14:56               ` Dan Cross
                                 ` (2 more replies)
  2021-07-04 12:48             ` Dan Cross
  1 sibling, 3 replies; 109+ messages in thread
From: Larry McVoy @ 2021-07-04  4:36 UTC (permalink / raw)
  To: Tomasz Rola; +Cc: tuhs

On Sun, Jul 04, 2021 at 02:47:57AM +0200, Tomasz Rola wrote:
> On Sat, Jul 03, 2021 at 09:20:57AM -0400, Dan Cross wrote:
> [...]
> > Much of Unix's early evolution and thus architecture and philosophy, came
> > from addressing a set of problems that people had in a historical context
> > that, one could argue, aren't that relevant anymore.

This is a response to Cross.  You are sort of right in that we are not on
uniprocessors where we disable interrupts to manage things.

Unix still matters.  It has mattered for a very long time, I could argue
it is the most important operating system in the world.  Yeah, windows
won, but it didn't win on merits.

In my opinion you couldn't be more wrong.  We still have the same problems,
we are still trying to grep an answer out of a bunch of info that has just
gotten bigger.

We still want to do the same things and we are doing them better with faster
CPUs, memory, disks, etc.

I maybe think the reason you think that things aren't relevant anymore are
because young people don't get Unix, they just pile on to this framework
and that framework, NONE OF WHICH THEY UNDERSTAND, they just push more
stuff onto the stack.

If you actually have a clue, if you can do stuff, all of that other stuff
becomes fluff.  Yep, some of it is useful but most of it is just there 
because it wants to feel important. 

Unix matters, the way that you can compose stuff still matters, people
who can do that run circles around the people who say Unix doesn't work.
My first job, they said 6 months, I did it 3 weeks by using what Unix 
gave me.

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-04  0:47           ` Tomasz Rola
  2021-07-04  4:36             ` Larry McVoy
@ 2021-07-04 12:48             ` Dan Cross
  2021-07-05  7:14               ` Tomasz Rola
  1 sibling, 1 reply; 109+ messages in thread
From: Dan Cross @ 2021-07-04 12:48 UTC (permalink / raw)
  To: Tomasz Rola; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 1074 bytes --]

On Sat, Jul 3, 2021 at 8:57 PM Tomasz Rola <rtomek@ceti.pl> wrote:

> On Sat, Jul 03, 2021 at 09:20:57AM -0400, Dan Cross wrote:
> [...]
> > Much of Unix's early evolution and thus architecture and philosophy, came
> > from addressing a set of problems that people had in a historical context
> > that, one could argue, aren't that relevant anymore.
>
> I mostly agree with you, but I think certain things should be
> expressed more explicitly, even if I do not want to be picky. So, I
> see the claim of Unix not being relevant anymore might be interpreted
> in two ways, and both are not (quite) true.


I didn't say that Unix is no longer relevant. I said that the problems that
were prominent when Unix was built, and that thus shaped its architecture,
are much _less_ relevant now than they were at the time. They still have
some relevance, but are no longer primary for the vast majority of use
cases Unix is used for. Further, the way people approach problems now has
changed, even if they still generically assume "Unix" as the base they
build on.

        - Dan C.

[-- Attachment #2: Type: text/html, Size: 1450 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-04  4:36             ` Larry McVoy
@ 2021-07-04 14:56               ` Dan Cross
  2021-07-04 16:07               ` Theodore Ts'o
  2021-07-04 20:10               ` David Barto
  2 siblings, 0 replies; 109+ messages in thread
From: Dan Cross @ 2021-07-04 14:56 UTC (permalink / raw)
  To: Larry McVoy; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 4491 bytes --]

On Sun, Jul 4, 2021 at 12:37 AM Larry McVoy <lm@mcvoy.com> wrote:

> On Sun, Jul 04, 2021 at 02:47:57AM +0200, Tomasz Rola wrote:
> > On Sat, Jul 03, 2021 at 09:20:57AM -0400, Dan Cross wrote:
> > [...]
> > > Much of Unix's early evolution and thus architecture and philosophy,
> came
> > > from addressing a set of problems that people had in a historical
> context
> > > that, one could argue, aren't that relevant anymore.
>
> This is a response to Cross.


Uh oh. I know I'm in trouble when I'm referred to by my last name only. :-)

You are sort of right in that we are not on
> uniprocessors where we disable interrupts to manage things.
>
> Unix still matters.  It has mattered for a very long time, I could argue
> it is the most important operating system in the world.  Yeah, windows
> won, but it didn't win on merits.
>

Whoa hold on now. I never said that _Unix_ doesn't still matter. It
emphatically does. I agree with you that it, or perhaps more precisely the
evolutionary tree of systems that fit under it's rather large umbrella, is
indeed the most important operating system family in the world. It runs
everywhere and it runs everything; it's literally out of this world. You
will get no argument from me there.

In my opinion you couldn't be more wrong.  We still have the same problems,
> we are still trying to grep an answer out of a bunch of info that has just

gotten bigger.
>


We still want to do the same things and we are doing them better with faster
> CPUs, memory, disks, etc.
>

But this I disagree with. We may still be trying to solve many of the same
problems, but entirely new categories of problems have sprung up since Unix
was invented, those problems have grown to frankly dwarf the size of the
previous problems, and our modes of interaction with the machine have
totally changed. Consider that the median interaction these days is some
RPC-ish call over HTTP between a user using a gesture interface on a phone
or tablet or something, and some service running somewhere "in the cloud".
Both sides of this (and the myriad stages in between) might be running on
machines that are running "Unix", but this emphatically is _not_ grep, at
least no more than extracting data out of IMS in COBOL under OS/360 was
also a glorified form of "grep".

I maybe think the reason you think that things aren't relevant anymore are
> because young people don't get Unix, they just pile on to this framework
> and that framework, NONE OF WHICH THEY UNDERSTAND, they just push more
> stuff onto the stack.
>

Yes, that was kind of my point. But not only do they not get Unix, they
don't want to, either. Why is that? I think that's the interesting question.


> If you actually have a clue, if you can do stuff, all of that other stuff
> becomes fluff.  Yep, some of it is useful but most of it is just there
> because it wants to feel important.


I agree with that last statement, but I'm not sure I agree with the former.
I'm not a front-end person; I'll never be a front-end person. Frankly, I
lack the artistic and aesthetic ability to ever be good at that kind of
work. My web site is frankly ugly; I don't care. But I've had the
opportunity to interact with folks who are actually good at front-end
stuff, and they're really sharp. They definitely have a clue. But it's an
entire type of clue that I simply do not nor ever will have.

Many of those people choose to use Linux as their daily driver environment;
bully for them. However, they don't reach for the compliment of tools that
many of us take for granted as being part and parcel of the Unix
environment are not what they reach for first. Most folks aren't using
`sed` to extract something from a typescript program or CouchDB or whatever.

Unix matters, the way that you can compose stuff still matters, people
> who can do that run circles around the people who say Unix doesn't work.
> My first job, they said 6 months, I did it 3 weeks by using what Unix
> gave me.
>

Forgive my saying this, but that was then, this is now. The kinds of
automation that I used to wow folks with when I was starting out by being
moderately facile with Unix tools just don't apply to entire classes of
problems anymore. And if the job is working on one of those problems....

This doesn't mean that Unix isn't still important. It obviously is. But it
means that what people are using it for, and how they're using it has
changed. And that does inevitably raise questions about system evolution.

        - Dan C.

[-- Attachment #2: Type: text/html, Size: 6363 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-04  4:36             ` Larry McVoy
  2021-07-04 14:56               ` Dan Cross
@ 2021-07-04 16:07               ` Theodore Ts'o
  2021-07-04 20:10               ` David Barto
  2 siblings, 0 replies; 109+ messages in thread
From: Theodore Ts'o @ 2021-07-04 16:07 UTC (permalink / raw)
  To: Larry McVoy; +Cc: tuhs

On Sat, Jul 03, 2021 at 09:36:15PM -0700, Larry McVoy wrote:
> In my opinion you couldn't be more wrong.  We still have the same problems,
> we are still trying to grep an answer out of a bunch of info that has just
> gotten bigger.
> 
> We still want to do the same things and we are doing them better with faster
> CPUs, memory, disks, etc.

I'm not sure I agree with you.  The same problems still exist, to be
sure.  But there are new problems that have also arisen, for which
Unix might not be the best match.  For example, Unix didn't have to
deal with devices appearing after the system has booted; or those
devices disappearing afterwards.  And of course, with services that
might need to be started or reconfigured or stopped as devices get
attached and detached.

The original Unix systems didn't have the requirement of automatically
detecting failed hardware or software, and taking action to mitigate
those faults without depending on operators in data centers.  If we
consider some of the structured logging systems, whether it's
Ultrix/OSF's uerf, or Solaris's SMF, or Tuttle (an internal system I
designed for $WORK), it's at least in part because scraping
/var/log/messages and having cluster management systems parse textual
log entries using an unholy mess of regex's is error prone and not
necessarily reliable or maintainable.

There are also new problems, such as Machine Learning, which has led
to the introduction of coprocessors, which is a great example of
something which is *not* doing the "same things" but with better CPUs,
memory, disks, etc.

You can of course define away some of these new problems as "not
interesting", or "not the true spirit of Unix", but the reality is
there are a good reason why systems have been changing --- it's in
response the pressures of new requirements and new problems being
added, even if the old problems haven't gone away.

> I maybe think the reason you think that things aren't relevant anymore are
> because young people don't get Unix, they just pile on to this framework
> and that framework, NONE OF WHICH THEY UNDERSTAND, they just push more
> stuff onto the stack.

This is perhaps an unavoidable response to increasing complexity.
When I was an undergraduate we started by building a computer using
TTL chips on a breadboard, and then we went on to learn Scheme and the
Lambda calculus, and then built up from there.  But a few years ago,
MIT decided this wasn't sufficiently relevant to how most software
engineers work today, so the intro to computing class was redone to
use Python, and it was a stated goal to give students experience in
how to build on top of a framework which is not sufficiently
documented, and where they need to figure out what the #?$!@? was
going on via experimentation, supplemented with whatever documentation
and sources that students could find/read/understand.

(And if you look at the depth of a stack trace after a typical Java
application crashes due to an unhandled exception, MIT has probably
gotten it right in terms of what most CS students will experience
after they graduate.  I remember looking 80-90 Java function names in
a Lotus Notes stack dump after a crash, involving the Eclipse *and*
the Standard Java library frameworks, with awe and horror...)

As a result, I've found that most new college grads don't have much in
the way of Systems training or experience, and for those groups that
need that kind of Systems thinking at $WORK, it's something we need to
train, and not something I can assume new hires will have.  (This is
also true for many candidates with significant industry experience
which I've interviewed, alas.)

						- Ted

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-02 21:56   ` Henry Bent
  2021-07-02 23:12     ` Steve Nickolas
@ 2021-07-04 18:17     ` John Dow via TUHS
  2021-07-04 19:46       ` Clem Cole
  1 sibling, 1 reply; 109+ messages in thread
From: John Dow via TUHS @ 2021-07-04 18:17 UTC (permalink / raw)
  To: Henry Bent; +Cc: The Unix Heritage Society mailing list

> On 2 Jul 2021, at 22:58, Henry Bent <henry.r.bent@gmail.com> wrote:
> 
> 
> After rattling around in the back of a train car with nothing but my thoughts, I emerged and said:
> 
> "systemd is a pox on our way of life"
> 
> and then promptly rolled over and went back to sleep.

I prefer to think of it as the answer to a question that no one was asking.

J

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-04 18:17     ` John Dow via TUHS
@ 2021-07-04 19:46       ` Clem Cole
  2021-07-05  1:33         ` Noel Hunt
  0 siblings, 1 reply; 109+ messages in thread
From: Clem Cole @ 2021-07-04 19:46 UTC (permalink / raw)
  To: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 10161 bytes --]

Hmm ... do I want to get in the middle of a fight with my friends and get
them all mad at me… but I guess I cann’t just keep quiet on this debate.



Basically I think Larry and Dan are having the Dan Akroyd/Gilda Radner
 “Shimmer”
Dessert Topping / Floor Wax <https://www.youtube.com/watch?v=wPO8PqHGWFU>
debate.   That said, I do think systemd is a giant step backward for most
of the same reasons others have already said, so I’ll not pile on and stick
to the basic what does it mean to be Unix or not/what is an improvement or
not, discussion.



Larry observes:  “I maybe think the reason you think that things aren't
relevant anymore are because young people don't get Unix, they just pile on
to this framework and that framework, NONE OF WHICH THEY UNDERSTAND, they
just push more stuff onto the stack.”



Simply, I could not have said that better.  But that is a symptom of the
problem.  Frankly, with frameworks and the like, we have added so many
levels of abstraction, we have completely lost sight of the real issues and
many folks really don’t “think like a programmer” any more.   Remember,
UNIX was a system *by programmers for programmers.*  Much work since has
been hiding a lot of what made it great IMO - which I think has caused the
most damage.



What makes a system a flavor UNIX or not is almost personal, based on what
you value (or not).  But the fact is no matter what you call it, “UNIX” is
the core kernel with a small and simple set of operations, plus a set of
programs that are all based on the simple ideas, basic concepts and mostly
solid mathematics/computer science that a small group of people in NJ came
up with at the time, when they suggested there might be a better way to put
things together than had been otherwise popular at the time.   Their
concepts and ideas collectively were different enough that they were
encouraged to write a paper about it.  That paper was accepted and
published in CACM, it got a lot of people in the community interested in
those ideas and  the rest is history as we say.



But a huge difference between then and now is the *economics* and thus the
matching equipment they used to explore those new ideas.  So some
solutions, we take for granted today, would not be practical, much less
even possible in those times.



Just like some people trying to claim that ‘Linux’ is not UNIX, falls away
deftly as well as it did years ago when other ‘progressive’ features were
added to 'Research UNIX' and we got systems like BSD 4.1 much less 4.2.
Remember smart people from Rob’s “cat -v” paper to Henry Spencer (“BSD is
just like UNIX, only different” net.noise comment) in those days railed on
the new BSD system as not being ‘UNIX’ either.  For good or for bad, many
of the things we take for granted today as being required in a UNIX system,
go back to UCB features (or features from AUUG, MIT, CMU or similar).



I’ve also stated many times, I do so miss the simplicity and cleanliness of
V6 and V7, but I would not want to use it for my daily work today.  So many
of those very features that Henry and Rob pointed out back in the day, have
somehow proven in fact to be ‘helpful’ or at least comfortable.  While a
few of us probably could live with something like ed(1), particularly with
a front-end/feature more like Plan9’s sam(1), than absolutely having to
have VI/EMACS.



Let me offer an example as a thought exercise.     I was recently helping a
young hacker trying to get V5/V6/V7 going on a PDP-11 simulator so he could
play with it to learn more about the 11 and UNIX itself.  I gave him some
help and without thinking about it in my email I mentioned that he try
something, but I had forgotten that the program head(1) was a Bill Joy
program wrote in 1977.  What UNIX box would not have it today?  As it is
screwed into the ROM in my fingers when I type (actually the sources to head
(1)  is less than 200 lines with ¼ that being the BSD copyright, I sent him
them and suggested he recompile it - partly as an exercise to see how C
changed).



But note that when wnj wrote head(1), Joy followed the famous ‘Unix
Philosophy’ of  doing one (small) job well.   Which means he did not add a
feature *i.e. *abusing, an old program, like cat(1), and add some new
switch to it that that told the program stop outputting after n lines.
Instead Joy wrote a simple new tool.



This is the problem with what Larry was pointing out, I think frameworks
and like are just adding features on features on features to current
subsystems.   Yeech!!!



To me what made Unix ‘great’ was that this small (16-bit) system, with
48k-256K of memory can could practically owned by a department at $150-250K
would allow you to do the same sorts of things that took a $1-4M type of
system (be it a PDP-10 or IBM or the like).



Mashey did his ACM lectures called “Small Is Beautiful”, sometime after
they completed PWB 1.0 and this was one of his points.   Unix was a small
SW system on a small HW system platform, but was clean and did what you
(the programmer) needed without a lot of extra cruft.  IIRC for PWB 1.0,
that was an 11/45 as the recommended system to run it.  Not the smallest
(11/40), but hardly the 11/70 either.   But as the 32-bit systems became
available, many of the different constraints of the PDP-11 were removed –
first data size and then text size.  When the VAX came and 32-bits was
infinite (probably still is for text space), performance got better (and
cheaper), disks got bigger, *etc*.



And because it was less and less of a problem, quickly programmers got a
tad lazy or at least stopped paying attention to things they were required
to consider in the past.  Ex (or for that matter EMACS) in thinking was the
first the UCB “explosions”  and maybe where UNIX began to be a tad
different and look less and less like what had come from the NJ avengers.
The resources of the new systems were such that you did not need a new set
of small programs, you added features (extensions) to the old – and that
was (is) a trap which we seem to follow today.



I think other folks like to point out all the new wonder new functionality
in the Linux kernel (or FreeBSd or macOS etc...) have brought to the UNIX
world besides BSD’s sockets networking (loadable drivers, dynamic devices
are super and I would not want to be without either).  I like shared
libraries, and better memory systems.  Although of course on my
supercomputer what are two things we turn off (shared libraries and much of
the fancy memory stuff - cause they get in the way of real programs ;-).


BTW:  I also think sockets were (are) a terrible addition to UNIX and we
did not really need them. We are stuck with them now, but  I would rather
have had something more like what ChaosNet and the original UofI Arpanet
code, or what UNET did, where much of the network stack was in user space
[which in fact was what the Interface BBN originally used].  In other
places, over time, we have added window managers GUI’s *et al*.  Hey the
core C compiler’s code generator is remarkable compared to what it was for
the PDP-11, plus we have new languages be they Rust, Go or even C++ and
Java.



The key point is few people are really asking the question, *if I add this
new feature what does it cost, and what do we really get for it?*



The truth is many of these features are all comforts and *I do like many of
them* and I use them and want many of them.   I like kernel support for MxN
threading, but that took kernel support – what did we get beyond fork/exec
that we really did not have (the answer is probably better performance due
to tighter control, but I wonder if we bought much more than that).  I’m
typing on a Mac, while most of why ‘work’ these days is targeted Linux.
But what is common is mostly I can work in the manner I want.  I have
something that does make some things nice (easier)… Both are ‘UNIX’ in some
manner – that core of both are the same core ideas I see in V5/V6/V7 –
that’s good.  *i.e.* they both work as a floor wax….



But all of those new features have come at a cost, the biggest is
complexity/bloat.  We have lost a lot in the design because today’s
programmers just don’t have to deal with the kind of constraints that some
of had to deal with years ago and I think you hear a number of us groaning
that when papers like this one come out, they have completely missed the
point and really don’t understand what it means to be UNIX.



I suspect if VM/TSO or VMS had become the core technology of the future,
the papers being written today would be just as damning.  If you never
lived in the old world, I’m not sure you understand the improvement.



I’m going to toss the gauntlet with a slightly even more political
statement.  When I read that paper, it reminded me of the current round of
anti-vaxxers.   Those of us that remember polio and other ‘childhood
diseases’ have no desire to go back in time and relive it.  I really don't
want to run VMS/RSX or for that matter TSS/360 which was the first system I
really ever knew how to use well.



So in the same way, UNIX is the core technology we have today and I’m
damned glad it ‘won’ be it called Linux, FreeBSD or macOS.  It was not
perfect then and it is hardly perfect today.  But it was different from
what we had at that time, and so much better - *that today we now use those
ideas* from UNIX as our core ideas when we build systems.   I just wish
people would understand what they have and see it for the positive instead
of trying to knock it down - ‘standing on the shoulders of the giants’ and
showing what they did as a help, but based on solid ideas, instead of
stepping on the toes trying to make their new thing/feature more valuable
and claim it’s not made from the origin story.



Hey Chevy… can I have some “shimmer” for my pudding.




ᐧ

[-- Attachment #2: Type: text/html, Size: 25136 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-04  4:36             ` Larry McVoy
  2021-07-04 14:56               ` Dan Cross
  2021-07-04 16:07               ` Theodore Ts'o
@ 2021-07-04 20:10               ` David Barto
  2021-07-05  0:25                 ` Larry McVoy
  2021-07-05  1:23                 ` John Cowan
  2 siblings, 2 replies; 109+ messages in thread
From: David Barto @ 2021-07-04 20:10 UTC (permalink / raw)
  To: Larry McVoy; +Cc: TUHS main list



> On Jul 3, 2021, at 9:36 PM, Larry McVoy <lm@mcvoy.com> wrote:
> 
> On Sun, Jul 04, 2021 at 02:47:57AM +0200, Tomasz Rola wrote:
>> On Sat, Jul 03, 2021 at 09:20:57AM -0400, Dan Cross wrote:
>> [...]
>>> Much of Unix's early evolution and thus architecture and philosophy, came
>>> from addressing a set of problems that people had in a historical context
>>> that, one could argue, aren't that relevant anymore.
> 
> Unix still matters.  It has mattered for a very long time, I could argue
> it is the most important operating system in the world.  Yeah, windows
> won, but it didn't win on merits.
> 

One thing I’d like to point out is that Linux On Windows is becoming
a thing, to the point that you can execute native Linux applications for
the most part directly in this weird environment.

So even M$ sees that Linux could be a threat in the long term and is
making an attempt to prevent it from harming the M$ universe.

See, you can run your Linux code on Windows, why do you want to
run Linux now? 8^)

	David


^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-03 13:20         ` Dan Cross
  2021-07-03 17:37           ` Theodore Ts'o
  2021-07-04  0:47           ` Tomasz Rola
@ 2021-07-04 20:10           ` Tony Finch
  2021-07-05  3:59             ` Theodore Ts'o
  2021-07-05  3:52           ` Bakul Shah
  3 siblings, 1 reply; 109+ messages in thread
From: Tony Finch @ 2021-07-04 20:10 UTC (permalink / raw)
  To: Dan Cross; +Cc: tuhs

Dan Cross <crossd@gmail.com> wrote:
>
> Systemd is both good and bad.

I thought this article was well-informed and informative, but VERY long:
https://blog.darknedgy.net/technology/2020/05/02/0/
"systemd, 10 years later: a historical and technical retrospective"

Tony.
-- 
f.anthony.n.finch  <dot@dotat.at>  https://dotat.at/
Humber, Thames: Southeast veering southwest, 3 to 5. Smooth or slight.
Thundery showers. Good, occasionally poor.


^ permalink raw reply	[flat|nested] 109+ messages in thread

* [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view)
  2021-07-03 15:49   ` Andy Kosela
@ 2021-07-04 23:24     ` Derek Fawcus
  2021-07-04 23:50       ` Nemo Nusquam
                         ` (4 more replies)
  0 siblings, 5 replies; 109+ messages in thread
From: Derek Fawcus @ 2021-07-04 23:24 UTC (permalink / raw)
  To: The Unix Heritage Society mailing list

On Sat, Jul 03, 2021 at 05:49:57PM +0200, Andy Kosela wrote:
> They also think that C is obsolete

I'd not say it is obsolete, but despite it having been my main language for the
last 30 years, these days I would be inclined to restrict the range of new tasks
I'd use it for.

So in thinking of how to solve certain problems, I'd split a subset of the problem
in to something in C, and the rest in to another language - probably Go.  That may
simply reflect the nature of the problems I tackle, I can imagine that others
might not merit any use of C.

DF

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view)
  2021-07-04 23:24     ` [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view) Derek Fawcus
@ 2021-07-04 23:50       ` Nemo Nusquam
  2021-07-05  0:15         ` Dan Stromberg
  2021-07-05  0:21       ` Larry McVoy
                         ` (3 subsequent siblings)
  4 siblings, 1 reply; 109+ messages in thread
From: Nemo Nusquam @ 2021-07-04 23:50 UTC (permalink / raw)
  To: tuhs

On 2021-07-04 19:24, Derek Fawcus wrote (in part):
> So in thinking of how to solve certain problems, I'd split a subset of 
> the problem
> in to something in C, and the rest in to another language - probably 
> Go. That may simply reflect the nature of the problems I tackle, I can 
> imagine that others might not merit any use of C.
This would be very much domain specific.  I work with embedded systems, 
where one's choice of language is limited.

N.

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view)
  2021-07-04 23:50       ` Nemo Nusquam
@ 2021-07-05  0:15         ` Dan Stromberg
  0 siblings, 0 replies; 109+ messages in thread
From: Dan Stromberg @ 2021-07-05  0:15 UTC (permalink / raw)
  To: Nemo Nusquam; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 861 bytes --]

On Sun, Jul 4, 2021 at 4:52 PM Nemo Nusquam <cym224@gmail.com> wrote:

> On 2021-07-04 19:24, Derek Fawcus wrote (in part):
> > So in thinking of how to solve certain problems, I'd split a subset of
> > the problem
> > in to something in C, and the rest in to another language - probably
> > Go. That may simply reflect the nature of the problems I tackle, I can
> > imagine that others might not merit any use of C.
> This would be very much domain specific.  I work with embedded systems,
> where one's choice of language is limited.
>


Have you looked at Micropython?  It's written in C, and targets embedded
applications.
https://micropython.org/

I recently compared its threading to a couple of other implementations of
Python, and found that Micropython was the best of those compared.
https://stromberg.dnsalias.org/~strombrg/python-thread-comparison/

[-- Attachment #2: Type: text/html, Size: 1430 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view)
  2021-07-04 23:24     ` [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view) Derek Fawcus
  2021-07-04 23:50       ` Nemo Nusquam
@ 2021-07-05  0:21       ` Larry McVoy
  2021-07-05  2:36         ` John Cowan
  2021-07-05 12:11         ` Thomas Paulsen
  2021-07-05  4:08       ` Dan Stromberg
                         ` (2 subsequent siblings)
  4 siblings, 2 replies; 109+ messages in thread
From: Larry McVoy @ 2021-07-05  0:21 UTC (permalink / raw)
  To: Derek Fawcus; +Cc: The Unix Heritage Society mailing list

On Mon, Jul 05, 2021 at 12:24:18AM +0100, Derek Fawcus wrote:
> On Sat, Jul 03, 2021 at 05:49:57PM +0200, Andy Kosela wrote:
> > They also think that C is obsolete
> 
> I'd not say it is obsolete, but despite it having been my main language for the
> last 30 years, these days I would be inclined to restrict the range of new tasks
> I'd use it for.
> 
> So in thinking of how to solve certain problems, I'd split a subset of the problem
> in to something in C, and the rest in to another language - probably Go.  That may
> simply reflect the nature of the problems I tackle, I can imagine that others
> might not merit any use of C.

I had a systems people get together at my place maybe a year ago.
Kirk McKusick and Eric Allman came (as well as a bunch of others,
perhaps most notable was Pugs).  I had a very pleasant chat with Eric
about programming styles.  I came to the conclusion that I'd love working
with Eric, I think I could work for him and it's even possible he could
work for me.  We both love C, we are both disciplined enough to write
maintainable/extendable code in C, it works for us.  We really clicked
over our love of C.

I've tried to like other languages, Java is a huge tangled mess, C++ is,
well C++, every company that uses C++ has a list of things you are not
supposed to use, I tried to like Go, maybe I haven't given it enough of
a chance but it didn't grab me, lots of people I know have gone to Rust,
I tried to like that but, man, all the reinventing of syntax drove me 
nuts, D seemed like it sort of was close but it changed the syntax too
much.

If I had infinite energy and money, I would fund a dialect of C that 
made it more useful.  The thing I hate about new programming languages
is they all feel the need to invent a new syntax.  There are really
two syntaxes, C and Lisp (and that's a tip of a hat to Lisp, I'm not
a fan).  Pick one of those and evolve it.

My whole career I've felt that when you add to the set of code that is
in the world, you should build on what people already know.  Everyone
knows C syntax, yeah there are some areas that could do with a cleanup
(looking at you function pointer declarations) but for the most part,
most people can look at C code and know what it does.  So why invent 
a new syntax and invalidate all of that knowledge that everyone has?

For all its faults, C++ is the closest to a modern language that sort
of fits with what I want but it is a kitchen sink language.  It really
wanted the Go trio to say "no" to a lot of stuff, it didn't have that 
so it has every language feature you can imagine and it's hard to 
onboard people to that.  Harder yet to write maintainable code.

So, yeah, I like C.  A lot.  I wish it would evolve in a sensible way
but as a buddy says, we live in an imperfect world.

--lm

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-04 20:10               ` David Barto
@ 2021-07-05  0:25                 ` Larry McVoy
  2021-07-05  1:23                 ` John Cowan
  1 sibling, 0 replies; 109+ messages in thread
From: Larry McVoy @ 2021-07-05  0:25 UTC (permalink / raw)
  To: David Barto; +Cc: TUHS main list

On Sun, Jul 04, 2021 at 01:10:14PM -0700, David Barto wrote:
> See, you can run your Linux code on Windows, why do you want to
> run Linux now? 8^)

I know that is tongue in cheek, but because I want a real kernel
that has sane semantics.  The windows kernel is pretty nuts.

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-04 20:10               ` David Barto
  2021-07-05  0:25                 ` Larry McVoy
@ 2021-07-05  1:23                 ` John Cowan
  1 sibling, 0 replies; 109+ messages in thread
From: John Cowan @ 2021-07-05  1:23 UTC (permalink / raw)
  To: david; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 1398 bytes --]

On Sun, Jul 4, 2021 at 4:11 PM David Barto <david@kdbarto.org> wrote:


> So even M$ sees that Linux could be a threat in the long term and is
> making an attempt to prevent it from harming the M$ universe.
>

MS is making a mint letting people run Linux on Azure rent-a-boxen, which
now runs on more Azure instances than Windows does.  They aren't anti-Linux
any more.  Indeed, they are contributing to the Linux kernel now to make it
run better on WSL.

See, you can run your Linux code on Windows, why do you want to
> run Linux now? 8^)s


I think the real motivation is to get developers off Macs.  Many people
(including me at my last two jobs) are developing on Mac OS and deploying
on Linux.  With WSL, developers can develop on Linux(-on-Windows) and
deploy on Linux (for reals).  This also makes corporate IT happy, because
they want all employees to have Windows, Office, etc. etc.



John Cowan          http://vrici.lojban.org/~cowan        cowan@ccil.org
The Penguin shall hunt and devour all that is crufty, gnarly and
bogacious; all code which wriggles like spaghetti, or is infested with
blighting creatures, or is bound by grave and perilous Licences shall it
capture.  And in capturing shall it replicate, and in replicating shall
it document, and in documentation shall it bring freedom, serenity and
most cool froodiness to the earth and all who code therein.  --Gospel of Tux

[-- Attachment #2: Type: text/html, Size: 3038 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-04 19:46       ` Clem Cole
@ 2021-07-05  1:33         ` Noel Hunt
  2021-07-05  2:38           ` Clem Cole
  2021-07-06  5:10           ` Nevin Liber
  0 siblings, 2 replies; 109+ messages in thread
From: Noel Hunt @ 2021-07-05  1:33 UTC (permalink / raw)
  To: Clem Cole; +Cc: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 561 bytes --]

> But note that when wnj wrote head(1), Joy followed the
> famous `Unix Philosophy' of  doing one (small) job
> well.   Which means he did not add a feature *i.e.*
> abusing, an old program, like cat(1), and add some new
> switch to it that that told the program stop outputting
> after n lines.  Instead Joy wrote a simple new tool.

He didn't need to abuse any existing program by adding new
flags or the like; unless I am mistaken, `sed Nq', for some
number `N', does exactly what `head -N' would do on a single
file, obviating the very need for head(1).

>

[-- Attachment #2: Type: text/html, Size: 1006 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view)
  2021-07-05  0:21       ` Larry McVoy
@ 2021-07-05  2:36         ` John Cowan
  2021-07-05  2:59           ` Richard Salz
  2021-07-05  3:47           ` Larry McVoy
  2021-07-05 12:11         ` Thomas Paulsen
  1 sibling, 2 replies; 109+ messages in thread
From: John Cowan @ 2021-07-05  2:36 UTC (permalink / raw)
  To: Larry McVoy; +Cc: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 2162 bytes --]

On Sun, Jul 4, 2021 at 8:22 PM Larry McVoy <lm@mcvoy.com> wrote:


> We both love C, we are both disciplined enough to write
> maintainable/extendable code in C, it works for us.  We really clicked
> over our love of C.
>

Are you really sure that all the C code the two of you have written in your
careers carefully avoids all 191 kinds of undefined behavior in C99 (the
number has grown since then)?  Give me leave to doubt it.

Consider this example from an earlier version of the Linux kernel:

static void __devexit agnx_pci_remove (struct pci_dev *pdev)
{
  struct ieee80211_hw *dev = pci_get_drvdata(pdev);
  struct agnx_priv *priv = dev->priv;

  if (!dev) return;
  ... do stuff using dev ...
}

The trouble here is that dev is used before it is checked for being a null
pointer.  In Java, you'd have a NullPointerException.  In C, "gcc -O2" will
make this analysis:

Case 1: dev == NULL.  This is undefined behavior and the compiler has no
obligations.

Case 2: dev != NULL.  Since dev can't be null (the compiler always assumes
the programmer did not intend to use UB), the check can be removed.

The result is that there is no check for NULL in either case, the compiler
is silent, and since this is the kernel, dereferencing NULL probably pwns
your system.  So whereas things that were technically UB used to work more
or less as you'd expect them to, nowadays they work as *nobody* expects
them to.

This is the result of the three values of C programmers:  (1) fast
execution, (2) fast compilation, (3) I lied; there is no (3).  And as
optimizers get better and better, the number of UB-related disasters
increases.  (The same is true for C++ compiler developers, except that they
sometimes prioritize (2) over (1) because benchmarks.)

> If I had infinite energy and money, I would fund a dialect of C that
> made it more useful.


See <https://blog.regehr.org/archives/1287>, about how an attempt to reduce
the amount of UB int C failed because nobody could agree with anybody else
on what changes to make.


> For all its faults, C++ is the closest to a modern language that sort
> of fits with what I want
>

C++, is it?  C++2011 has 203 UBs.

[-- Attachment #2: Type: text/html, Size: 5120 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-05  1:33         ` Noel Hunt
@ 2021-07-05  2:38           ` Clem Cole
  2021-07-05  2:51             ` Warner Losh
                               ` (2 more replies)
  2021-07-06  5:10           ` Nevin Liber
  1 sibling, 3 replies; 109+ messages in thread
From: Clem Cole @ 2021-07-05  2:38 UTC (permalink / raw)
  To: Noel Hunt; +Cc: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 859 bytes --]

Noel.  Pls check the TUHS archives and I think you will see sed does yet
exist in 6th edition when Joy wrote head in 1977.  Certainly not yet at
UCB.

On Sun, Jul 4, 2021 at 9:34 PM Noel Hunt <noel.hunt@gmail.com> wrote:

> > But note that when wnj wrote head(1), Joy followed the
> > famous `Unix Philosophy' of  doing one (small) job
> > well.   Which means he did not add a feature *i.e.*
> > abusing, an old program, like cat(1), and add some new
> > switch to it that that told the program stop outputting
> > after n lines.  Instead Joy wrote a simple new tool.
>
> He didn't need to abuse any existing program by adding new
> flags or the like; unless I am mistaken, `sed Nq', for some
> number `N', does exactly what `head -N' would do on a single
> file, obviating the very need for head(1).
>
>> --
Sent from a handheld expect more typos than usual

[-- Attachment #2: Type: text/html, Size: 1540 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-05  2:38           ` Clem Cole
@ 2021-07-05  2:51             ` Warner Losh
  2021-07-05  3:03               ` Clem Cole
  2021-07-05  3:01             ` Clem Cole
  2021-07-05  5:22             ` Noel Hunt
  2 siblings, 1 reply; 109+ messages in thread
From: Warner Losh @ 2021-07-05  2:51 UTC (permalink / raw)
  To: Clem Cole; +Cc: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 978 bytes --]

On Sun, Jul 4, 2021, 8:39 PM Clem Cole <clemc@ccc.com> wrote:

> Noel.  Pls check the TUHS archives and I think you will see sed does yet
> exist in 6th edition when Joy wrote head in 1977.  Certainly not yet at
> UCB.
>

V6 doesn't have sed.

Warner

On Sun, Jul 4, 2021 at 9:34 PM Noel Hunt <noel.hunt@gmail.com> wrote:
>
>> > But note that when wnj wrote head(1), Joy followed the
>> > famous `Unix Philosophy' of  doing one (small) job
>> > well.   Which means he did not add a feature *i.e.*
>> > abusing, an old program, like cat(1), and add some new
>> > switch to it that that told the program stop outputting
>> > after n lines.  Instead Joy wrote a simple new tool.
>>
>> He didn't need to abuse any existing program by adding new
>> flags or the like; unless I am mistaken, `sed Nq', for some
>> number `N', does exactly what `head -N' would do on a single
>> file, obviating the very need for head(1).
>>
>>> --
> Sent from a handheld expect more typos than usual
>

[-- Attachment #2: Type: text/html, Size: 2201 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view)
  2021-07-05  2:36         ` John Cowan
@ 2021-07-05  2:59           ` Richard Salz
  2021-07-05  3:47           ` Larry McVoy
  1 sibling, 0 replies; 109+ messages in thread
From: Richard Salz @ 2021-07-05  2:59 UTC (permalink / raw)
  To: John Cowan; +Cc: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 433 bytes --]

> Are you really sure that all the C code the two of you have written in
> your careers carefully avoids all 191 kinds of undefined behavior in C99
> (the number has grown since then)?  Give me leave to doubt it.
>
> Consider this example from an earlier version of the Linux kernel:
>
> static void __devexit agnx_pci_remove (struct pci_dev *pdev)
>

Bogus argument.  The Linux kernel is free to rely on specific compiler
behavior.

[-- Attachment #2: Type: text/html, Size: 1117 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-05  2:38           ` Clem Cole
  2021-07-05  2:51             ` Warner Losh
@ 2021-07-05  3:01             ` Clem Cole
  2021-07-05  5:22             ` Noel Hunt
  2 siblings, 0 replies; 109+ messages in thread
From: Clem Cole @ 2021-07-05  3:01 UTC (permalink / raw)
  To: Noel Hunt; +Cc: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 1032 bytes --]

Dyslexic typo.  Does NOT

On Sun, Jul 4, 2021 at 10:38 PM Clem Cole <clemc@ccc.com> wrote:

> Noel.  Pls check the TUHS archives and I think you will see sed does yet
> exist in 6th edition when Joy wrote head in 1977.  Certainly not yet at
> UCB.
>
> On Sun, Jul 4, 2021 at 9:34 PM Noel Hunt <noel.hunt@gmail.com> wrote:
>
>> > But note that when wnj wrote head(1), Joy followed the
>> > famous `Unix Philosophy' of  doing one (small) job
>> > well.   Which means he did not add a feature *i.e.*
>> > abusing, an old program, like cat(1), and add some new
>> > switch to it that that told the program stop outputting
>> > after n lines.  Instead Joy wrote a simple new tool.
>>
>> He didn't need to abuse any existing program by adding new
>> flags or the like; unless I am mistaken, `sed Nq', for some
>> number `N', does exactly what `head -N' would do on a single
>> file, obviating the very need for head(1).
>>
>>> --
> Sent from a handheld expect more typos than usual
>
-- 
Sent from a handheld expect more typos than usual

[-- Attachment #2: Type: text/html, Size: 2025 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-05  2:51             ` Warner Losh
@ 2021-07-05  3:03               ` Clem Cole
  0 siblings, 0 replies; 109+ messages in thread
From: Clem Cole @ 2021-07-05  3:03 UTC (permalink / raw)
  To: Warner Losh; +Cc: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 1387 bytes --]

Right.  A dyslexic typo sorry doesn’t

That was my point.  He needed a tool and didn’t have one so he wrote a
singular tool to solve the problem.  We now consider that tool pretty much
standard equipment

On Sun, Jul 4, 2021 at 10:51 PM Warner Losh <imp@bsdimp.com> wrote:

>
>
> On Sun, Jul 4, 2021, 8:39 PM Clem Cole <clemc@ccc.com> wrote:
>
>> Noel.  Pls check the TUHS archives and I think you will see sed does yet
>> exist in 6th edition when Joy wrote head in 1977.  Certainly not yet at
>> UCB.
>>
>
> V6 doesn't have sed.
>
> Warner
>
> On Sun, Jul 4, 2021 at 9:34 PM Noel Hunt <noel.hunt@gmail.com> wrote:
>>
>>> > But note that when wnj wrote head(1), Joy followed the
>>> > famous `Unix Philosophy' of  doing one (small) job
>>> > well.   Which means he did not add a feature *i.e.*
>>> > abusing, an old program, like cat(1), and add some new
>>> > switch to it that that told the program stop outputting
>>> > after n lines.  Instead Joy wrote a simple new tool.
>>>
>>> He didn't need to abuse any existing program by adding new
>>> flags or the like; unless I am mistaken, `sed Nq', for some
>>> number `N', does exactly what `head -N' would do on a single
>>> file, obviating the very need for head(1).
>>>
>>>> --
>> Sent from a handheld expect more typos than usual
>>
> --
Sent from a handheld expect more typos than usual

[-- Attachment #2: Type: text/html, Size: 2972 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view)
  2021-07-05  2:36         ` John Cowan
  2021-07-05  2:59           ` Richard Salz
@ 2021-07-05  3:47           ` Larry McVoy
  2021-07-05  4:02             ` Dan Stromberg
  1 sibling, 1 reply; 109+ messages in thread
From: Larry McVoy @ 2021-07-05  3:47 UTC (permalink / raw)
  To: John Cowan; +Cc: The Unix Heritage Society mailing list

On Sun, Jul 04, 2021 at 10:36:46PM -0400, John Cowan wrote:
> > We both love C, we are both disciplined enough to write
> > maintainable/extendable code in C, it works for us.  We really clicked
> > over our love of C.
> 
> Are you really sure that all the C code the two of you have written in your
> careers carefully avoids all 191 kinds of undefined behavior in C99 (the
> number has grown since then)?  Give me leave to doubt it.

I'm sure there are all sorts of problems with C.  Somehow we both never
encountered them.

My experience is that people come up with this or that, and they are
right, there is a there there, but Eric and I, he more than me, we just
wrote a ton of code in C, like 3-4 decades, all of that code works and
works better than the people who second guess us.  Our code works.

I've been through coverity trying to tell us where the problems were,
they found none.  I've been through Intel's fancy compiler that would
make things faster, it was a .1-.5%.  I don't remember, it was nothing.

I'm good with my understanding of C.  I get that you get it at a more
detailed level, go you.  I get it at a more basic practical level and
that has served me well.

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-03 13:20         ` Dan Cross
                             ` (2 preceding siblings ...)
  2021-07-04 20:10           ` Tony Finch
@ 2021-07-05  3:52           ` Bakul Shah
  3 siblings, 0 replies; 109+ messages in thread
From: Bakul Shah @ 2021-07-05  3:52 UTC (permalink / raw)
  To: Dan Cross; +Cc: The Unix Heritage Society mailing list


> On Jul 3, 2021, at 6:20 AM, Dan Cross <crossd@gmail.com> wrote:
> 
> Systemd is both good and bad.

Independent of systemd, the idea of regularizing service management is quite useful. A single function in a way, & fits the Unix Philosophy [which is really about factoring out common components and making them easily reusable. The fact that the shell allows that sort of composition with minimal syntax is, to me, one of its greatest contributions]

Similarly there are other such new functions where this philosophy can be applied.

> We aren't teaching, yes, but moreover they don't want to learn because the problems they're trying to solve are different. We need to accept and internalize that and think hard about how the tools we use can be brought to bear on the problems that are relevant _now_, not 30 years ago.
> 
> It does beg the question: is a Unix-style kernel still the appropriate foundation for this style of computing? Why are we still using this system: is it because of its intrinsic power and versatility, or because of inertia? Systemd, containers, etc, all suggest that our models are inadequate.


Containers, and many other tech. solutions have evolved in response to changing needs. The question that interests me is can we apply the Unix philosophy in this changing environment and if so how.

 One thing I like about Apple is their continuous focus  & s/w evolution on helping the end-user (we may disagree on specific changes but I bet many people on this list are using Apple devices as their "terminal"). It is a real joy to use some of these programs. And yet it is a real shame to not see open standards for such things. 


^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-04 20:10           ` Tony Finch
@ 2021-07-05  3:59             ` Theodore Ts'o
  2021-07-05 15:08               ` Steffen Nurpmeso
  0 siblings, 1 reply; 109+ messages in thread
From: Theodore Ts'o @ 2021-07-05  3:59 UTC (permalink / raw)
  To: Tony Finch; +Cc: tuhs

On Sun, Jul 04, 2021 at 09:10:49PM +0100, Tony Finch wrote:
> Dan Cross <crossd@gmail.com> wrote:
> >
> > Systemd is both good and bad.
> 
> I thought this article was well-informed and informative, but VERY long:
> https://blog.darknedgy.net/technology/2020/05/02/0/
> "systemd, 10 years later: a historical and technical retrospective"

This is also a really good talk, by Benno Rice.  Benno is a FreeBSD
developer, and has served on the FreeBSD Core Team.  He gave this talk
at Linux.Conf.au 2019, and it was a repeat of a talk he gave at BSDCan
2018, entitled "The Tragedy of Systemd" ("Tragedy" in the title was
used in the Greek drama context):

	https://www.youtube.com/watch?v=o_AIw9bGogo

It's a pretty fair talk about the why systemd came up, and what we
might be able to learn from systemd.  His closing line was, which I
think is quite good was:

   "What I would challenge every one here is to look at systemd, and
    try find one thing you like --- and then go try to implement it."

					- Ted

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view)
  2021-07-05  3:47           ` Larry McVoy
@ 2021-07-05  4:02             ` Dan Stromberg
  2021-07-05 13:45               ` Steffen Nurpmeso
  0 siblings, 1 reply; 109+ messages in thread
From: Dan Stromberg @ 2021-07-05  4:02 UTC (permalink / raw)
  To: Larry McVoy; +Cc: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 690 bytes --]

On Sun, Jul 4, 2021 at 8:49 PM Larry McVoy <lm@mcvoy.com> wrote:

> On Sun, Jul 04, 2021 at 10:36:46PM -0400, John Cowan wrote:
> > > We both love C, we are both disciplined enough to write
> > > maintainable/extendable code in C, it works for us.  We really clicked
> > > over our love of C.
> >
> > Are you really sure that all the C code the two of you have written in
> your
> > careers carefully avoids all 191 kinds of undefined behavior in C99 (the
> > number has grown since then)?  Give me leave to doubt it.
>
> I'm sure there are all sorts of problems with C.  Somehow we both never
> encountered them.
>

Actually, haven't there been many security holes discovered in sendmail?

[-- Attachment #2: Type: text/html, Size: 1095 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view)
  2021-07-04 23:24     ` [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view) Derek Fawcus
  2021-07-04 23:50       ` Nemo Nusquam
  2021-07-05  0:21       ` Larry McVoy
@ 2021-07-05  4:08       ` Dan Stromberg
  2021-07-05  4:23         ` George Michaelson
  2021-07-05 15:53       ` Mike Markowski
  2021-07-05 16:39       ` Warner Losh
  4 siblings, 1 reply; 109+ messages in thread
From: Dan Stromberg @ 2021-07-05  4:08 UTC (permalink / raw)
  To: Derek Fawcus; +Cc: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 1156 bytes --]

On Sun, Jul 4, 2021 at 4:33 PM Derek Fawcus <
dfawcus+lists-tuhs@employees.org> wrote:

> On Sat, Jul 03, 2021 at 05:49:57PM +0200, Andy Kosela wrote:
> > They also think that C is obsolete
>
> I'd not say it is obsolete, but despite it having been my main language
> for the
> last 30 years, these days I would be inclined to restrict the range of new
> tasks
> I'd use it for.
>
Me too - I write far fewer things in C today than I once did.  Last time I
used C, it was for something that needed mlockall() and very minimalist OS
dependencies.  And the client was in Python - only the server was in C.

I think Rust is probably the better language, even though Go is ahead of it
in the Tiobe language popularity rankings.  Go is more like what most
people are accustomed to, but Rust is a genuine advancement in language
design that justifies "fighting the borrow checker".

On the other hand, I did a little comparison of Rust md5 (I know, it's
broken) implementations, and found them surprisingly slow.  Slower than
CPython - though CPython is just linking to OpenSSL, and OpenSSL is
probably using hand-coded assembler, or at least highly optimized C.

[-- Attachment #2: Type: text/html, Size: 1624 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view)
  2021-07-05  4:08       ` Dan Stromberg
@ 2021-07-05  4:23         ` George Michaelson
  2021-07-05 14:43           ` Larry McVoy
  0 siblings, 1 reply; 109+ messages in thread
From: George Michaelson @ 2021-07-05  4:23 UTC (permalink / raw)
  Cc: The Unix Heritage Society mailing list

Forgive me a side note, but has it not been shown for some time that
apart from a very gifted few people, hand-crafted machine-code is
usually slower than the best optimising compilers these days? With out
of order instruction stuff, side effects (inter-core locking) cache
coherency &c it isn't hard to wind up using "simpler" machine code
which performs worserer.

Doesn't really alter the language debate, but it does go to "compilers
are pretty smart these days"

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-05  2:38           ` Clem Cole
  2021-07-05  2:51             ` Warner Losh
  2021-07-05  3:01             ` Clem Cole
@ 2021-07-05  5:22             ` Noel Hunt
  2 siblings, 0 replies; 109+ messages in thread
From: Noel Hunt @ 2021-07-05  5:22 UTC (permalink / raw)
  To: Clem Cole; +Cc: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 1125 bytes --]

Thank you for the correction. Having only used Unix from Seventh
Edition on, research-editions only, I have never used head(1) and
didn't realize it was written so early.


On Mon, Jul 5, 2021 at 12:38 PM Clem Cole <clemc@ccc.com> wrote:

> Noel.  Pls check the TUHS archives and I think you will see sed does yet
> exist in 6th edition when Joy wrote head in 1977.  Certainly not yet at
> UCB.
>
> On Sun, Jul 4, 2021 at 9:34 PM Noel Hunt <noel.hunt@gmail.com> wrote:
>
>> > But note that when wnj wrote head(1), Joy followed the
>> > famous `Unix Philosophy' of  doing one (small) job
>> > well.   Which means he did not add a feature *i.e.*
>> > abusing, an old program, like cat(1), and add some new
>> > switch to it that that told the program stop outputting
>> > after n lines.  Instead Joy wrote a simple new tool.
>>
>> He didn't need to abuse any existing program by adding new
>> flags or the like; unless I am mistaken, `sed Nq', for some
>> number `N', does exactly what `head -N' would do on a single
>> file, obviating the very need for head(1).
>>
>>> --
> Sent from a handheld expect more typos than usual
>

[-- Attachment #2: Type: text/html, Size: 2327 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-04 12:48             ` Dan Cross
@ 2021-07-05  7:14               ` Tomasz Rola
  2021-07-05 16:26                 ` John Cowan
  2021-07-06 16:05                 ` Clem Cole
  0 siblings, 2 replies; 109+ messages in thread
From: Tomasz Rola @ 2021-07-05  7:14 UTC (permalink / raw)
  To: Dan Cross; +Cc: tuhs

On Sun, Jul 04, 2021 at 08:48:23AM -0400, Dan Cross wrote:
[...]
> I didn't say that Unix is no longer relevant. I said that the problems that
> were prominent when Unix was built, and that thus shaped its architecture,
> are much _less_ relevant now than they were at the time. They still have
> some relevance, but are no longer primary for the vast majority of use
> cases Unix is used for. Further, the way people approach problems now has
> changed, even if they still generically assume "Unix" as the base they
> build on.

I have misread you then. I suppose the main problem that Unix solved
fifty years ago was the need for OS on a not very big (but cheap!)
machine. There were quite a few such systems and various such 16,
18-bit machines. So Unix was one of many, and what made it successful
was rewriting in C (I think I have read about it somewhere). Various
guys ported C compiler to their machines and to test it, they compiled
Unix sources (I am sure I have read about this). So as C gained
popularity, so did Unix gained popularity and C was the prog lang of
Unix, thus with every new Unix install C gained even more popularity.

So, looking from this perspective, maybe there was nothing
particularly special in Unix as such. It was just a double pump of
C-Unix, mutually pumping each other's success story.

I am not sure. I tried to find some time and install old OS on
simh/pdp11, yet there was always something more pressing to do. Some
alternatives to Unix, judging by their wikipedia descriptions, did not
convince me - like, one OS booted straight into debugger, if memory
serves. I do not think I would like that in everyday use. And after
reading about TECO, plenty of editors seem like better choice for me
:-).

Anyway, my own reasons to stick to Unix(-like) did not change. In
middle 1990-ties, after few years of playing with Amiga, and few years
of playing with SunOS/Solaris, it was clear to me what I wanted next
(Amiga was getting old rather quickly, as Commodore ran aground). The
OS was to be reliable, multitasking and if possible, cheap. If I was
to pay for something, it had to work, so Windows was ruled out. Later
on, as I hooked up to the net, security became important, too. Somehow
Linux on a 486 clone became a nice drop in replacement for Sunstation
and Solaris I used at the uni. I was able to run TeX on it, a
database, a program which abused database, an editor (Emacs, joe, vi)
with which I made the program and a compiler. All of this on eight
megs of ram. With "a bit" of swapping. That was terrific. To do same
thing on Windows, I imagined, at least Pentium and about 3x as much
ram was needed, plus a very expensive NT, because I did not expected
W95 could be relied upon.

If I still stick to Unix, it is because I still need something
dependable and allowing my various experiments or small time
developments. Windows somewhat improved during last twenty five years,
but not enough to pay for it - just MHO.

In other post you ask, why folks do not find Unix interesting
anymore. I still suggest they are following the money. They are the
kind of folk who never would find Unix interesting enough based on
merits only. Asking about their choices leads us nowwhere, because
their choices are not based on technical criteria.

What I would like to ask instead, is how many people had been using
Unix and decided to drop it, because it no longer worked for them. But
not because they had to earn for the living in a World of Windoze,
only because Unix tools stopped doing the job and could not be helped
with some handmade script or program. I guess reading more about such
users could tell something about Unix future or current relevance.

Also, reading about why people keep using Unix. Myself, I like doing
various things with computer. I create my own needs and then I write
small pieces of code to help myself out of the trap. Or something like
this. Nowadays, a lot of those needs revolve around learning and
organising notes. Of course I could not be using specialised note
taking program. Instead, I went with Emacs and org-mode. In the
process I had to learn a bit of Elisp and dot-emacs file. Some
defaults in Emacs are not comfy for my eyes - fonts, colors, it had to
be fine tuned to my liking.

So, I think my case could be summed up so: I need dependable software
and I have some needs which I know how to satisfy using this
software. If the software alone does not suit me, I can make some of
lacking elements myself and Unix allows me to join existing and new
elements easily.

I wonder if other Unix (ab)users share something with me? Like,
specialised single-person needs, or putting together building blocks
of command line tools, or preference for terminal based software
(because it works more often than not)? I guess yes.

What do people indifferent to Unix share with themselves, other than
general lack of interest, unwillingness to learn awk, disregard to
keeping information in text based, line oriented files?

HTH, sorry for long email, I tried to trim it but had to left
something too :-).

-- 
Regards,
Tomasz Rola

--
** A C programmer asked whether computer had Buddha's nature.      **
** As the answer, master did "rm -rif" on the programmer's home    **
** directory. And then the C programmer became enlightened...      **
**                                                                 **
** Tomasz Rola          mailto:tomasz_rola@bigfoot.com             **

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view)
  2021-07-05  0:21       ` Larry McVoy
  2021-07-05  2:36         ` John Cowan
@ 2021-07-05 12:11         ` Thomas Paulsen
  1 sibling, 0 replies; 109+ messages in thread
From: Thomas Paulsen @ 2021-07-05 12:11 UTC (permalink / raw)
  To: Larry McVoy; +Cc: tuhs

> So why invent a new syntax and invalidate all of that knowledge that everyone has?
I subscribe 100% to your statement!

Well I think it's enough to add new concepts like modules/packages/classes to C, and a jdk a-like all purposes library, as libc isn't enough for the 21ths century.

I still love C because I'm able of writing small and very fast programs. go is cool to, but a memory hunk compared to C. The rust hype is declining since 2 years and java is the new cobol, ie. it's good for business oriented applications, whereas C and go are (beside ASM) the only choices for system oriented people. Has C a future? Yes, it already has just begun. 



^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view)
  2021-07-05  4:02             ` Dan Stromberg
@ 2021-07-05 13:45               ` Steffen Nurpmeso
  2021-07-05 20:15                 ` Dan Stromberg
  0 siblings, 1 reply; 109+ messages in thread
From: Steffen Nurpmeso @ 2021-07-05 13:45 UTC (permalink / raw)
  To: Dan Stromberg; +Cc: The Unix Heritage Society mailing list

Dan Stromberg wrote in
 <CAGGBd_o=_LhF8FtxNF+VrC6TTN6mKZMx3Li4-TCA8qiK45BPBQ@mail.gmail.com>:
 |On Sun, Jul 4, 2021 at 8:49 PM Larry McVoy <lm@mcvoy.com> wrote:
 |> On Sun, Jul 04, 2021 at 10:36:46PM -0400, John Cowan wrote:
 |>>> We both love C, we are both disciplined enough to write
 |>>> maintainable/extendable code in C, it works for us.  We really clicked
 |>>> over our love of C.
 |>>
 |>> Are you really sure that all the C code the two of you have written in
 |> your
 |>> careers carefully avoids all 191 kinds of undefined behavior in C99 (the
 |>> number has grown since then)?  Give me leave to doubt it.
 |>
 |> I'm sure there are all sorts of problems with C.  Somehow we both never
 |> encountered them.
 |>
 |
 |Actually, haven't there been many security holes discovered in sendmail?

Even more actually that is a completely unfair statement.
sendmail grew out of other software that is way over 40 years as
of this writing, and to the best of my knowledge.  It was not
written to face a vicious and hostile environment at first, even
the protocol it serves evolved over time, as anything else
surrounding it.
And then, what have the holes to do with the programming language?

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view)
  2021-07-05  4:23         ` George Michaelson
@ 2021-07-05 14:43           ` Larry McVoy
  2021-07-05 15:17             ` Steffen Nurpmeso
  0 siblings, 1 reply; 109+ messages in thread
From: Larry McVoy @ 2021-07-05 14:43 UTC (permalink / raw)
  To: George Michaelson; +Cc: The Unix Heritage Society mailing list

On Mon, Jul 05, 2021 at 02:23:36PM +1000, George Michaelson wrote:
> Forgive me a side note, but has it not been shown for some time that
> apart from a very gifted few people, hand-crafted machine-code is
> usually slower than the best optimising compilers these days? With out
> of order instruction stuff, side effects (inter-core locking) cache
> coherency &c it isn't hard to wind up using "simpler" machine code
> which performs worserer.

I dunno where my team sat on the "gifted" scale, I like to think they
were pretty good.  We ran our code through Intel's fancy C compiler and
it made less than a 1% difference vs GCC.  We cared about performance
and had already done the by hand work to make the critical paths go fast.

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-05  3:59             ` Theodore Ts'o
@ 2021-07-05 15:08               ` Steffen Nurpmeso
  0 siblings, 0 replies; 109+ messages in thread
From: Steffen Nurpmeso @ 2021-07-05 15:08 UTC (permalink / raw)
  To: Theodore Ts'o; +Cc: tuhs

Theodore Ts'o wrote in
 <YOKDovR8h8r1yoIP@mit.edu>:
 |On Sun, Jul 04, 2021 at 09:10:49PM +0100, Tony Finch wrote:
 |> Dan Cross <crossd@gmail.com> wrote:
 |>>
 |>> Systemd is both good and bad.
 |> 
 |> I thought this article was well-informed and informative, but VERY long:
 |> https://blog.darknedgy.net/technology/2020/05/02/0/
 |> "systemd, 10 years later: a historical and technical retrospective"
 |
 |This is also a really good talk, by Benno Rice.  Benno is a FreeBSD
 |developer, and has served on the FreeBSD Core Team.  He gave this talk
 |at Linux.Conf.au 2019, and it was a repeat of a talk he gave at BSDCan
 |2018, entitled "The Tragedy of Systemd" ("Tragedy" in the title was
 |used in the Greek drama context):
 |
 | https://www.youtube.com/watch?v=o_AIw9bGogo
 |
 |It's a pretty fair talk about the why systemd came up, and what we
 |might be able to learn from systemd.  His closing line was, which I
 |think is quite good was:
 |
 |   "What I would challenge every one here is to look at systemd, and
 |    try find one thing you like --- and then go try to implement it."

Disclaimer: i .. have not .. seen this.

Everybody may use systemd until they die, my very personal concern
is only that the infrastructure boils down to be unusable without
it.  Also as noone cares, just a few voices here and there, even
more so.  Many small tools exist that can do things, but as
systemd grows they do not, they are not extended to follow suit
Linux kernel features.

I mean, i could hack instead of complaining, but as i use
unshare(1) not docker/systemd i find it unpleasant that i have to
use capsh(1) in addition for example instead of simply feeding
unshare(1) also with the capabilities.  For systemd users this is
just a single line, and often programs come with readily prepared
unit files, take iwd for example

  [Service]
  Type=dbus
  BusName=net.connman.iwd
  ExecStart=@libexecdir@/iwd
[taken from source not installed base as not installed, no
systemd here.]
  NotifyAccess=main
  LimitNPROC=1

I have cgroups yes, but unshare(1) does not do that by itself.
So my shell script wrapper moves the PID by itself, which it is,
essentially.

  Restart=on-failure

I have a cron based watchdog on the server.  (Which never had to
trigger in >5 years of operation.)  But yes...

  CapabilityBoundingSet=CAP_NET_ADMIN CAP_NET_RAW

Not unshare.  I am about to add this via capsh(1).

  PrivateTmp=true

So private tmp directories only via boxing into overlayfs views,
but not as it is done here.

  NoNewPrivileges=true

Capabilities not yet.
Maybe one should go and write an unshare which can this too.

  DevicePolicy=closed
  DeviceAllow=/dev/rfkill rw
  ProtectHome=yes
  ProtectSystem=strict
  ProtectControlGroups=yes
  ProtectKernelModules=yes

You see systemd making touchdowns, and i mean it.
Because only it can.  And i played football with my feet.

  ConfigurationDirectory=iwd
  StateDirectory=iwd
  StateDirectoryMode=0700

Granted, all these tortured administrators have a life way easier
by using systemd and just adjusting the unit, if at all needed.
Easier to just browse and copy+paste.

I have to take back my iwd complaint about syslog.  It is just
they did not bother at all.  This Intel program uses the Intel
"ell" (Embedded Linux library), and whereas that offers

  ell/log.c: * l_log_set_ident:
  ell/log.c:LIB_EXPORT void l_log_set_ident(const char *ident)
  ell/log.c: * l_log_set_handler:
  ell/log.c:LIB_EXPORT void l_log_set_handler(l_log_func_t function)
  ell/log.c: * l_log_set_null:
  ell/log.c:LIB_EXPORT void l_log_set_null(void)
  ell/log.c: * l_log_set_stderr:
  ell/log.c:LIB_EXPORT void l_log_set_stderr(void)
  ell/log.c: * l_log_set_syslog:
  ell/log.c:LIB_EXPORT void l_log_set_syslog(void)
  ell/log.c: * l_log_set_journal:
  ell/log.c:LIB_EXPORT void l_log_set_journal(void)
  ell/test.c:     l_log_set_stderr();
  ell/log.h:void l_log_set_ident(const char *ident);
  ell/log.h:void l_log_set_handler(l_log_func_t function);
  ell/log.h:void l_log_set_null(void);
  ell/log.h:void l_log_set_stderr(void);
  ell/log.h:void l_log_set_syslog(void);
  ell/log.h:void l_log_set_journal(void);

they use

  #?0|kent:iwd-1.15$ grep -r l_log_set
  tools/hwsim.c:  l_log_set_stderr();
  tools/probe-req.c:      l_log_set_stderr();
  client/main.c:  l_log_set_stderr();
  src/main.c:     l_log_set_stderr();
  wired/main.c:   l_log_set_stderr();

One could at least be happy they did not fixate "journal", maybe.
Anyhow, the actual error output is for developers only.

Bitrot everywhere.  So just jump on the train that delivers and
pass the rest.

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view)
  2021-07-05 14:43           ` Larry McVoy
@ 2021-07-05 15:17             ` Steffen Nurpmeso
  2021-07-05 15:36               ` Steffen Nurpmeso
  0 siblings, 1 reply; 109+ messages in thread
From: Steffen Nurpmeso @ 2021-07-05 15:17 UTC (permalink / raw)
  To: George Michaelson; +Cc: The Unix Heritage Society mailing list

Larry McVoy wrote in
 <20210705144314.GV817@mcvoy.com>:
 |On Mon, Jul 05, 2021 at 02:23:36PM +1000, George Michaelson wrote:
 |> Forgive me a side note, but has it not been shown for some time that
 |> apart from a very gifted few people, hand-crafted machine-code is
 |> usually slower than the best optimising compilers these days? With out

Some *BSDs moved from such to plain C code for strlen() for
example, a couple of years back, yes.

 |> of order instruction stuff, side effects (inter-core locking) cache
 |> coherency &c it isn't hard to wind up using "simpler" machine code
 |> which performs worserer.
 |
 |I dunno where my team sat on the "gifted" scale, I like to think they
 |were pretty good.  We ran our code through Intel's fancy C compiler and
 |it made less than a 1% difference vs GCC.  We cared about performance
 |and had already done the by hand work to make the critical paths go fast.

Funnily just a couple of months ago at least FreeBSD (but i think
also DragonFly BSD, a tad different) moved back, on at least
x86_64.

(I personally no longer look nor care that much, as (a) i cannot
help it anyway, (b) there are so many CPU etc. models out there.
Why try to go backward because Cyrix is so good there, use code
alignment of X for processor A and Y for processor B?  That is
just a tremendous maintenance mess, and that for free and with so
few time.  No.  Fun fact is that my one NVME disc can read/write
1.3 gigabytes per second doing "btrfs scrub", that is i think at
least four times the speed of the main memory the mentioned Cyrix
had available.)

 --End of <20210705144314.GV817@mcvoy.com>

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view)
  2021-07-05 15:17             ` Steffen Nurpmeso
@ 2021-07-05 15:36               ` Steffen Nurpmeso
  0 siblings, 0 replies; 109+ messages in thread
From: Steffen Nurpmeso @ 2021-07-05 15:36 UTC (permalink / raw)
  To: tuhs

Steffen Nurpmeso wrote in
 <20210705151732.6PbNl%steffen@sdaoden.eu>:
 |Larry McVoy wrote in
 | <20210705144314.GV817@mcvoy.com>:

Just to mention that i did address Larry McVoy, no, as can be seen
in the Mail-Followup-To: header.  I do not know why this Python
program removes him from Cc: just because it strips his name from
its actual to-be-send-to list, likely to "avoid duplicates" or
something, what i think what happens here.  Very unkind.  Not me.

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view)
  2021-07-04 23:24     ` [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view) Derek Fawcus
                         ` (2 preceding siblings ...)
  2021-07-05  4:08       ` Dan Stromberg
@ 2021-07-05 15:53       ` Mike Markowski
  2021-07-05 16:39       ` Warner Losh
  4 siblings, 0 replies; 109+ messages in thread
From: Mike Markowski @ 2021-07-05 15:53 UTC (permalink / raw)
  To: tuhs

On 7/4/21 7:24 PM, Derek Fawcus wrote:
> On Sat, Jul 03, 2021 at 05:49:57PM +0200, Andy Kosela wrote:
>> They also think that C is obsolete
> 
> I'd not say it is obsolete, but despite it having been my main
> language for the last 30 years, these days I would be inclined to
> restrict the range of new tasks I'd use it for.

Hoping to correctly recall computability theory, a language needs only 
statements

S := 0
S := S + 1
if S == 0 goto Label

to be Turing complete.  It doesn't mean the language is always pleasant 
to use, but is enough - just as an editor with only char ins/del is 
technically enough.  C has its place, and so do tools for other classes 
of problems.

I enjoy python for signal processing work.  Matrices, complex numbers, 
many libraries are all there.  Libraries even parcel out calcs to cpu 
cores behind the scenes, letting the DSP guy worry (mostly) only about 
math.  And then C is speedy for the end product.  As comp sci evolves, 
we get more blades in our Swiss Army knife.

Mike Markowski

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-05  7:14               ` Tomasz Rola
@ 2021-07-05 16:26                 ` John Cowan
  2021-07-06 23:17                   ` Tomasz Rola
  2021-07-06 16:05                 ` Clem Cole
  1 sibling, 1 reply; 109+ messages in thread
From: John Cowan @ 2021-07-05 16:26 UTC (permalink / raw)
  To: Tomasz Rola; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 3295 bytes --]

On Mon, Jul 5, 2021 at 3:15 AM Tomasz Rola <rtomek@ceti.pl> wrote:


> So, looking from this perspective, maybe there was nothing
> particularly special in Unix as such. It was just a double pump of
> C-Unix, mutually pumping each other's success story.
>

I think there is more to it than that.  See <
http://www.catb.org/~esr/writings/unix-koans/zealot.html>.

I am not sure. I tried to find some time and install old OS on
> simh/pdp11, yet there was always something more pressing to do. Some
> alternatives to Unix, judging by their wikipedia descriptions, did not
> convince me - like, one OS booted straight into debugger, if memory
> serves.


ITS, yes.  But the debugger was not just a debugger, it was also a general
command-line interpreter, a shell in modern terms.  So while it is possible
to debug an empty memory into doing whatever you want, it is also possible
to run "advent", aka Colossal Cave Adventure.

> And after
> reading about TECO, plenty of editors seem like better choice for me
> :-).
>

I switched from Teco to ex at some point, and never went either forward or
back.  (Occasionally I drop into vi mode for things like parenthesis
checking.)

By the way, did anyone else start out on Unix-alikes before using actual
Unix?  I read the BSTJ issue and became an instant convert, but only when
$EMPLOYER got a MicroVAX was I able to work like that.  Next came the MKS
Toolkit on DOS, and then Cygwin for many years.  I suppose it's only my
last two $EMPLOYERs standardizing on the Mac that has left me running,
like, actual Unix.

If I still stick to Unix, it is because I still need something
> dependable and allowing my various experiments or small time
> developments.
>

"Computers are the greatest set of electric trains in the world."


>  I still suggest they are following the money. They are the
> kind of folk who never would find Unix interesting enough based on
> merits only. Asking about their choices leads us nowwhere, because
> their choices are not based on technical criteria.
>

True.  But then, many of us geeks make our choices not on technical
criteria but on tribal loyalty.  Which is *technically* superior, vi or
emacs?  (Please don't answer that.)

> Of course I could not be using specialised note
> taking program. Instead, I went with Emacs and org-mode. In the
> process I had to learn a bit of Elisp and dot-emacs file. Some
> defaults in Emacs are not comfy for my eyes - fonts, colors, it had to
> be fine tuned to my liking.
>

Note that Emacs is probably the oldest import into the Unix ecosystem from
outside, and it bears the marks of its origin: monolithic (but
programmable), one tool does it all.


> I wonder if other Unix (ab)users share something with me? Like,
> specialised single-person needs, or putting together building blocks
> of command line tools, or preference for terminal based software
> (because it works more often than not)?
>

Without doubt.  I am not loyal to a kernel or a set of utilities, I simply
follow the Way of Unix: <http://vrici.lojban.org/~cowan/upc/> (sadly
incomplete)



John Cowan          http://vrici.lojban.org/~cowan        cowan@ccil.org
"Hacking is the true football."  --F.W. Campbell (1863) in response to a
successful attempt to ban shin-kicking from soccer.  Today, it's biting.

[-- Attachment #2: Type: text/html, Size: 7158 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view)
  2021-07-04 23:24     ` [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view) Derek Fawcus
                         ` (3 preceding siblings ...)
  2021-07-05 15:53       ` Mike Markowski
@ 2021-07-05 16:39       ` Warner Losh
  2021-07-05 19:02         ` Clem Cole
  4 siblings, 1 reply; 109+ messages in thread
From: Warner Losh @ 2021-07-05 16:39 UTC (permalink / raw)
  To: Derek Fawcus; +Cc: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 244 bytes --]

On Sat, Jul 03, 2021 at 05:49:57PM +0200, Andy Kosela wrote:
> > They also think that C is obsolete
>

Hammers are not obsolete because we have screws, even though they
are mostly useless with screws. There's plenty of nails out there.

Warner

[-- Attachment #2: Type: text/html, Size: 532 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view)
  2021-07-05 16:39       ` Warner Losh
@ 2021-07-05 19:02         ` Clem Cole
  0 siblings, 0 replies; 109+ messages in thread
From: Clem Cole @ 2021-07-05 19:02 UTC (permalink / raw)
  To: Warner Losh; +Cc: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 444 bytes --]

On Mon, Jul 5, 2021 at 12:40 PM Warner Losh <imp@bsdimp.com> wrote:

>
>
> On Sat, Jul 03, 2021 at 05:49:57PM +0200, Andy Kosela wrote:
>> > They also think that C is obsolete
>>
>
> Hammers are not obsolete because we have screws, even though they
> are mostly useless with screws. There's plenty of nails out there.
>
Amen...

FWIW: I wrote this a few years ago:
https://www.quora.com/Is-Fortran-obsolete/answer/Clem-Cole[-- Attachment #2: Type: text/html, Size: 1702 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view)
  2021-07-05 13:45               ` Steffen Nurpmeso
@ 2021-07-05 20:15                 ` Dan Stromberg
  2021-07-05 21:05                   ` Larry McVoy
  2021-07-05 21:29                   ` Clem Cole
  0 siblings, 2 replies; 109+ messages in thread
From: Dan Stromberg @ 2021-07-05 20:15 UTC (permalink / raw)
  To: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 260 bytes --]

On Mon, Jul 5, 2021 at 6:45 AM Steffen Nurpmeso <steffen@sdaoden.eu> wrote:

>  And then, what have the holes to do with the programming language?
>
> Really?

A null-terminated array of char is a petri dish.  A proper string type is
more like a disinfectant.

[-- Attachment #2: Type: text/html, Size: 617 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view)
  2021-07-05 20:15                 ` Dan Stromberg
@ 2021-07-05 21:05                   ` Larry McVoy
  2021-07-05 21:29                   ` Clem Cole
  1 sibling, 0 replies; 109+ messages in thread
From: Larry McVoy @ 2021-07-05 21:05 UTC (permalink / raw)
  To: Dan Stromberg; +Cc: The Unix Heritage Society mailing list

On Mon, Jul 05, 2021 at 01:15:10PM -0700, Dan Stromberg wrote:
> On Mon, Jul 5, 2021 at 6:45 AM Steffen Nurpmeso <steffen@sdaoden.eu> wrote:
> 
> >  And then, what have the holes to do with the programming language?
> >
> > Really?
> 
> A null-terminated array of char is a petri dish.  A proper string type is
> more like a disinfectant.

I'd really like it if C got a proper everything type.  Some container
type that lets you have arrays of anything and the run time system does
the realloc when you extend the container.  We built such a thing in
BitKeeper but it was awkward, any sort of modification was

	thing = modifies(thing, args...);

because the modification might realloc().
-- 
---
Larry McVoy            	     lm at mcvoy.com             http://www.mcvoy.com/lm 

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view)
  2021-07-05 20:15                 ` Dan Stromberg
  2021-07-05 21:05                   ` Larry McVoy
@ 2021-07-05 21:29                   ` Clem Cole
  2021-07-05 22:22                     ` Brantley Coile
  2021-07-06  4:35                     ` Dan Stromberg
  1 sibling, 2 replies; 109+ messages in thread
From: Clem Cole @ 2021-07-05 21:29 UTC (permalink / raw)
  To: Dan Stromberg; +Cc: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 5336 bytes --]

On Mon, Jul 5, 2021 at 4:16 PM Dan Stromberg <drsalists@gmail.com> wrote:

> A null-terminated array of char is a petri dish.  A proper string type is
> more like a disinfectant.
>
Hrrmpt.... maybe (in theory), but I can say that never seen it really work
in practice --  bwk in Why Pascal is Not My Favorite Programming Language
<http://www.lysator.liu.se/c/bwk-on-pascal.html> describes much of the
practical realities of this sort of choice:

2.1.  The size of an array is part of its typeIf one declares

     var     arr10 : array [1..10] of integer;
             arr20 : array [1..20] of integer;

then arr10 and arr20 are arrays of 10 and 20 integers respectively.  Suppose
we want to write a procedure 'sort' to sort an integer array.  Because
arr10 and arr20 have different types, it is not possible to write a single
procedure that will sort them both.

The place where this affects Software Tools particularly, and I think
programs in general, is that it makes it difficult indeed to create a
library of routines for doing common, general-purpose operations like
sorting.

The particular data type most often affected is 'array of char', for in
Pascal a string is an array of characters.  Consider writing a function
'index(s,c)' that will return the position in the string s where the
character c first occurs, or zero if it does not.  The problem is how to
handle the string argument of 'index'.  The calls 'index('hello',c)' and '
index('goodbye',c)' cannot both be legal, since the strings have different
lengths.  (I pass over the question of how the end of a constant string
like 'hello' can be detected, because it can't.) The next try is

     var     temp : array [1..10] of char;
     temp := 'hello';

     n := index(temp,c);

but the assignment to 'temp' is illegal because 'hello' and 'temp' are of
different lengths.

The only escape from this infinite regress is to define a family of
routines with a member for each possible string size, or to make all
strings (including constant strings like 'define' ) of the same length.

The latter approach is the lesser of two great evils.  In 'Tools', a type
called 'string' is declared as

     type    string = array [1..MAXSTR] of char;

where the constant 'MAXSTR' is ``big enough,'' and all strings in all
programs are exactly this size.  This is far from ideal, although it made
it possible to get the programs running.  It does not solve the problem of
creating true libraries of useful routines.

There are some situations where it is simply not acceptable to use the
fixed-size array representation.  For example, the 'Tools' program to sort
lines of text operates by filling up memory with as many lines as will fit;
its running time depends strongly on how full the memory can be packed.

Thus for 'sort', another representation is used, a long array of characters
and a set of indices into this array:

     type    charbuf = array [1..MAXBUF] of char;
             charindex = array [1..MAXINDEX] of 0..MAXBUF;

But the procedures and functions written to process the fixed-length
representation cannot be used with the variable-length form; an entirely
new set of routines is needed to copy and compare strings in this
representation.  In Fortran or C the same functions could be used for both.

As suggested above, a constant string is written as

     'this is a string'

and has the type 'packed array [1..n] of char', where n is the length.  Thus
each string literal of different length has a different type.  The only way
to write a routine that will print a message and clean up is to pad all
messages out to the same maximum length:

     error('short message                    ');
     error('this is a somewhat longer message');

Many commercial Pascal compilers provide a 'string' data type that
explicitly avoids the problem; 'string's are all taken to be the same type
regardless of size.  This solves the problem for this single data type, but
no other.  It also fails to solve secondary problems like computing the
length of a constant string; another built-in function is the usual
solution.

Pascal enthusiasts often claim that to cope with the array-size problem one
merely has to copy some library routine and fill in the parameters for the
program at hand, but the defense sounds weak at best:(12
<http://www.lysator.liu.se/c/bwk-on-pascal.html#lit-12>)

``Since the bounds of an array are part of its type (or, more exactly, of
the type of its indexes), it is impossible to define a procedure or
function which applies to arrays with differing bounds.  Although this
restriction may appear to be a severe one, the experiences we have had with
Pascal tend to show that it tends to occur very infrequently.  [...]
However, the need to bind the size of parametric arrays is a serious defect
in connection with the use of program libraries.''

This botch is the biggest single problem with Pascal.  I believe that if it
could be fixed, the language would be an order of magnitude more usable.  The
proposed ISO standard for Pascal(13
<http://www.lysator.liu.se/c/bwk-on-pascal.html#lit-13>) provides such a
fix (``conformant array schemas''), but the acceptance of this part of the
standard is apparently still in doubt.
ᐧ

[-- Attachment #2: Type: text/html, Size: 10704 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view)
  2021-07-05 21:29                   ` Clem Cole
@ 2021-07-05 22:22                     ` Brantley Coile
  2021-07-06  4:35                     ` Dan Stromberg
  1 sibling, 0 replies; 109+ messages in thread
From: Brantley Coile @ 2021-07-05 22:22 UTC (permalink / raw)
  To: Clem Cole; +Cc: The Unix Heritage Society mailing list

Wirth fixed it in Modula-2 and Oberon.

> On Jul 5, 2021, at 5:29 PM, Clem Cole <clemc@ccc.com> wrote:
> 
> 
> 
> On Mon, Jul 5, 2021 at 4:16 PM Dan Stromberg <drsalists@gmail.com> wrote:
> A null-terminated array of char is a petri dish.  A proper string type is more like a disinfectant.
> Hrrmpt.... maybe (in theory), but I can say that never seen it really work in practice --  bwk in Why Pascal is Not My Favorite Programming Language describes much of the practical realities of this sort of choice:
> 
> 2.1.  The size of an array is part of its type
> 
> If one declares
>      var     arr10 : array [1..10] of integer;
>              arr20 : array [1..20] of integer;
> 
> then arr10 and arr20 are arrays of 10 and 20 integers respectively.  Suppose we want to write a procedure 'sort' to sort an integer array.  Because arr10 and arr20 have different types, it is not possible to write a single procedure that will sort them both.
> The place where this affects Software Tools particularly, and I think programs in general, is that it makes it difficult indeed to create a library of routines for doing common, general-purpose operations like sorting.
> 
> The particular data type most often affected is 'array of char', for in Pascal a string is an array of characters.  Consider writing a function 'index(s,c)' that will return the position in the string s where the character c first occurs, or zero if it does not.  The problem is how to handle the string argument of 'index'.  The calls 'index('hello',c)' and 'index('goodbye',c)' cannot both be legal, since the strings have different lengths.  (I pass over the question of how the end of a constant string like 'hello' can be detected, because it can't.) The next try is
> 
>      var     temp : array [1..10] of char;
>      temp := 'hello';
> 
>      n := index(temp,c);
> 
> but the assignment to 'temp' is illegal because 'hello' and 'temp' are of different lengths.
> The only escape from this infinite regress is to define a family of routines with a member for each possible string size, or to make all strings (including constant strings like 'define' ) of the same length.
> 
> The latter approach is the lesser of two great evils.  In 'Tools', a type called 'string' is declared as
> 
>      type    string = array [1..MAXSTR] of char;
> 
> where the constant 'MAXSTR' is ``big enough,'' and all strings in all programs are exactly this size.  This is far from ideal, although it made it possible to get the programs running.  It does not solve the problem of creating true libraries of useful routines.
> There are some situations where it is simply not acceptable to use the fixed-size array representation.  For example, the 'Tools' program to sort lines of text operates by filling up memory with as many lines as will fit; its running time depends strongly on how full the memory can be packed.
> 
> Thus for 'sort', another representation is used, a long array of characters and a set of indices into this array:
> 
>      type    charbuf = array [1..MAXBUF] of char;
>              charindex = array [1..MAXINDEX] of 0..MAXBUF;
> 
> But the procedures and functions written to process the fixed-length representation cannot be used with the variable-length form; an entirely new set of routines is needed to copy and compare strings in this representation.  In Fortran or C the same functions could be used for both.
> As suggested above, a constant string is written as
> 
>      'this is a string'
> 
> and has the type 'packed array [1..n] of char', where n is the length.  Thus each string literal of different length has a different type.  The only way to write a routine that will print a message and clean up is to pad all messages out to the same maximum length:
>      error('short message                    ');
>      error('this is a somewhat longer message');
> 
> Many commercial Pascal compilers provide a 'string' data type that explicitly avoids the problem; 'string's are all taken to be the same type regardless of size.  This solves the problem for this single data type, but no other.  It also fails to solve secondary problems like computing the length of a constant string; another built-in function is the usual solution.
> Pascal enthusiasts often claim that to cope with the array-size problem one merely has to copy some library routine and fill in the parameters for the program at hand, but the defense sounds weak at best:(12)
> 
> ``Since the bounds of an array are part of its type (or, more exactly, of the type of its indexes), it is impossible to define a procedure or function which applies to arrays with differing bounds.  Although this restriction may appear to be a severe one, the experiences we have had with Pascal tend to show that it tends to occur very infrequently.  [...] However, the need to bind the size of parametric arrays is a serious defect in connection with the use of program libraries.''
> This botch is the biggest single problem with Pascal.  I believe that if it could be fixed, the language would be an order of magnitude more usable.  The proposed ISO standard for Pascal(13) provides such a fix (``conformant array schemas''), but the acceptance of this part of the standard is apparently still in doubt. 
> ᐧ


^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view)
  2021-07-05 21:29                   ` Clem Cole
  2021-07-05 22:22                     ` Brantley Coile
@ 2021-07-06  4:35                     ` Dan Stromberg
  2021-07-06  4:44                       ` Warner Losh
                                         ` (2 more replies)
  1 sibling, 3 replies; 109+ messages in thread
From: Dan Stromberg @ 2021-07-06  4:35 UTC (permalink / raw)
  To: Clem Cole; +Cc: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 690 bytes --]

On Mon, Jul 5, 2021 at 2:29 PM Clem Cole <clemc@ccc.com> wrote:

>
> On Mon, Jul 5, 2021 at 4:16 PM Dan Stromberg <drsalists@gmail.com> wrote:
>
>> A null-terminated array of char is a petri dish.  A proper string type is
>> more like a disinfectant.
>>
> Hrrmpt.... maybe (in theory), but I can say that never seen it really work
> in practice --  bwk in Why Pascal is Not My Favorite Programming Language
> <http://www.lysator.liu.se/c/bwk-on-pascal.html> describes much of the
> practical realities of this sort of choice:
>

I think language designers since Pascal have learned from Pascal's mistake
there.

Supposedly even Borland's TurboPascal had better strings than vanilla
Pascal.

[-- Attachment #2: Type: text/html, Size: 1538 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view)
  2021-07-06  4:35                     ` Dan Stromberg
@ 2021-07-06  4:44                       ` Warner Losh
  2021-07-06  5:58                       ` Rico Pajarola
  2021-07-06 13:05                       ` Clem Cole
  2 siblings, 0 replies; 109+ messages in thread
From: Warner Losh @ 2021-07-06  4:44 UTC (permalink / raw)
  To: Dan Stromberg; +Cc: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 998 bytes --]

On Mon, Jul 5, 2021, 10:37 PM Dan Stromberg <drsalists@gmail.com> wrote:

>
> On Mon, Jul 5, 2021 at 2:29 PM Clem Cole <clemc@ccc.com> wrote:
>
>>
>> On Mon, Jul 5, 2021 at 4:16 PM Dan Stromberg <drsalists@gmail.com> wrote:
>>
>>> A null-terminated array of char is a petri dish.  A proper string type
>>> is more like a disinfectant.
>>>
>> Hrrmpt.... maybe (in theory), but I can say that never seen it really
>> work in practice --  bwk in Why Pascal is Not My Favorite Programming
>> Language <http://www.lysator.liu.se/c/bwk-on-pascal.html> describes much
>> of the practical realities of this sort of choice:
>>
>
> I think language designers since Pascal have learned from Pascal's mistake
> there.
>
> Supposedly even Borland's TurboPascal had better strings than vanilla
> Pascal.
>

That's true. You could do things in Turbo Pascal you couldn't do in
standard Pascals. It was close enough to C that a lot of system level code
was written in it. A bit verbose, but serviceable.

Warner

>

[-- Attachment #2: Type: text/html, Size: 2495 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-05  1:33         ` Noel Hunt
  2021-07-05  2:38           ` Clem Cole
@ 2021-07-06  5:10           ` Nevin Liber
  2021-07-06 13:30             ` Clem Cole
                               ` (2 more replies)
  1 sibling, 3 replies; 109+ messages in thread
From: Nevin Liber @ 2021-07-06  5:10 UTC (permalink / raw)
  To: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 542 bytes --]

>
>
> He didn't need to abuse any existing program by adding new
> flags or the like; unless I am mistaken, `sed Nq', for some
> number `N', does exactly what `head -N' would do on a single
> file, obviating the very need for head(1).
>

To summarize this discussion:

cat -v should be a separate executable and not a command line option.
That's the Unix way.

head isn't needed because we can already do it with the command line
options for sed.  That's the Unix way.
-- 
 Nevin ":-)" Liber  <mailto:nevin@eviloverlord.com>  +1-847-691-1404

[-- Attachment #2: Type: text/html, Size: 1225 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view)
  2021-07-06  4:35                     ` Dan Stromberg
  2021-07-06  4:44                       ` Warner Losh
@ 2021-07-06  5:58                       ` Rico Pajarola
  2021-07-06 13:05                       ` Clem Cole
  2 siblings, 0 replies; 109+ messages in thread
From: Rico Pajarola @ 2021-07-06  5:58 UTC (permalink / raw)
  To: Dan Stromberg; +Cc: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 1160 bytes --]

On Mon, Jul 5, 2021 at 9:37 PM Dan Stromberg <drsalists@gmail.com> wrote:

> Supposedly even Borland's TurboPascal had better strings than vanilla
> Pascal.
>
and then they used only 1 byte for the string length, limiting the max
length to 255, which made it completely pointless, because the moment you
had some real world data, you were back in
manage-my-own-pointers-to-arrays-of-chars-land.

(I loved Turbo Pascal, it's the first language I used for "serious"
programming. They got a lot of things right, but strings were emphatically
not among those things).

My other gripe was that Turbo Pascal programs spend half their time on
pointer normalization (every pointer access has a preamble that makes sure
that the offset is < 16). I'd really love to understand why on earth they
decided to do it that way (I'm sure there's a reason, I just fail to see
it). It's not like anyone could do pointer arithmetic behind the compiler's
back... it slows things down, and it's probably the reason why an
equivalent Turbo C program is only half the code size. And it makes it
impossible to do clever things with segment registers (18 year old me
didn't like that).

[-- Attachment #2: Type: text/html, Size: 1668 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view)
  2021-07-06  4:35                     ` Dan Stromberg
  2021-07-06  4:44                       ` Warner Losh
  2021-07-06  5:58                       ` Rico Pajarola
@ 2021-07-06 13:05                       ` Clem Cole
  2 siblings, 0 replies; 109+ messages in thread
From: Clem Cole @ 2021-07-06 13:05 UTC (permalink / raw)
  To: Dan Stromberg; +Cc: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 2811 bytes --]

Exactly my point - *theory vs practice*.  It can be a petri dish, but that
is *not necessarily* a bad thing.  Different extended Pascal/Mod-2* et al *all
'fixed' the string data type in some manner (not even saying the
different solutions are good or bad).  But it always comes back to what was
(*is*) theory *vs*. practice.  As others have pointed out.  In the case of
Strings, you either made them more like C strings or you made them
basically useless - because, in practice, the null-terminated string works
pretty darned well for production code and can be made to be secure, but
the programmer can not be lazy.

Hey, I personally look like Pascal for what it is and I still think it's
the best teaching tool, particularly with Clancy and Cooper's book for
beginning programmers.  I personally learned both languages around the same
time, but I had already been writing in a number of assemblers, BASIC,
Fortran, Algol-W, SAIL, and BLISS before I saw either.  I've more written
way more C code than any other language --- why because it works and as a
professional, I know how to properly use it.

As bwk says in that same document which I pointed to earlier, "comparing C
and Pascal is the same as trying to compare a jetfighter with a Piper
Cub."  As my former Marine pilot B-I-L reminds me, he did not start pilot
training in Whidbey Island on jets - he worked his way up and showed he was
competent before the Navy made it easier for him to kill himself (and those
around him).  BTW: the Navy does not tend to try to land small prop planes
on the USS Kennedy either.  As my B-I-L says for all of his day and night
landings on same, he always somewhat scared the cr*p out him but he was
always careful to remember what he had been taught (and he says he never
had to use the 3 wire).

The bottom line becomes learning to pick and then using the proper tool for
the job and respect what is for and the constraints associated with using
it.

Clem
ᐧ

On Tue, Jul 6, 2021 at 12:35 AM Dan Stromberg <drsalists@gmail.com> wrote:

>
> On Mon, Jul 5, 2021 at 2:29 PM Clem Cole <clemc@ccc.com> wrote:
>
>>
>> On Mon, Jul 5, 2021 at 4:16 PM Dan Stromberg <drsalists@gmail.com> wrote:
>>
>>> A null-terminated array of char is a petri dish.  A proper string type
>>> is more like a disinfectant.
>>>
>> Hrrmpt.... maybe (in theory), but I can say that never seen it really
>> work in practice --  bwk in Why Pascal is Not My Favorite Programming
>> Language <http://www.lysator.liu.se/c/bwk-on-pascal.html> describes much
>> of the practical realities of this sort of choice:
>>
>
> I think language designers since Pascal have learned from Pascal's mistake
> there.
>
> Supposedly even Borland's TurboPascal had better strings than vanilla
> Pascal.
>

[-- Attachment #2: Type: text/html, Size: 4967 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-06  5:10           ` Nevin Liber
@ 2021-07-06 13:30             ` Clem Cole
  2021-07-06 16:23               ` Theodore Ts'o
  2021-07-06 13:40             ` [TUHS] [tuhs] The Unix shell: a 50-year view John Cowan
  2021-07-06 14:12             ` Chet Ramey
  2 siblings, 1 reply; 109+ messages in thread
From: Clem Cole @ 2021-07-06 13:30 UTC (permalink / raw)
  To: Nevin Liber; +Cc: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 2152 bytes --]

On Tue, Jul 6, 2021 at 1:13 AM Nevin Liber <nevin@eviloverlord.com> wrote:

> head isn't needed because we can already do it with the command line
>> options for sed.
>>
> Sigh ... fine except for the fact that sed(1) did not exist when head(1)
was written. So you missed the point of my example.  FWIW: if you want to
pick nits, in this case, the solution with the former is redundant with the
latter.  IMO, That's ok as it follows in the basic tradition.  UNIX has a
number of ways to do similar things/solve the same problem, because it
offers tools, the system designers do not try to dictate a singular
solution.

As I fear by the reaction, many of you have missed the point of what I was
trying to say. I guess I did not do that clearly.  Let me try in a shorter
form.

The basic idea of the original Unix was that was small and simple and in
Dennis' words, 'ran on modest hardware.'    The designers of UNIX also did
not try to solve any one particular problem but offered a set of tools for
a >>programmer<< take upon her/himself to do so.

The issue is that the target >>user<< of UNIX had devolved from that of a
'programmer' but rather the elusive 'end user' and her/his
view/requirements tend to be "solve my problem now -- I don't care how -
just do it I don't want to think about it - make it go away."   So over
time, we hid a lot of the simplicity in features that were built on
features (often warts) that were built on other features (often other
warts).  Mashey had a great visual in his "small is beautiful" talk using a
'build slide' in PPT terms that demonstrated the problem.

I was commenting on the OPs post of the paper picking on UNIX, the UNIX
Shell, and where we are today *vs.* 50+ years ago.  My other point is the
authors need to get over themselves and recognize that* they are not making
a really new argument*.    Folks were not too happy with many of the BSD
'features' either, but now those same features (like head(1) or BSD sockets(3))
are considered SOP for nay new UNIX and you have to have them - even if
there are other if not 'better' ways of doing the same thing.
ᐧ

[-- Attachment #2: Type: text/html, Size: 5126 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-06  5:10           ` Nevin Liber
  2021-07-06 13:30             ` Clem Cole
@ 2021-07-06 13:40             ` John Cowan
  2021-07-06 14:12             ` Chet Ramey
  2 siblings, 0 replies; 109+ messages in thread
From: John Cowan @ 2021-07-06 13:40 UTC (permalink / raw)
  To: Nevin Liber; +Cc: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 1751 bytes --]

On Tue, Jul 6, 2021 at 1:12 AM Nevin Liber <nevin@eviloverlord.com> wrote
in summary:


> cat -v should be a separate executable and not a command line option.
> That's the Unix way.
>
> head isn't needed because we can already do it with the command line
> options for sed.  That's the Unix way.
>

Hmm, it seems like I must write my own Master Foo koan (this is a first
draft only, improvements welcome):

After Master Foo completed his discourse on the Unix nature <
http://www.catb.org/~esr/writings/unix-koans/unix-nature.html>, a tall,
thin, gray-haired, balding, bespectacled man entered the room.

Master Foo looked up from his lectern.

"Greetings," said Master Foo.  "Who are you?"

"I am the Master Planner," said the Master Planner.  "Ninth of my dharma
line," he added modestly.

"I am Master Foo," said Master Foo.  "What do you plan for?"

"To reach enlightenment by concentration on a single focus," said the
Master Planner.  "Therefore I remove all that is unnecessary from my life."

Master Foo said nothing, but began to type fiercely.  Those students who
could see what he was doing were astonished that he appeared to be typing
complete gibberish.

When he stopped typing, Master Foo pushed the device to the Master
Planner.  "Read me, if you would, the last maxim of the Zen of Python."

The Master Planner appeared as astonished as the students.  "But all men
know that the twenty-one maxims of the Zen of Python have been unchanged
since the Master Sorter wrote them down long ago!" he protested.

"Nevertheless," said Master Foo softly.

The Master Planner gave the words of command and then spoke very softly.
"It says -- it says, 'Inclusion is better than exclusion.'"

On hearing these words, all present were enlightened.

[-- Attachment #2: Type: text/html, Size: 5799 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-06  5:10           ` Nevin Liber
  2021-07-06 13:30             ` Clem Cole
  2021-07-06 13:40             ` [TUHS] [tuhs] The Unix shell: a 50-year view John Cowan
@ 2021-07-06 14:12             ` Chet Ramey
  2021-07-07  0:53               ` Nevin Liber
  2 siblings, 1 reply; 109+ messages in thread
From: Chet Ramey @ 2021-07-06 14:12 UTC (permalink / raw)
  To: Nevin Liber, The Unix Heritage Society mailing list

On 7/6/21 1:10 AM, Nevin Liber wrote:

> head isn't needed because we can already do it with the command line 
> options for sed.  That's the Unix way.

Except, in this case, `head' came first.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
		 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet@case.edu    http://tiswww.cwru.edu/~chet/

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-05  7:14               ` Tomasz Rola
  2021-07-05 16:26                 ` John Cowan
@ 2021-07-06 16:05                 ` Clem Cole
  2021-07-09 22:19                   ` Tomasz Rola
  1 sibling, 1 reply; 109+ messages in thread
From: Clem Cole @ 2021-07-06 16:05 UTC (permalink / raw)
  To: Tomasz Rola; +Cc: tuhs

[-- Attachment #1: Type: text/plain, Size: 7942 bytes --]

On Mon, Jul 5, 2021 at 3:16 AM Tomasz Rola <rtomek@ceti.pl> wrote:

> I have misread you then. I suppose the main problem that Unix solved
> fifty years ago was the need for OS on a not very big (but cheap!)
> machine. There were quite a few such systems and various such 16,
> 18-bit machines. So Unix was one of many, and what made it successful
> was rewriting in C (I think I have read about it somewhere). Various
> guys ported C compiler to their machines and to test it, they compiled
> Unix sources (I am sure I have read about this). So as C gained
> popularity, so did Unix gained popularity and C was the prog lang of
> Unix, thus with every new Unix install C gained even more popularity.


Ouch!!! This is so much important that you have missed in this statement,
as this is a great example of not seeing the forest because of all the
trees.  You are right that C and UNIX's success was intertwined but I think
you are missing the drivers for that.   They were successful because of
other things and because they were successful, we have them today.

So, Tomasz let me try again .. as an old f*rt that lived this era and
played a role in it.  Wrote some of the memos that 'justified UNIX instead
of a commercial solution, *etc*.'  UNIX was hardly a slam and it could have
easily not been successful, a lot of things had to occur for the result we
see today.

As I have mentioned before much of this topic has been discussed here
before.  In fact, I wrote and published a much longer explanation of all of
this is in a paper I published a few years ago for a French group looking
at how computers and UNIX affected society:  If can be found at:
http://technique-societe.cnam.fr
/colloque-international-unix-en-france-et-aux-etats-unis-innovation-diffusion-et-appropriation--945215.kjsp
  or send me an e-mail privately I can send you a PDF [note its formatted
for A4 paper, but will print on US letter successfully].

First ignore UNIX, Multics, or anything else.  Please concentrate on the
systems that IBM, BUNCH, DG, and DG had in the 1960s and 1970s.   An IBM
term was 'access method.'   Every program has to use a specifically defined
scheme to read the data:  ISAM/VSAM or in DEC's case RMS.  One of my
friends once said of RMS, it was not so bad that RMS had 256 to a thousand
optional features on each QIO call, but Culter had to check for each one of
them sequentially.  Yecch!!!!!

Unix as a programming system was a real breath of fresh air -- just
treating files as an unstructured bag of bits and the program figure out
how to deal with them was giant and very much against the thought of the
times.  And if you used ASCII, not a binary format, it means that humans
might even be able to debug it the system using the data!!  Don't write
huge programs to solve every possible problem you encounter with your
system.  Instead, write little ones that can be hooked together and reuse
the different smaller programs.  Build a library of complete simple
programs.

BTW:  UNIX was not the only system in those days exposing these types of
ideas.  I believe that it was Bert Sullivan (Ivan's brother) who built a
system at Lincoln labs that using what IIRC was Simula objects, that could
hook up small things together to manipulate something larger - not unlike
Doug's pipe ideas.   This stuff gave way to the later OOB programming
movement [and I would suggest much of the C++ we live with today too, but I
digress].

The idea of lots of little programs that cooperate with each other is not
what IBM and the like wanted/was providing.  They were selling closed
'solutions' complete SW systems ("walled gardens" controlled by them or
their programming minions) and yes needed the big iron they sold.  Note
the IBM 'solutions were not sold to engineers, their products were sold on
the golf course to the 'managers' of what we know call IT shops.

FWIW:  DEC was in a strange position.  It had come from serving the
engineers, but systems like VMS have slowly become more and more
interesting to the enterprise and that was a much more lucrative market.
 Remember what did Charlie Brown (AT&T's CEO) do after they were broken up
-- he wants to go after IBM!!!  AT&T's new marketing folks even change the
name of the codebase from the 'Programmers Workbench' to 'System III'
because they are afraid 'programmer' will not play well in enterprise sales.

The point here is that the 'enterprise type of system is really different
than engineers at BTL, Lincoln Labs, much less in the research campuses of
important universities.

The important thing is that the latter group (not enterprise) did not have
as much money but was a completely different population.   UNIX is 100% a
'Christiansen style Disruption'  ( read his book completely if you have
not, please).  It's a 'lessor technology,' running on smaller equipment,
targeting a new and different consumer who does not care that it is 'not as
good' as the established products.  If you compare UNIX to IBM's OS or VMS
for that matter, UNIX does not have the 'features' that are valued by the
'enterprise market.'

Ken/Dennis, *et al*, clearly do not care -- they were building something
they (the engineers) needed and wanted (thankfully).  And yes it had to run
on modest hardware because that is what they could afford and had access
to.  But because since the HW was modest, that forces a mindset of what are
we really doing on the SW?  How can we do it effectively.  The designers
are asking an important research question? *Maybe some of these other
schemes are not necessary*.

You are correct the C grew because of UNIX, but you kind of have it
backward. I'm a perfect example of what happened.  These new
microprocessors from Intel/Zilog/Moto/MOS Tech became available to us
(engineers mind you).  Hey, I was at CMU in the mid-1970s, I even had
access to the BLISS sources, but most people did not.  A BLISS cross
compiler binary cost $5K per CPU!!! And by the time what would become the
M68K shows up (I had access to the experimental chip that was not numbered)
I was at Tektronix and I then did not have BLISS sources.  *But I did have
the UNIX sources and those included a C compiler that dmr had written for
the PDP-11.   So I retargeted it.*   Same story with Steve Ward's crew in
the RTS at MIT, although they used both dmr and Steve's compilers.   I know
that same song played again at Purdue, also in UNSW, as well in the EU in a
couple of places if you look on the USENIX tapes.

It was *not that we tested our compiler by porting UNIX*.  We wanted a *C
compiler for whatever* program/project we had for these new devices.  Most
of the time we cross-compiled on our local UNIX box, of course. This was
true of the C8086 and Z80 tools I had in 1978/79.  But we all ran it on our
PDP-11s or later Vaxen.   It takes a few years before 'JAWS' start and we
see all the UNIX ports, but the C *vs.* Pascal wars were well underweight
by that time, and the UNIX vs. VMS vs. TOPS vs. Tenex vs. VM/CMS vs. TSS
vs. MTS, *etc*. wars had already become the stuff of legend.

The key point is that UNIX was inexpensive and worked on modest hardware.
Yes C came with it and that is why I think we use it not BLISS, BCPL, or
some flavor of PLx today.

It's simply a Christiansen style Disruption Technology -- something valued
by a new market that grew faster than the old market.   But the problem we
have with UNIX is as it displaced the old market, the old ideas/behaviors
of the other players that survived started to added back into the new
system their old scheme (with new names mind you -- but the same effect).

i.e. those not willing to learn from the errors and why success occurred in
history will repeat the errors of before.  Sadly, I have personally lived
this statement multiple times in my professional employment.
ᐧ

[-- Attachment #2: Type: text/html, Size: 12682 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-06 13:30             ` Clem Cole
@ 2021-07-06 16:23               ` Theodore Ts'o
  2021-07-07  1:57                 ` Dan Cross
  0 siblings, 1 reply; 109+ messages in thread
From: Theodore Ts'o @ 2021-07-06 16:23 UTC (permalink / raw)
  To: Clem Cole; +Cc: The Unix Heritage Society mailing list

On Tue, Jul 06, 2021 at 09:30:31AM -0400, Clem Cole wrote:
> The basic idea of the original Unix was that was small and simple and in
> Dennis' words, 'ran on modest hardware.'    The designers of UNIX also did
> not try to solve any one particular problem but offered a set of tools for
> a >>programmer<< take upon her/himself to do so.
> 
> The issue is that the target >>user<< of UNIX had devolved from that of a
> 'programmer' but rather the elusive 'end user' and her/his
> view/requirements tend to be "solve my problem now -- I don't care how -
> just do it I don't want to think about it - make it go away."   So over
> time, we hid a lot of the simplicity in features that were built on
> features (often warts) that were built on other features (often other
> warts).

I'd go even farther than that.  Hardware is no longer as modest, or as
simple.  And even if the target user is still the "programmer" it may
not be the case that worked well wtih the hardware and the problems of
50+ years ago is in fact that best answer today.  Including, for
example, the claim that everything should be represented in terms of a
byte stream and/or a file.

I'll refer people to the former FreeBSD core team member and currrent
FreeBSD developer, Benno Rice's presentation from linux.conf.au 2020,
"What UNIX Cost Us":

	https://www.youtube.com/watch?v=9-IWMbJXoLM

> I was commenting on the OPs post of the paper picking on UNIX, the UNIX
> Shell, and where we are today *vs.* 50+ years ago.  My other point is the
> authors need to get over themselves and recognize that* they are not making
> a really new argument*.    Folks were not too happy with many of the BSD
> 'features' either, but now those same features (like head(1) or BSD sockets(3))
> are considered SOP for nay new UNIX and you have to have them - even if
> there are other if not 'better' ways of doing the same thing.

And if the Unix patriaches were perhaps mistaken about how useful
"head" might be and whether or not it should have been considered
verboten, perhaps there is something to the claim that extreme
simplicity and forcing everything to implement in terms of the
smallest, simplest operations, might not make sense.  After all, taken
to extreme, if simplicity is the only good, then instead of Intel
CPU's, or PDP-11's, maybe we should be programming everything in terms
of a Turing Computer --- after all, "small is beautiful" and even a
PDP-11 is unnecessary complexity.  :-)

Or maybe not.

One of Benno's claims is even the Unix philosophy, when taken to
extremes, can be as much of an ideology as say, Fundamentalist
Christianity.  My Episcopalean roots may be showing, but the words of
Scripture (or the writings of the Unix Patriarchs) is not the only
source of truth; and Tradition by itself is also not enough; we also
need to apply our own Reason and Experience.

     	    	     	       	    - Ted

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-05 16:26                 ` John Cowan
@ 2021-07-06 23:17                   ` Tomasz Rola
  2021-07-06 23:47                     ` Steve Nickolas
                                       ` (3 more replies)
  0 siblings, 4 replies; 109+ messages in thread
From: Tomasz Rola @ 2021-07-06 23:17 UTC (permalink / raw)
  To: John Cowan; +Cc: tuhs

On Mon, Jul 05, 2021 at 12:26:04PM -0400, John Cowan wrote:
> On Mon, Jul 5, 2021 at 3:15 AM Tomasz Rola <rtomek@ceti.pl> wrote:
> 
[...]
> By the way, did anyone else start out on Unix-alikes before using actual
> Unix?  I read the BSTJ issue and became an instant convert, but only when
> $EMPLOYER got a MicroVAX was I able to work like that.  Next came the MKS
> Toolkit on DOS, and then Cygwin for many years.  I suppose it's only my
> last two $EMPLOYERs standardizing on the Mac that has left me running,
> like, actual Unix.

Not quite a Unix-lookalike, but AmigaDOS was a mix of ideas - named
volumes (a bit) like in CP/M, multitasking (a bit) like in Unix (but
on my model, I only had M68K and no memory protection), command line
tools allowing for batch programming more sophisticated than what
MSDOS had on table (well, I really never got to the point of abusing
this). Also, Aztec C came with a suite of programs resembling what I
could later find on Unix: cc, ld, make, ar and few others I forgot. At
one point my workflow on Amiga looked similar to my workflow on
Solaris and I could help myself with some Amiga programs after reading
matching man pages on big box.

Better Amigas also had memory protection and newer Workbench
(graphical interface and stuff in ROM) and allowed for AREXX (yes,
clone of mainframe REXX, never learned it) and with AREXX one could
manipulate applications which sported so called REXX-port (if memory
serves) and do things very much resembling what one could today do
with COM/DCOM objects on Windows. I.e. your AREXX script could perform
batch process with painting program, with word processor, perhaps with
desktop publishing and so on.

[...]
> True.  But then, many of us geeks make our choices not on technical
> criteria but on tribal loyalty.  Which is *technically* superior, vi or
> emacs?  (Please don't answer that.)

Mu!

[

 Mu (negative) / The Mu-kōan :

 https://en.wikipedia.org/wiki/Mu_(negative)#The_Mu-k%C5%8Dan

]


> > Of course I could not be using specialised note
> > taking program. Instead, I went with Emacs and org-mode. In the
> > process I had to learn a bit of Elisp and dot-emacs file. Some
> > defaults in Emacs are not comfy for my eyes - fonts, colors, it had to
> > be fine tuned to my liking.
> >
> 
> Note that Emacs is probably the oldest import into the Unix ecosystem from
> outside, and it bears the marks of its origin: monolithic (but
> programmable), one tool does it all.

Well, when "everything" was small enough I really liked it. Nowadays
there seems to be a trend of making Emacs into another OS, like with
abomination we call the browser.

https://www.emacswiki.org/emacs/EmacsApplicationFramework

As long as I am able to trim it during compilation, they may put
whatever they want inside, but when I tried to unpack one of the
latest browser source code, it took more than 2.5 gigabytes (I am not
sure, it could have been a nightmare). I hope they will not apply this
crazyness to Emacs. I hope Emacs version 23 will keep compiling for a
while. 

[...]
> Without doubt.  I am not loyal to a kernel or a set of utilities, I simply
> follow the Way of Unix: <http://vrici.lojban.org/~cowan/upc/> (sadly
> incomplete)

Oh. I think you just need to bath in Tao, sink deep and breathe it
in. After that, words will form themselves out of the void and fall
right into the void opened with editor, all without any effort on your
side. 

-- 
Regards,
Tomasz Rola

--
** A C programmer asked whether computer had Buddha's nature.      **
** As the answer, master did "rm -rif" on the programmer's home    **
** directory. And then the C programmer became enlightened...      **
**                                                                 **
** Tomasz Rola          mailto:tomasz_rola@bigfoot.com             **

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-06 23:17                   ` Tomasz Rola
@ 2021-07-06 23:47                     ` Steve Nickolas
  2021-07-06 23:49                       ` Warner Losh
  2021-07-06 23:48                     ` John Cowan
                                       ` (2 subsequent siblings)
  3 siblings, 1 reply; 109+ messages in thread
From: Steve Nickolas @ 2021-07-06 23:47 UTC (permalink / raw)
  To: Tomasz Rola; +Cc: tuhs

On Wed, 7 Jul 2021, Tomasz Rola wrote:

> Well, when "everything" was small enough I really liked it. Nowadays
> there seems to be a trend of making Emacs into another OS, like with
> abomination we call the browser.

I don't think that's anything new.  "Great OS, where's the editor?" is a 
really old joke about EMACS.

-uso.

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-06 23:17                   ` Tomasz Rola
  2021-07-06 23:47                     ` Steve Nickolas
@ 2021-07-06 23:48                     ` John Cowan
  2021-07-07  0:46                     ` Theodore Ts'o
  2021-07-07 13:54                     ` Tony Finch
  3 siblings, 0 replies; 109+ messages in thread
From: John Cowan @ 2021-07-06 23:48 UTC (permalink / raw)
  To: Tomasz Rola; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 372 bytes --]

On Tue, Jul 6, 2021 at 7:17 PM Tomasz Rola <rtomek@ceti.pl> wrote:


> Oh. I think you just need to bath in Tao, sink deep and breathe it
> in. After that, words will form themselves out of the void and fall
> right into the void opened with editor, all without any effort on your
> side.


Well, yes.  But to achieve that state of effortlessness requires so much
effort!

[-- Attachment #2: Type: text/html, Size: 1110 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-06 23:47                     ` Steve Nickolas
@ 2021-07-06 23:49                       ` Warner Losh
  0 siblings, 0 replies; 109+ messages in thread
From: Warner Losh @ 2021-07-06 23:49 UTC (permalink / raw)
  To: Steve Nickolas; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 484 bytes --]

On Tue, Jul 6, 2021 at 5:48 PM Steve Nickolas <usotsuki@buric.co> wrote:

> On Wed, 7 Jul 2021, Tomasz Rola wrote:
>
> > Well, when "everything" was small enough I really liked it. Nowadays
> > there seems to be a trend of making Emacs into another OS, like with
> > abomination we call the browser.
>
> I don't think that's anything new.  "Great OS, where's the editor?" is a
> really old joke about EMACS.
>

There were even april fools day posts about booting vmunix.el...

Warner

[-- Attachment #2: Type: text/html, Size: 903 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-06 23:17                   ` Tomasz Rola
  2021-07-06 23:47                     ` Steve Nickolas
  2021-07-06 23:48                     ` John Cowan
@ 2021-07-07  0:46                     ` Theodore Ts'o
  2021-07-07  0:58                       ` George Michaelson
  2021-07-07 18:32                       ` Tomasz Rola
  2021-07-07 13:54                     ` Tony Finch
  3 siblings, 2 replies; 109+ messages in thread
From: Theodore Ts'o @ 2021-07-07  0:46 UTC (permalink / raw)
  To: Tomasz Rola; +Cc: tuhs

On Wed, Jul 07, 2021 at 01:17:00AM +0200, Tomasz Rola wrote:
> 
> Well, when "everything" was small enough I really liked it. Nowadays
> there seems to be a trend of making Emacs into another OS, like with
> abomination we call the browser.
> 
> https://www.emacswiki.org/emacs/EmacsApplicationFramework
> 
> As long as I am able to trim it during compilation, they may put
> whatever they want inside, but when I tried to unpack one of the
> latest browser source code, it took more than 2.5 gigabytes (I am not
> sure, it could have been a nightmare). I hope they will not apply this
> crazyness to Emacs. I hope Emacs version 23 will keep compiling for a
> while.

Well, the old joke was that emacs stood for "eight megabytes and
constantly swapping".  These days, sure, starting a fresh Emacs
version 27 process has a SIZE of 364 megabytes with an RSS of 78
megabytes.

OTOH, starting a fresh copy of Konsole (KDE's current terminal
emulator) has a SIZE 1383 megabytes with an RSS of 114 megabytes, and
the single Konsole process running all of my terminal windows has a
SIZE of 2160 megabytes (or just a touch over 2GB) with an RSS of 189
megabytes.

As a percentage of the 32 GB physical memory in my Desktop machine,
I'm not too worried about the memory consumption of either the
terminal windows or emacs, especially since the browser takes a lot
more memory.  These days, I run my browser in a container to limit its
physical memory usage to 12GB; systemd makes setting this up via a
user unit file really easy.  :-)

	      	      	      	    - Ted

# ~/.config/systemd/chrome.service
[Unit]
Description=Chrome Browser

[Service]
ExecStart=/usr/bin/google-chrome
KillMode=process
MemoryAccounting=true
MemoryMax=12G

P.S.  On my laptop I constrain the browser to only use 8GB, which just
means that if I keep huge numbers of tabs open, some of them might get
automatically killed and will have to get reloaded when I swtich back
to that tab.  Sure, this wouldn't fly on a PDP-11, but as long as I'm
more productive, I don't really worry about it.

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-06 14:12             ` Chet Ramey
@ 2021-07-07  0:53               ` Nevin Liber
  2021-07-07 13:08                 ` Chet Ramey
  0 siblings, 1 reply; 109+ messages in thread
From: Nevin Liber @ 2021-07-07  0:53 UTC (permalink / raw)
  To: chet.ramey; +Cc: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 476 bytes --]

On Tue, Jul 6, 2021 at 9:12 AM Chet Ramey <chet.ramey@case.edu> wrote:

> On 7/6/21 1:10 AM, Nevin Liber wrote:
>
> > head isn't needed because we can already do it with the command line
> > options for sed.  That's the Unix way.
>
> Except, in this case, `head' came first.
>

While I may be mis-remembering, IIRC that wasn't true in System V Unix from
AT&T (which is where I was first exposed to it).
-- 
 Nevin ":-)" Liber  <mailto:nevin@eviloverlord.com>  +1-847-691-1404

[-- Attachment #2: Type: text/html, Size: 1036 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-07  0:46                     ` Theodore Ts'o
@ 2021-07-07  0:58                       ` George Michaelson
  2021-07-07  2:48                         ` Larry McVoy
  2021-07-07 18:32                       ` Tomasz Rola
  1 sibling, 1 reply; 109+ messages in thread
From: George Michaelson @ 2021-07-07  0:58 UTC (permalink / raw)
  To: Theodore Ts'o; +Cc: tuhs

The emacs manual *printed, one-sided, bound to the desk with rods of
steel* which I read in the 1980s in leeds was one of the best
explanations of Virtual Memory I saw. I really struggled with the idea
of address segments, maps, the idea of address space being bigger than
physical memory (I think I drove a PhD student doing tutoring close to
tears on this in '79) but the Emacs manual said really clearly
up-front: "look, you can't address 32 bits of memory in "me" I only do
24, but this is how I do them, if you're interested"

The Vax VMS manual along side it (another 2 feet of single-sided
print) was probably as good, but more aimed at real engineers who
could think in RPN and had pocket protectors.

This was in a context where it was probably the go-to basis to try and
play with LISP because nobody really told you about any other REPL to
run in. I think even then I realised I wasn't going to ever want to
code a towers-of-hanoi, nor even really explore 24 bits of (virtual)
address space.

I hate the cult. I decided to re-learn the finger muscle memory, now I
can do bare-minimum in emacs for ORG and I think I'll go back to vi
where I belong. vi suffered from insufficient love. I had to stop
hating vim when it became the only real choice. (hate.. culty word,
that) VScode was interesting, as was Atom, and I suspect more than a
few people who code for a way of life here think this editor-wars
stuff is tedious.

I actually "think" in ed. I can't escape line-based semantic intent. I
carry my own personal koan which basically says "any algorithm which
needs more than 2 sentences or 1 screen of code to implement is
probably beyond you". Its a bit of a flaw.

On Wed, Jul 7, 2021 at 10:47 AM Theodore Ts'o <tytso@mit.edu> wrote:
>
> On Wed, Jul 07, 2021 at 01:17:00AM +0200, Tomasz Rola wrote:
> >
> > Well, when "everything" was small enough I really liked it. Nowadays
> > there seems to be a trend of making Emacs into another OS, like with
> > abomination we call the browser.
> >
> > https://www.emacswiki.org/emacs/EmacsApplicationFramework
> >
> > As long as I am able to trim it during compilation, they may put
> > whatever they want inside, but when I tried to unpack one of the
> > latest browser source code, it took more than 2.5 gigabytes (I am not
> > sure, it could have been a nightmare). I hope they will not apply this
> > crazyness to Emacs. I hope Emacs version 23 will keep compiling for a
> > while.
>
> Well, the old joke was that emacs stood for "eight megabytes and
> constantly swapping".  These days, sure, starting a fresh Emacs
> version 27 process has a SIZE of 364 megabytes with an RSS of 78
> megabytes.
>
> OTOH, starting a fresh copy of Konsole (KDE's current terminal
> emulator) has a SIZE 1383 megabytes with an RSS of 114 megabytes, and
> the single Konsole process running all of my terminal windows has a
> SIZE of 2160 megabytes (or just a touch over 2GB) with an RSS of 189
> megabytes.
>
> As a percentage of the 32 GB physical memory in my Desktop machine,
> I'm not too worried about the memory consumption of either the
> terminal windows or emacs, especially since the browser takes a lot
> more memory.  These days, I run my browser in a container to limit its
> physical memory usage to 12GB; systemd makes setting this up via a
> user unit file really easy.  :-)
>
>                                     - Ted
>
> # ~/.config/systemd/chrome.service
> [Unit]
> Description=Chrome Browser
>
> [Service]
> ExecStart=/usr/bin/google-chrome
> KillMode=process
> MemoryAccounting=true
> MemoryMax=12G
>
> P.S.  On my laptop I constrain the browser to only use 8GB, which just
> means that if I keep huge numbers of tabs open, some of them might get
> automatically killed and will have to get reloaded when I swtich back
> to that tab.  Sure, this wouldn't fly on a PDP-11, but as long as I'm
> more productive, I don't really worry about it.

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-06 16:23               ` Theodore Ts'o
@ 2021-07-07  1:57                 ` Dan Cross
  2021-07-07  2:52                   ` Larry McVoy
  2021-07-07 18:28                   ` Jon Steinhart
  0 siblings, 2 replies; 109+ messages in thread
From: Dan Cross @ 2021-07-07  1:57 UTC (permalink / raw)
  To: Theodore Ts'o; +Cc: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 7141 bytes --]

On Tue, Jul 6, 2021 at 12:25 PM Theodore Ts'o <tytso@mit.edu> wrote:

> On Tue, Jul 06, 2021 at 09:30:31AM -0400, Clem Cole wrote:
> > The basic idea of the original Unix was that was small and simple and in
> > Dennis' words, 'ran on modest hardware.'    The designers of UNIX also
> did
> > not try to solve any one particular problem but offered a set of tools
> for
> > a >>programmer<< take upon her/himself to do so.
> >
> > The issue is that the target >>user<< of UNIX had devolved from that of a
> > 'programmer' but rather the elusive 'end user' and her/his
> > view/requirements tend to be "solve my problem now -- I don't care how -
> > just do it I don't want to think about it - make it go away."   So over
> > time, we hid a lot of the simplicity in features that were built on
> > features (often warts) that were built on other features (often other
> > warts).
>
> I'd go even farther than that.  Hardware is no longer as modest, or as
> simple.  And even if the target user is still the "programmer" it may
> not be the case that worked well wtih the hardware and the problems of
> 50+ years ago is in fact that best answer today.  Including, for
> example, the claim that everything should be represented in terms of a
> byte stream and/or a file.
>
> I'll refer people to the former FreeBSD core team member and currrent
> FreeBSD developer, Benno Rice's presentation from linux.conf.au 2020,
> "What UNIX Cost Us":
>
>         https://www.youtube.com/watch?v=9-IWMbJXoLM


Interestingly, much of this is kind of the point I was trying to make
earlier: many of the problems are different now. Trying to force them into
a metaphor that made sense for the problems that were primary 40 years ago
can lead to awkward and inefficient solutions. It doesn't always have to,
but sometimes it does.

"Everything is a file" and "files are streams of bytes" was great for
approaching so many of the sorts of problems that were interesting in the
1970s and 80s. Now days, there are whole new types of problems that don't
fit into that model easily; is that the fault of the problem, or a
suggestion that we need to expand our model and reexamine the basic
structure of our systems? Even, to some extent, the notion of a global
hierarchical file namespace has broken down. Now, I want to be able to
partition and organize datasets across multiple dimensions and across
multiple machines and group things on an ad hoc basis; it's all very
dynamic, and by comparison the model of a 7th Edition filesystem is static
and downright limited.

Fork is another great example: it went in because a) Ken knew about it, and
b) it was easy to implement (22 instructions of PDP-7 assembler or
something) and c) it got the job done. After the fact, it turned out to
have all kinds of neat properties that let it compose with pipelines,
redirection, background jobs, etc. That model for process management served
well for a long time. But then people wanted to have multithreaded
processes, because it turns out that independent threads of execution
inside of a shared address space are an excellent mechanism for
representing concurrency. But they discovered that fork composes poorly
with threads: the problem space changed and the model broke down.

Btw, Benno Rice is entertaining to watch, and I enjoy his presentations.
His talk on COBOL is also interesting, and I would argue somewhat relevant:
https://www.youtube.com/watch?v=BCqGjGzWI48

> I was commenting on the OPs post of the paper picking on UNIX, the UNIX
> > Shell, and where we are today *vs.* 50+ years ago.  My other point is the
> > authors need to get over themselves and recognize that* they are not
> making
> > a really new argument*.    Folks were not too happy with many of the BSD
> > 'features' either, but now those same features (like head(1) or BSD
> sockets(3))
> > are considered SOP for nay new UNIX and you have to have them - even if
> > there are other if not 'better' ways of doing the same thing.
>
> And if the Unix patriaches were perhaps mistaken about how useful
> "head" might be and whether or not it should have been considered
> verboten, perhaps there is something to the claim that extreme
> simplicity and forcing everything to implement in terms of the
> smallest, simplest operations, might not make sense.  After all, taken
> to extreme, if simplicity is the only good, then instead of Intel
> CPU's, or PDP-11's, maybe we should be programming everything in terms
> of a Turing Computer --- after all, "small is beautiful" and even a
> PDP-11 is unnecessary complexity.  :-)
>
> Or maybe not.
>
> One of Benno's claims is even the Unix philosophy, when taken to
> extremes, can be as much of an ideology as say, Fundamentalist
> Christianity.  My Episcopalean roots may be showing, but the words of
> Scripture (or the writings of the Unix Patriarchs) is not the only
> source of truth; and Tradition by itself is also not enough; we also
> need to apply our own Reason and Experience.
>

I don't know that the "Unix Patriarchs" ever suggested or intended that
"head" was "verboten" or that programs be forced into their smallest,
simplest expression. To continue your sectarian metaphor, that's an extreme
interpretation of the early church's teachings regarding its scripture.

I tend to think of these things as schools of art, rather than religious
orders. Much of what we think of as "Unix" carries with it an implicit
allegiance to a class of aesthetics: file namespaces and byte streams are
great examples. We claim our way is better, but if you sort of squint at it
right, you can kinda-sorta imagine a Unix system as an image based language
environment, where the "image" is the filesystem and running kernel
instance, and the REPL is the shell. Is it so different, then, than
Smalltalk or a Lisp? In some sense, no, it's not. Instead of CONS cells
making lists, I have streams of bytes, but in some regard that's just an
aesthetic thing: it's my fundamental building block, but it's just a
building block and all systems have building blocks.

But like the art world, is it any wonder that the big arguments are over
minutia? The question you asked me earlier, about whether System V is
"really Unix" isn't so dissimilar from the questions over whether Common
Lisp was "really Lisp". Similarly, discarding "head" for the generality of
"seq 10q" is an aesthetic statement, not a matter of engineering principle.

Rob Pike in a black turtleneck espousing the benefits of simplistic
minimalism as an end unto itself is an exponent of that school, but there
are other schools, as Benno reminds us.

The challenge for the new generation of artists is to absorb what's come
before and create something new. If they draw from many schools, that's
fine. What perhaps irks so many of us is that the school is changing, and
without full appreciation of the aesthetics of existing canon. "It is sad,
this state of affairs. Look how ugly is the systemd!" "But Dan...your
/etc/rc is so 1970s." "Yes! See the beauty! Perceive it!" "You are old." "I
weep for you."

The question I'm asking is if the kids are reaching far enough.

        - Dan C.

[-- Attachment #2: Type: text/html, Size: 8656 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-07  0:58                       ` George Michaelson
@ 2021-07-07  2:48                         ` Larry McVoy
  0 siblings, 0 replies; 109+ messages in thread
From: Larry McVoy @ 2021-07-07  2:48 UTC (permalink / raw)
  To: George Michaelson; +Cc: tuhs

On Wed, Jul 07, 2021 at 10:58:50AM +1000, George Michaelson wrote:
> I actually "think" in ed. 

I think in pic, I can look at pic code and see what it will draw.
To me, pic is one of the best things that came out of Bell Labs.
Unix is awesome, no doubt.  But for the essence of small but you
can see what it will do, pic is greater than ed.  No disrespect
to ed, I've spent a pile of time in ed.

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-07  1:57                 ` Dan Cross
@ 2021-07-07  2:52                   ` Larry McVoy
  2021-07-07  5:19                     ` Andrew Warkentin
  2021-07-07 18:28                   ` Jon Steinhart
  1 sibling, 1 reply; 109+ messages in thread
From: Larry McVoy @ 2021-07-07  2:52 UTC (permalink / raw)
  To: Dan Cross; +Cc: The Unix Heritage Society mailing list

On Tue, Jul 06, 2021 at 09:57:38PM -0400, Dan Cross wrote:
> Fork is another great example: it went in because a) Ken knew about it, and
> b) it was easy to implement (22 instructions of PDP-7 assembler or
> something) and c) it got the job done. After the fact, it turned out to
> have all kinds of neat properties that let it compose with pipelines,
> redirection, background jobs, etc. That model for process management served
> well for a long time. But then people wanted to have multithreaded
> processes, because it turns out that independent threads of execution
> inside of a shared address space are an excellent mechanism for
> representing concurrency. 

http://lkml.iu.edu/hypermail/linux/kernel/0106.2/0405.html

I wasn't completely right 20 years ago but I was close.  I'm tired,
if you want to know where I'm wrong, ask and I'll tell you how I
tried to get Linus to fix it.

In general, Rob was on point.  He usually is.

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-07  2:52                   ` Larry McVoy
@ 2021-07-07  5:19                     ` Andrew Warkentin
  0 siblings, 0 replies; 109+ messages in thread
From: Andrew Warkentin @ 2021-07-07  5:19 UTC (permalink / raw)
  To: The Unix Heritage Society mailing list; +Cc: coff

On 06/07/2021, Larry McVoy <lm@mcvoy.com> wrote:
>
> http://lkml.iu.edu/hypermail/linux/kernel/0106.2/0405.html
>
> I wasn't completely right 20 years ago but I was close.  I'm tired,
> if you want to know where I'm wrong, ask and I'll tell you how I
> tried to get Linus to fix it.
>
> In general, Rob was on point.  He usually is.
>

I've never been a fan of clone(). It always strikes me as something
that seems like an elegant simplification at first, but the practical
realization (on Linux that is) requires several rather ugly
library-level hacks to make it work right for typical use cases.

UX/RT will use the "processes are containers for threads" model rather
than rfork()/clone() since that's the model the seL4 kernel basically
uses (in a very generalized form with address spaces , capability
spaces, and threads being separate objects and each thread being
associated with a capability space and address space), and it would
also be slightly easier to create the helper threads that will be
required in certain parts of the IPC transport layer.

The base process creation primitive (efork(), for "empty/eviscerated
fork") will create a completely blank non-runnable child process with
no memory mappings or file descriptors, and return a context FD that
the parent can use to manipulate the state of the child with normal
APIs, including copying FDs and memory mappings. To actually start the
child the parent will perform an exec*() within the child context
(either a regular exec*() to make the child run a different program,
or a new eexec() function that takes an entry point rather than a
command line to run the process with whatever memory mappings were set
up), after which point the parent will no longer be able to manipulate
the child's state.

This will eliminate the overhead of fork() for spawning processes
running other programs, but will still allow for a library-level
fork() implementation that has comparable overhead to traditional
implementations. Also, it will do what Plan 9 never did and make the
process/memory APIs file-oriented (I still don't get why Plan 9 went
with a rather limited anonymous memory API rather than using
memory-mapped files for everything).

Also, we're straying a bit from historical Unix here and should have
probably moved to COFF several messages ago.

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-07  0:53               ` Nevin Liber
@ 2021-07-07 13:08                 ` Chet Ramey
  2021-07-07 15:15                   ` Richard Salz
  0 siblings, 1 reply; 109+ messages in thread
From: Chet Ramey @ 2021-07-07 13:08 UTC (permalink / raw)
  To: Nevin Liber; +Cc: The Unix Heritage Society mailing list

On 7/6/21 8:53 PM, Nevin Liber wrote:
> On Tue, Jul 6, 2021 at 9:12 AM Chet Ramey <chet.ramey@case.edu 
> <mailto:chet.ramey@case.edu>> wrote:
> 
>     On 7/6/21 1:10 AM, Nevin Liber wrote:
> 
>      > head isn't needed because we can already do it with the command line
>      > options for sed.  That's the Unix way.
> 
>     Except, in this case, `head' came first.
> 
> 
> While I may be mis-remembering, IIRC that wasn't true in System V Unix from 
> AT&T (which is where I was first exposed to it).

This came up on the list last week. Joy wrote `head' while using 6th
Edition systems at Berkeley; `sed' first appeared in the 7th Edition.


-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
		 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet@case.edu    http://tiswww.cwru.edu/~chet/

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-06 23:17                   ` Tomasz Rola
                                       ` (2 preceding siblings ...)
  2021-07-07  0:46                     ` Theodore Ts'o
@ 2021-07-07 13:54                     ` Tony Finch
  3 siblings, 0 replies; 109+ messages in thread
From: Tony Finch @ 2021-07-07 13:54 UTC (permalink / raw)
  To: Tomasz Rola; +Cc: tuhs

Tomasz Rola <rtomek@ceti.pl> wrote:
>
> Not quite a Unix-lookalike, but AmigaDOS was a mix of ideas - named
> volumes (a bit) like in CP/M, multitasking (a bit) like in Unix (but
> on my model, I only had M68K and no memory protection), command line
> tools allowing for batch programming more sophisticated than what
> MSDOS had on table (well, I really never got to the point of abusing
> this).

The more unixy parts of AmigaDOS came from TRIPOS
https://www.pagetable.com/?p=193
https://en.wikipedia.org/wiki/TRIPOS

TRIPOS was originally developed by a team led by Martin Richards, who had
previously developed BCPL.

> Also, Aztec C came with a suite of programs resembling what I
> could later find on Unix: cc, ld, make, ar and few others I forgot.

Also true for the Acorn / Norcroft ARM C Compiler on the Archimedes, which
was my first contact with unix-like tools :-)

Tony.
-- 
f.anthony.n.finch  <dot@dotat.at>  https://dotat.at/
Northwest Fitzroy, Sole: Northwesterly 4 or 5, occasionally 6 at
first, backing westerly or southwesterly 3 or 4 later. Moderate,
occasionally rough at first. Showers. Good, occasionally moderate.


^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-07 13:08                 ` Chet Ramey
@ 2021-07-07 15:15                   ` Richard Salz
  0 siblings, 0 replies; 109+ messages in thread
From: Richard Salz @ 2021-07-07 15:15 UTC (permalink / raw)
  To: chet.ramey; +Cc: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 444 bytes --]

> > While I may be mis-remembering, IIRC that wasn't true in System V Unix
> from
> > AT&T (which is where I was first exposed to it).
>
>
According to
http://bitsavers.trailing-edge.com/pdf/att/3b1/999-801-312IS_ATT_UNIX_PC_System_V_Users_Manual_Volume_1.pdf
and
https://archive.org/details/bitsavers_attunixSystemVRelease4ProgrammersReferenceManual19_55505648/page/n13/mode/2up
there is no 'head' command.  So Nevin's experience is accurate.

[-- Attachment #2: Type: text/html, Size: 960 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-07  1:57                 ` Dan Cross
  2021-07-07  2:52                   ` Larry McVoy
@ 2021-07-07 18:28                   ` Jon Steinhart
  2021-07-10 11:51                     ` [TUHS] " Ralph Corderoy
  1 sibling, 1 reply; 109+ messages in thread
From: Jon Steinhart @ 2021-07-07 18:28 UTC (permalink / raw)
  To: The Unix Heritage Society mailing list

Dan Cross writes:
>
> Interestingly, much of this is kind of the point I was trying to make
> earlier: many of the problems are different now. Trying to force them into
> a metaphor that made sense for the problems that were primary 40 years ago
> can lead to awkward and inefficient solutions. It doesn't always have to,
> but sometimes it does.
>
> "Everything is a file" and "files are streams of bytes" was great for
> approaching so many of the sorts of problems that were interesting in the 1970s
> and 80s. Now days, there are whole new types of problems that don't fit into
> that model easily; is that the fault of the problem, or a suggestion that we
> need to expand our model and reexamine the basic structure of our systems?
> Even, to some extent, the notion of a global hierarchical file namespace has
> broken down. Now, I want to be able to partition and organize datasets across
> multiple dimensions and across multiple machines and group things on an ad hoc
> basis; it's all very dynamic, and by comparison the model of a 7th Edition
> filesystem is static and downright limited.

Been trying hard to resist but here goes...

On the snarky side, yeah, problems are different now, and the shell paper
is a great example.  Such a bad paper winning a best paper award from
the ACM illustrates how the profession just ain't what it used to be.
I would claim that a large chunk of "software engineering" today involves
trying to build environments that minimize the amount of harm that can
be done by under-skilled practitioners.  I have difficulty envisioning
success as there's a heavy focus on memory management which to me is a
great competency test.  Just because one is coddled and can't corrupt
memory doesn't mean that the the remainder of their code is correct.
These things go hand-in-hand.

I'm going to stick my neck out and claim that to a large degree we're the
victims of our own success.  When I was writing my book, Clem pointed
out to me that a big assumption that Stallman made was that people
who would contribute to open source software would be of his caliber.
I don't think that he ever thought that we'd get to the point where
every programming class project would be published online and reused by
others without much inspection.  To me it's analogous to what happened
with MIDI when one could make "music" without having suffered for their
art like their predecessors.  Just decreased the signal-to-noise ratio
for me and made it much more cumbersome to find music that I liked.

Many of the problems that people seem to be trying to solve today are
of our own making.  I don't see the benefit of Linux distributions
being incompatible in annoying ways.  If each distro didn't put things
in different places for no discernible reason, we wouldn't need to
be putting as much non-productive energy into tools for deployment.
Likewise if there was any rhyme or reason for what's in what library
and more attention to API design and stability.  I kinda have to go
back to my 1989 Usenix panel session; given the choice between having a
good base and covering up a bad base with a pile of other code I choose
the first.  Sure, if we make stuff with tons of preventable dependencies
then we have to work with that, but I'd rather not have the problem in
the first place.

Another one - I recall when a lot of work went into making it possible
to link C and FORTRAN programs.  Instead of continuing along that path,
now we have multiple systems (PHP, Python, Perl, Java, ...) that have
each written their own libraries for everything.  To me, I would
have preferred to have a single set of libraries and the ability
to link.  Aside from the enormous waste of person-hours, there is a
huge non-intersecting set of bugs.  In theory we'd have fewer bugs
if more people were using the same libraries.  And better programmer
portability too.

I'm often surprised when people claim that the UNIX philosophy no
longer works.  I've been told "pipes only work for ASCII data" and have
gotten blank looks when I respond with "ImageMagick seems to work fine".
And "you can't do modern web crud" but then along came things like "jq".

To me, the UNIX philosophy beats the big monolithic program philosophy
every time.  Sure, these two don't interact when big monolithic programs
are crafted such that they don't play well with others but that's not
the fault of the UNIX philosophy, it's the fault of the monolithic
program design.  Watch Matt Blaze's 2016 congressional testimony if
you haven't already done so.  He makes a great case that we know that
we can't prove program correctness despite trying to do so since the
beginning of time, so the best security comes from simplicity which
minimizes the number of attack surfaces.

So rather than saying that the UNIX philosophy no longer works, how
about giving some serious thought to how to do what you want to do in
the context of the philosophy?  In particular, how to you extend the
existing tool set to accomplish your purpose instead of just throwing up
your hands and creating more big monolithic non-interoperable packages?
Certainly the first is harder, but lasts longer and tastes great too.
A question that I asked in my book was why ~0% of the 1200 pages of
"Inside Macintosh" from 1985 are used today compared to ~100% of the
323 pages of UNIX V6 from 1975.  My answer is "abstractions".  I don't
see any of the "modern packages" having that level of staying power.

There is a lot more to the UNIX philosophy than "Everything is a file"
and "files are streams of bytes"; it's a mindset.  I have no problem with
model expansion and structure reexamination.  But piling poorly thought
out crap on top of other poorly thought out crap isn't a solution to me.
What new problems are you trying to solve and what are the appropriate
abstractions (and how much of this was done in Plan 9 and ignored)?

Haven't thought about this in forever as it's several lifetimes ago.
Once upon a time I was responsible for implementing GKS (the useless
ISO Graphical Kernel System) on a workstation project and was
participating in the ANSI standardization effort.  At the time, there
was another standard (PHIGS - Programmer's Hierarchical Interface
to GraphicS) in the works; I remember asking why.  The answer was
that there was no way to add segment hierarchy and editing to GKS in a
compatible way.  In reality, the folks just wanted to do their own thing;
they were furious when I published a paper on how to easily add that
functionality to GKS.  It's easy to say that something can't be done;
harder to put in some serious thought to figure out how it can be done.

Bottom line is, just read the news.  Is today's software methodology
working?  Does the fact that Facebook is funding a PR campaign to
convince people that privacy breaches are normal tell you anything?
There's a lot of similarity between modern software and the Surfside
condo; shiny stuff on top of a crumbling foundation is bound to fail.
In lectures these days, I'm asking the question "We haven't managed
to off more than a thousand or so people at a time with a software bug
yet so what's going to be software's Union Carbide Bhopal moment?"

Jon Steinhart

P.S.  On another topic, GUIs were the killer app for threads.  Sure, I would
      much prefer faster process context switching.  Still waiting for that.

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-07  0:46                     ` Theodore Ts'o
  2021-07-07  0:58                       ` George Michaelson
@ 2021-07-07 18:32                       ` Tomasz Rola
  2021-07-07 20:50                         ` Michael Kjörling
  1 sibling, 1 reply; 109+ messages in thread
From: Tomasz Rola @ 2021-07-07 18:32 UTC (permalink / raw)
  To: Theodore Ts'o; +Cc: tuhs

On Tue, Jul 06, 2021 at 08:46:34PM -0400, Theodore Ts'o wrote:
> On Wed, Jul 07, 2021 at 01:17:00AM +0200, Tomasz Rola wrote:
> > 
[...]
> > https://www.emacswiki.org/emacs/EmacsApplicationFramework
[...]
> > 
> 
> Well, the old joke was that emacs stood for "eight megabytes and
> constantly swapping". 

Yeah, during my 486 days it was indeed doing stuff with disk. But I
was too happy to have this "workstation lookalike" running "Unix
lookalike" installed from dozens of floppies so I did not complained
about some minor problems.

> These days, sure, starting a fresh Emacs
> version 27 process has a SIZE of 364 megabytes with an RSS of 78
> megabytes.
> 
> OTOH, starting a fresh copy of Konsole (KDE's current terminal
> emulator) has a SIZE 1383 megabytes with an RSS of 114 megabytes, and
> the single Konsole process running all of my terminal windows has a
> SIZE of 2160 megabytes (or just a touch over 2GB) with an RSS of 189
> megabytes.

An excerpt from my ps:

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND

xxxek      576  0.0  0.2 283392 34172 pts/122  SNl  Jun29   0:06 emacs

xxxek    23684  0.3  1.7 379624 211872 pts/21  SN   Jul03  22:06 emacs /ho

xxxon    12331 12.5 20.4 5898360 2519640 ?     TNsl Mar29 18278:11 firefox-esr

xxxon     3553  0.0  1.2 218028 150656 pts/67  TN   Jun30   0:24 dillo

xxxek    12588  0.0  0.0  38108   752 pts/183  T     2020   0:00 lynx -nocolor -force-html /ho

xxxek     6785  0.0  0.0 310580 10052 pts/22   Sl    2020  41:51 roxterm

xxxek     5321  0.0  0.0 102712  1456 tty2     S     2020   1:56 rxvt -bg Black

xxxek     5322  0.0  0.0  66784  1988 tty2     S     2020 165:48 xterm -bg

xxxek     5596  0.0  0.0  21608   536 pts/121  S+    2020   1:35 /usr/bin/screen


Emacs ver.23, firefox ver. "very old". I do not trust my browsers too
much, so when I go to sleep they go to STOP freeze. The second emacs
is editing my pathetic########glorious######## wiki with org-mode,
file length about 55 megabytes, with unicodes, so the rss is multiple
of file size. I have long ago replaced konsole with roxterm. As you
can see, it eats nowhere near a gigabyte. And it does not read a whole
disk when starting. It usually goes with three tabs opened, so maybe
in your case memsize would have been bigger. I make a lot of use with
virtual desktops offered by fvwm, and when I need terminal on one of
those desktops (I usually do), I open some rxvt variation; position,
resize and keep it there for further use.

As I have only 12gigs of ram and more than half goes to ramdisk
(always doing something there, including compiling huge piles of code
from the net), so I have to keep an eye on firefox. When it goes above
2.9g of rss I usually start to fret and kill unneeded tabs. I set it
up to use only two or three fonts and completely disregard
author-provided fonts. So far, so good. Ah, and I use noscript,
too. Yet despite of all those precautions, I have to kill -STOP it,
because the mere thought that it will keep eating 50-90% of cpu while
doing nothing, well, this drives me up the wall. Likewise, back in the
past, when I saw how *idle* program (firefox) constantly wrote *gigs*
of bytes to my disk, I simply went mad. Long story short, now it only
can write to temporary place in ramdisk. It still does not stop it
from opening multiple files on disk in /tmp, deleting them and keeping
opened/deleted forever, using up my precious ram. It sometimes closes
those files, but not very often.

I usually open new pages in dillo, w3m or lynx (or curl) and use
firefox when nothing else would do. Not perfect solution - dillo has
some memory problem, so I usually have many of them opened and slash
those who grow too much. Like, 300megs for dillo is enough. Read up
all the tabs in it and kill. And it seems I have (pidof lynx|wc -w)
170 lynx processes, all kill-stopped and awaiting my
attention. Craps. Chances are I did not stopped being mad. At least
Unix command line tools allow me to measure it.

> As a percentage of the 32 GB physical memory in my Desktop machine,
> I'm not too worried about the memory consumption of either the
> terminal windows or emacs, especially since the browser takes a lot
> more memory.  These days, I run my browser in a container to limit its
> physical memory usage to 12GB; systemd makes setting this up via a
> user unit file really easy.  :-)
> 
> 	      	      	      	    - Ted
> 
> # ~/.config/systemd/chrome.service
> [Unit]
> Description=Chrome Browser
> 
> [Service]
> ExecStart=/usr/bin/google-chrome
> KillMode=process
> MemoryAccounting=true
> MemoryMax=12G

Now I am happy that I do not use systemd. Now I want to find more time
and migrate to OpenBSD. I really do (I know there are some Linux
distros which evaded systemd - for a while, but I consider a whole
Linux land to be lost already, just a matter of time, and I foresee
that OpenBSD will fall as the very last, if ever). I will sleep less
and program more. I will rewrite all my old python2x scripts in r7rs
scheme. Not really so many. Fruits of hard work shall be sweet and
plentiful.

12g for a browser. I am not criticising you, but I could say a bit
about people who made programs and websites contributing to such
sizes. 

-- 
Regards,
Tomasz Rola

--
** A C programmer asked whether computer had Buddha's nature.      **
** As the answer, master did "rm -rif" on the programmer's home    **
** directory. And then the C programmer became enlightened...      **
**                                                                 **
** Tomasz Rola          mailto:tomasz_rola@bigfoot.com             **

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-07 18:32                       ` Tomasz Rola
@ 2021-07-07 20:50                         ` Michael Kjörling
  2021-07-08  6:46                           ` [TUHS] Overgrown ffox (was: The Unix shell: a 50-year view) Tomasz Rola
  2021-07-08 21:47                           ` [TUHS] [tuhs] The Unix shell: a 50-year view Theodore Ts'o
  0 siblings, 2 replies; 109+ messages in thread
From: Michael Kjörling @ 2021-07-07 20:50 UTC (permalink / raw)
  To: tuhs

On 7 Jul 2021 20:32 +0200, from rtomek@ceti.pl (Tomasz Rola):
> An excerpt from my ps:
> 
> USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> 
> xxxon    12331 12.5 20.4 5898360 2519640 ?     TNsl Mar29 18278:11 firefox-esr

I'm going to stick my neck out here by saying that the VSZ and RSS
values reported by ps, at least for Firefox, are largely meaningless.

I started my usual Firefox instance, which has a handful of plugins,
about a metric gazillion bookmarks, and has been my main web browser
profile for years (so it probably has collected some crud over time).
`ps auxw` reported that process as having a total RSS of a whopping
374 GB.

It is downright _impossible_ that Firefox could actually be using that
amount of volatile memory, because my system has "only" 32 GB RAM + 64
GB swap. The root file system, which on my system also holds the likes
of /usr/{bin,lib}, totals 29701 MB used according to `df -m`, plus ~
totals 22550 MB, so even if Firefox read _all_ of that into memory it
still wouldn't be anywhere _close_ to the reported RSS value, falling
short by about a factor 7x (and Firefox has no reason to even look at
much of what's there, like the 2.5 GB in /usr/share/games/flightgear).

While that Firefox instance was running, I ran `free -m`, which
reported 250 MB swap used (65285 MB free) and 26008 MB "memory
available". Very soon after closing Firefox (and soon enough to try to
ensure that nothing else major changed), I ran `free -m` again, which
then reported the same 250 MB swap used and a slightly larger 26298 MB
memory available; a difference of 290 MB between when Firefox was
running and when it was not.

That's a _factor almost 2300x_ difference between the reported RSS,
and the amount of memory that was actually freed up by closing the
browser.

The same experiment done with a brand-new browser profile gave a total
RSS of 751 GB for Firefox's processes combined, while free reports a
difference of 321 MB between running and not-running. That's within
striking distance of a factor 2400x difference.

Does this mean that software bloat isn't an issue? I would argue that
it does not. Does it mean that Firefox can't have, say, memory leaks,
causing it to over time have more memory allocated than it actually
uses? It does not; and in fact I believe that's something that has
been worked on over the years. But I think it's worth trying to be
honest about all this, and at least _try_ to not compare apples to
stars just because both resemble oblate spheroids and exert
gravitational influence. On modern systems, with everything from
shared libraries to memory-mapped I/O to routine use of memory
overcommitting, the resident set size is clearly a poor indicator of
the actual amount of memory actively used by a complex process.

-- 
Michael Kjörling • https://michael.kjorling.se • michael@kjorling.se
 “Remember when, on the Internet, nobody cared that you were a dog?”


^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Overgrown ffox (was: The Unix shell: a 50-year view)
  2021-07-07 20:50                         ` Michael Kjörling
@ 2021-07-08  6:46                           ` Tomasz Rola
  2021-07-08 13:59                             ` Derek Fawcus
  2021-07-08 22:23                             ` Kevin Bowling
  2021-07-08 21:47                           ` [TUHS] [tuhs] The Unix shell: a 50-year view Theodore Ts'o
  1 sibling, 2 replies; 109+ messages in thread
From: Tomasz Rola @ 2021-07-08  6:46 UTC (permalink / raw)
  To: Michael Kjörling; +Cc: tuhs

On Wed, Jul 07, 2021 at 08:50:51PM +0000, Michael Kjörling wrote:
> On 7 Jul 2021 20:32 +0200, from rtomek@ceti.pl (Tomasz Rola):
> > An excerpt from my ps:
> > 
> > USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> > 
> > xxxon    12331 12.5 20.4 5898360 2519640 ?     TNsl Mar29 18278:11 firefox-esr
> 
> I'm going to stick my neck out here by saying that the VSZ and RSS
> values reported by ps, at least for Firefox, are largely meaningless.
> 
> I started my usual Firefox instance, which has a handful of plugins,
> about a metric gazillion bookmarks, and has been my main web browser
> profile for years (so it probably has collected some crud over time).
> `ps auxw` reported that process as having a total RSS of a whopping
> 374 GB.
> 
> It is downright _impossible_ that Firefox could actually be using that

This is quite strange for me. Without looking at your system I can only
suspect it has something to do with multithreading.

If I do two different commands as root, with firefox pid here
.eq. 12331, as above:

=>  (500 15):    lsof -p 12331 | wc -l
402

=>  (500 17):   lsof | awk '$2==12331' | wc -l
22055

The first column gives a name, and in second case it not always is
'firefox'. I am yet to study manpage for lsof and play with it, but it
surely shows interesting things.

On my system, when firefox gets killed, 'free' shows a difference - if
I recall, free mem increases by the size of rss plus all the stuff
which was opened and released from buffers. I did not pay much
attention, I assumed numbers would match and this is what they
probably did :-).

OS on my box used to report to me as Debian, and still does, but some
years ago I have decided to skip the usual system upgrade, and after
some more time I started to upgrade various elements by hand. So it is
more like a tattered patchwork right now. But it does what I expect,
hopefully.

[...]
> That's a _factor almost 2300x_ difference between the reported RSS,
> and the amount of memory that was actually freed up by closing the
> browser.

Yeah, strange.

[...]
> On modern systems, with everything from shared libraries to
> memory-mapped I/O to routine use of memory overcommitting, the
> resident set size is clearly a poor indicator of the actual amount
> of memory actively used by a complex process.

Hard to tell - first I would like to learn where the hundred-giga rss
came from...

-- 
Regards,
Tomasz Rola

--
** A C programmer asked whether computer had Buddha's nature.      **
** As the answer, master did "rm -rif" on the programmer's home    **
** directory. And then the C programmer became enlightened...      **
**                                                                 **
** Tomasz Rola          mailto:tomasz_rola@bigfoot.com             **

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Overgrown ffox (was: The Unix shell: a 50-year view)
  2021-07-08  6:46                           ` [TUHS] Overgrown ffox (was: The Unix shell: a 50-year view) Tomasz Rola
@ 2021-07-08 13:59                             ` Derek Fawcus
  2021-07-08 19:25                               ` Steffen Nurpmeso
  2021-07-08 22:23                             ` Kevin Bowling
  1 sibling, 1 reply; 109+ messages in thread
From: Derek Fawcus @ 2021-07-08 13:59 UTC (permalink / raw)
  To: tuhs

On Thu, Jul 08, 2021 at 08:46:53AM +0200, Tomasz Rola wrote:
> On Wed, Jul 07, 2021 at 08:50:51PM +0000, Michael Kjörling wrote:
> > On 7 Jul 2021 20:32 +0200, from rtomek@ceti.pl (Tomasz Rola):
> > > An excerpt from my ps:
> > > 
> > > USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> > > 
> > > xxxon    12331 12.5 20.4 5898360 2519640 ?     TNsl Mar29 18278:11 firefox-esr
> > 
> > I'm going to stick my neck out here by saying that the VSZ and RSS
> > values reported by ps, at least for Firefox, are largely meaningless.

Or does it depend upon the OS?  From a mac instance:

$ ps -u 502 -o vsz,rss,ucomm,command|fgrep Firefox-78esr.app|awk '$3!="fgrep" {print int($1/1024),int($2/1024),$3}'
10113 189 firefox
5564 73 plugin-container
6051 46 plugin-container
6129 56 plugin-container
5487 55 plugin-container
5619 53 plugin-container
7201 49 plugin-container
6852 60 plugin-container
5876 62 plugin-container
6247 56 plugin-container
6050 13 plugin-container
6055 9 plugin-container

The display from Activity Monitor, which one would assume better presents
the mach VM gives for the 'firefox' process:

Real Memory ~ 171M
Private Memory ~ 80M
Shared Memory ~ 85M
Memory ~ 1.05G

and when inspected a Virtual Memory size of ~ 9.8 G but Real memory size of ~ 170M

Whereas a linux instance (viewing different pages) of 89.0.2 gives:

$ ps -u 100 -o vsx,rss,ucomm,command|fgrep firefox|awk '$3!="grep" {print int($1/1024),int($2/1024),$3}'
3147 356 firefox
2699 200 Web
2522 99 WebExtensions
3271 219 Web
2642 181 Web
2511 105 Web
2489 73 Web

and yeah - it does 'leak' as the main firefox process sitting 'idle' has now become:

3190 346 firefox

So there are defintitly issues in mapping the current use of various types
of memory to a simple display format.

DF

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Overgrown ffox (was: The Unix shell: a 50-year view)
  2021-07-08 13:59                             ` Derek Fawcus
@ 2021-07-08 19:25                               ` Steffen Nurpmeso
  2021-07-08 19:37                                 ` Steffen Nurpmeso
  2021-07-08 20:40                                 ` Steffen Nurpmeso
  0 siblings, 2 replies; 109+ messages in thread
From: Steffen Nurpmeso @ 2021-07-08 19:25 UTC (permalink / raw)
  To: Derek Fawcus; +Cc: tuhs

Derek Fawcus wrote in
 <YOcE3fre5cUTt6Br@clarinet.employees.org>:
 |On Thu, Jul 08, 2021 at 08:46:53AM +0200, Tomasz Rola wrote:
 |> On Wed, Jul 07, 2021 at 08:50:51PM +0000, Michael Kjörling wrote:
 |>> On 7 Jul 2021 20:32 +0200, from rtomek@ceti.pl (Tomasz Rola):
 |>>> An excerpt from my ps:
 |>>> 
 |>>> xxxon    12331 12.5 20.4 5898360 2519640 ?     TNsl Mar29 18278:11 \
 |>>> firefox-esr
 |>> 
 |>> I'm going to stick my neck out here by saying that the VSZ and RSS
 |>> values reported by ps, at least for Firefox, are largely meaningless.
 |
 |Or does it depend upon the OS?  From a mac instance:
 ...
 |So there are defintitly issues in mapping the current use of various types
 |of memory to a simple display format.

Another thing is that, when i used it, Mac command line tools did
not take care about exit status (as in "succeed though they should
not") or know about job signals.
But it is not only Mac, take iwd again, iwctl('s display handling)
definetely is buggy in respect to handling of SIGCONT after
a TSTP.  (Ie the screen starts scrolling line-wise but no display
is ever refreshed.)

Regarding the thing, here this seems pretty much fine for RSS.

  root      1454  0.0  0.0   7912  2480 ?        S    21:03   0:00 /bin/bash /x/pub/box-web.sh browse firefox
  root      1456  0.0  0.0   3012  2032 ?        S    21:03   0:00 /usr/bin/doas -u steffen /box.firefox
  steffen   1457  0.0  0.0   2384   700 ?        S    21:03   0:00 /bin/sh - /box.firefox
  steffen   1458 31.2  3.7 3029004 299700 ?      Sl   21:03   0:04 firefox --no-remote
  steffen   1545  4.0  1.3 2421200 108196 ?      Sl   21:03   0:00 /usr/lib/firefox/firefox-bin -contentproc -childID 1 -isForBrowser -prefsLen 1 -prefMapSize 244806 -parentBuildID 20210614221319 -appdir /usr/lib/f
  steffen   1583  2.2  1.1 2412124 89636 ?       Sl   21:03   0:00 /usr/lib/firefox/firefox-bin -contentproc -childID 2 -isForBrowser -prefsLen 4909 -prefMapSize 244806 -parentBuildID 20210614221319 -appdir /usr/li
  steffen   1636  0.6  0.7 2382844 59560 ?       Sl   21:03   0:00 /usr/lib/firefox/firefox-bin -contentproc -childID 3 -isForBrowser -prefsLen 5602 -prefMapSize 244806 -parentBuildID 20210614221319 -appdir /usr/li

Yeah i use --no-remote because otherwise firefox will find its
other-box instances via X cookies (i do not have Wayland here).
I deem it a critical error that firefox then starts up even though
the ~/.mozilla/XXX profile is a totally different one, but on the
IRC we just ended like this.

Btw i use

  setup_cgroup() {
  #return
    [ -e /sys/fs/cgroup/cgroup.procs ] || return
    [ -d /sys/fs/cgroup/_box_web ] && return

    (
      umask 0022
      mkdir /sys/fs/cgroup/_box_web || return
    )

    echo 1-3 > /sys/fs/cgroup/_box_web/cpuset.cpus
    echo 1 > /sys/fs/cgroup/_box_web/cpu.weight
    echo 300 > /sys/fs/cgroup/_box_web/pids.max
    echo 1000000000 > /sys/fs/cgroup/_box_web/memory.high
  }

on this box for box-web.sh, and i can read all my newspapers and
browse the web, even youtube.
If i open to many tabs, they start crashing.  This mostly seems
due to the pids.max, however, i already increased this from 250.

What _i_ find interesting is that for the first time i really get
impressive scheduling feedback.  This cpu.weight=1 can indeed
cause the browser to be delayed entirely for minutes, dependent on
the other tasks on the system.  I mean really, totally, entirely.
I was impressed by reading the sched manual once i came back to
Linux two years ago, but seeing cpu.weight=1 in action _like
that_, really impressive.

Btw .. happy to be on the other side of the spectrum

  steffen   1251  0.0  0.0  11240  6184 tty1     S    Jul05   0:06 cwm
  steffen   1261  0.0  0.1  19284  9096 ?        Ss   Jul05   1:11 st -n stgrey
  steffen   1262  0.0  0.0   8588  3896 pts/0    Ss+  Jul05   0:00 tmux a
  steffen   1334  0.0  1.1  94088 88680 ?        Ss   Jul05   1:25 tmux a
  steffen   5058  0.0  0.1  14672  8712 pts/1    S+   Jul05   0:11 s-nail -Aich
  steffen  30448  0.0  0.2 100800 19052 pts/7    Sl+  20:50   0:00 irssi
  steffen   1340  0.0  0.0  11848  6840 pts/6    Ss+  Jul05   0:00 vim -u /home/steffen/.vimrc

It all fits a single Terminal.app (if not taking into account tmux
history), even if online.  (Granted st does not have the history
patch, and firefox not running.)

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Overgrown ffox (was: The Unix shell: a 50-year view)
  2021-07-08 19:25                               ` Steffen Nurpmeso
@ 2021-07-08 19:37                                 ` Steffen Nurpmeso
  2021-07-08 20:40                                 ` Steffen Nurpmeso
  1 sibling, 0 replies; 109+ messages in thread
From: Steffen Nurpmeso @ 2021-07-08 19:37 UTC (permalink / raw)
  To: Derek Fawcus; +Cc: tuhs

Steffen Nurpmeso wrote in
 <20210708192538.jpOvc%steffen@sdaoden.eu>:
 ...
 |But it is not only Mac, take iwd again, iwctl('s display handling)
 |definetely is buggy in respect to handling of SIGCONT after
 |a TSTP.  (Ie the screen starts scrolling line-wise but no display
 |is ever refreshed.)

..at least if in an active "station X show" or so..

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Overgrown ffox (was: The Unix shell: a 50-year view)
  2021-07-08 19:25                               ` Steffen Nurpmeso
  2021-07-08 19:37                                 ` Steffen Nurpmeso
@ 2021-07-08 20:40                                 ` Steffen Nurpmeso
  1 sibling, 0 replies; 109+ messages in thread
From: Steffen Nurpmeso @ 2021-07-08 20:40 UTC (permalink / raw)
  To: Derek Fawcus; +Cc: tuhs

Steffen Nurpmeso wrote in
 <20210708192538.jpOvc%steffen@sdaoden.eu>:
 ...
 |  steffen   1458 31.2  3.7 3029004 299700 ?      Sl   21:03   0:04 \
 |  firefox --no-remote
 ...
 |Yeah i use --no-remote because otherwise firefox will find its
 |other-box instances via X cookies (i do not have Wayland here).
 |I deem it a critical error that firefox then starts up even though
 |the ~/.mozilla/XXX profile is a totally different one, but on the
 |IRC we just ended like this.

Not to mention that firefox-bin 89.0.1 as compiled mozilla.com
themselves no longer does that right, it will exit the other
instance in another box with another profile directory even if
both instances start with --no-remote.

Heck, software is a complicated beast.
Has to be said, humans produce so many bugs, i am happy that it is
software that controls american warships, nuclear power plants,
the chemical industry, and so many other important things!

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-07 20:50                         ` Michael Kjörling
  2021-07-08  6:46                           ` [TUHS] Overgrown ffox (was: The Unix shell: a 50-year view) Tomasz Rola
@ 2021-07-08 21:47                           ` Theodore Ts'o
  2021-07-09 20:14                             ` Michael Kjörling
  1 sibling, 1 reply; 109+ messages in thread
From: Theodore Ts'o @ 2021-07-08 21:47 UTC (permalink / raw)
  To: Michael Kjörling; +Cc: tuhs

On Wed, Jul 07, 2021 at 08:50:51PM +0000, Michael Kjörling wrote:
> 
> I'm going to stick my neck out here by saying that the VSZ and RSS
> values reported by ps, at least for Firefox, are largely meaningless.

No, VSZ is correct, but it's confusing since it includes things that
you might not expect.  VSZ includes *all* virtual memory space
consumed by a process.

For example, consider this simple program:

main(int argc, char **argv)
{
	printf("Hello, world\n");
	sleep(10);
	return 0;
}

It uses shared libraries:

% ldd /tmp/hello
        linux-vdso.so.1 (0x00007fff20bff000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd8caad9000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fd8cacca000)

If you run the program the ps listing looks like this:

USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
tytso    4060239  0.0  0.0   2300   512 pts/2    T    17:01   0:00 /tmp/hello

If you look at /proc/4060239/maps, you'll memory map for the process.

5559e8e7d000-5559e8e7e000 r--p 00000000 00:25 260006                     /tmp/hello
5559e8e7e000-5559e8e7f000 r-xp 00001000 00:25 260006                     /tmp/hello
5559e8e7f000-5559e8e80000 r--p 00002000 00:25 260006                     /tmp/hello
5559e8e80000-5559e8e81000 r--p 00002000 00:25 260006                     /tmp/hello
5559e8e81000-5559e8e82000 rw-p 00003000 00:25 260006                     /tmp/hello
5559ea383000-5559ea3a4000 rw-p 00000000 00:00 0                          [heap]
7f8419a44000-7f8419a69000 r--p 00000000 fc:00 10763398                   /usr/lib/x86_64-linux-gnu/libc-2.31.so
7f8419a69000-7f8419bb4000 r-xp 00025000 fc:00 10763398                   /usr/lib/x86_64-linux-gnu/libc-2.31.so
			  ...

If you add the sum total of all of the VM regions, i.e.:

(0x5559e8e7e000 - 0x5559e8e7d000) +
(0x5559e8e7f000 - 0x5559e8e7e000) +
(0x5559e8e80000 - 0x5559e8e7f000) +
	...

(ignore the last vsyscall region, since that's a kernel page mapped
into all processes)   You'll get the 2300k of the VSZ.

If the process mmap's in a large 2GB file, the VSZ will go up by 2GB,
even if none of the file is paged in.  Or if you mmap in 2MB of
anonymous memory backed by the zero page for the heap, the VSZ will go
up by 2MB.  

> I started my usual Firefox instance, which has a handful of plugins,
> about a metric gazillion bookmarks, and has been my main web browser
> profile for years (so it probably has collected some crud over time).
> `ps auxw` reported that process as having a total RSS of a whopping
> 374 GB.

I don't think that's right.  Are you sure it's not 374MB?  I started
firefox-esr on my Debian machine, and the PS output is:

USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
tytso    4063201  5.8  1.0 3027868 357248 pts/2  Sl   17:38   0:07 firefox-esr
tytso    4063274  0.5  0.3 2446608 105620 pts/2  Sl   17:38   0:00 /usr/lib/firefox-esr/firefox-esr -contentproc -childID 1 -isF
tytso    4063317  0.8  0.3 2457032 124200 pts/2  Sl   17:38   0:01 /usr/lib/firefox-esr/firefox-esr -contentproc -childID 2 -isF
tytso    4063341  2.9  0.8 2623896 282420 pts/2  Sl   17:38   0:03 /usr/lib/firefox-esr/firefox-esr -contentproc -childID 3 -isF

357248k is 343MB.  Maybe you were adding up the RSS's for all of the
processes?  That's misleading because if a physical 4k page is shared
across multiple process, that 4k page will be accounted in each of the
processes's RSS --- even if that process hasn't actually *used* that
particular page.  So for example, the hello world program had a 512k RSS:

USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
tytso    4060239  0.0  0.0   2300   512 pts/2    T    17:01   0:00 /tmp/hello

That's not because the hello world program actually used half a
megabyte worth of the C library; but rather, almost half a megabyte of
the C library has been paged in to support all of the processes
currently running in the system.  It also follows that every process
using shared library is going to have an RSS which is at least 512k.

Cheers,

					- Ted

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Overgrown ffox (was: The Unix shell: a 50-year view)
  2021-07-08  6:46                           ` [TUHS] Overgrown ffox (was: The Unix shell: a 50-year view) Tomasz Rola
  2021-07-08 13:59                             ` Derek Fawcus
@ 2021-07-08 22:23                             ` Kevin Bowling
  1 sibling, 0 replies; 109+ messages in thread
From: Kevin Bowling @ 2021-07-08 22:23 UTC (permalink / raw)
  To: Tomasz Rola; +Cc: tuhs

[-- Attachment #1: Type: text/plain, Size: 3176 bytes --]

Try typing “about:memory” into the address bar and hit measure.  You will
see where it is all going.

On Wed, Jul 7, 2021 at 11:48 PM Tomasz Rola <rtomek@ceti.pl> wrote:

> On Wed, Jul 07, 2021 at 08:50:51PM +0000, Michael Kjörling wrote:
> > On 7 Jul 2021 20:32 +0200, from rtomek@ceti.pl (Tomasz Rola):
> > > An excerpt from my ps:
> > >
> > > USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME
> COMMAND
> > >
> > > xxxon    12331 12.5 20.4 5898360 2519640 ?     TNsl Mar29 18278:11
> firefox-esr
> >
> > I'm going to stick my neck out here by saying that the VSZ and RSS
> > values reported by ps, at least for Firefox, are largely meaningless.
> >
> > I started my usual Firefox instance, which has a handful of plugins,
> > about a metric gazillion bookmarks, and has been my main web browser
> > profile for years (so it probably has collected some crud over time).
> > `ps auxw` reported that process as having a total RSS of a whopping
> > 374 GB.
> >
> > It is downright _impossible_ that Firefox could actually be using that
>
> This is quite strange for me. Without looking at your system I can only
> suspect it has something to do with multithreading.
>
> If I do two different commands as root, with firefox pid here
> .eq. 12331, as above:
>
> =>  (500 15):    lsof -p 12331 | wc -l
> 402
>
> =>  (500 17):   lsof | awk '$2==12331' | wc -l
> 22055
>
> The first column gives a name, and in second case it not always is
> 'firefox'. I am yet to study manpage for lsof and play with it, but it
> surely shows interesting things.
>
> On my system, when firefox gets killed, 'free' shows a difference - if
> I recall, free mem increases by the size of rss plus all the stuff
> which was opened and released from buffers. I did not pay much
> attention, I assumed numbers would match and this is what they
> probably did :-).
>
> OS on my box used to report to me as Debian, and still does, but some
> years ago I have decided to skip the usual system upgrade, and after
> some more time I started to upgrade various elements by hand. So it is
> more like a tattered patchwork right now. But it does what I expect,
> hopefully.
>
> [...]
> > That's a _factor almost 2300x_ difference between the reported RSS,
> > and the amount of memory that was actually freed up by closing the
> > browser.
>
> Yeah, strange.
>
> [...]
> > On modern systems, with everything from shared libraries to
> > memory-mapped I/O to routine use of memory overcommitting, the
> > resident set size is clearly a poor indicator of the actual amount
> > of memory actively used by a complex process.
>
> Hard to tell - first I would like to learn where the hundred-giga rss
> came from...
>
> --
> Regards,
> Tomasz Rola
>
> --
> ** A C programmer asked whether computer had Buddha's nature.      **
> ** As the answer, master did "rm -rif" on the programmer's home    **
> ** directory. And then the C programmer became enlightened...      **
> **                                                                 **
> ** Tomasz Rola          mailto:tomasz_rola@bigfoot.com             **
>

[-- Attachment #2: Type: text/html, Size: 3989 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-08 21:47                           ` [TUHS] [tuhs] The Unix shell: a 50-year view Theodore Ts'o
@ 2021-07-09 20:14                             ` Michael Kjörling
  0 siblings, 0 replies; 109+ messages in thread
From: Michael Kjörling @ 2021-07-09 20:14 UTC (permalink / raw)
  To: tuhs

On 8 Jul 2021 17:47 -0400, from tytso@mit.edu (Theodore Ts'o):
>> I'm going to stick my neck out here by saying that the VSZ and RSS
>> values reported by ps, at least for Firefox, are largely meaningless.
> 
> No, VSZ is correct, but it's confusing since it includes things that
> you might not expect.  VSZ includes *all* virtual memory space
> consumed by a process.

You may very well be correct. That would, however, seem to me to make
it _correct_, but still not necessarily _meaningful_ for trying to
determine the amount of memory used by a _specific_ process; and
similarly for the RSS.


>> I started my usual Firefox instance, which has a handful of plugins,
>> about a metric gazillion bookmarks, and has been my main web browser
>> profile for years (so it probably has collected some crud over time).
>> `ps auxw` reported that process as having a total RSS of a whopping
>> 374 GB.
> 
> I don't think that's right.  Are you sure it's not 374MB?  I started
> firefox-esr on my Debian machine, and the PS output is:

On this, I believe I must stand corrected.

For my previous post, I checked the man page for ps, and noted that
RSS and VSZ are both reported in units of 1 KiB. For an attempt just
now, the ps output is similarly:

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
michael  29890 51.8  0.7 2728696 255456 tty2   Sl+  21:46   0:06 firefox-esr
michael  29942  4.7  0.3 2415008 106448 tty2   Sl+  21:46   0:00 /usr/lib/firefox-esr/firefox-esr -contentproc -childID 1 -isF
michael  29994 38.3  0.7 34043160 240612 tty2  Sl+  21:46   0:04 /usr/lib/firefox-esr/firefox-esr -contentproc -childID 2 -isF
michael  30168  3.2  0.2 2400636 72992 tty2    Sl+  21:46   0:00 /usr/lib/firefox-esr/firefox-esr -contentproc -childID 3 -isF

On my system, unless I need to get a different calculator, 0.7% of
memory is either 229 ± <16 or 688 ± <49 MiB, depending on whether it
counts only RAM or also includes swap space.

The 249 MiB RSS of the main process also much more closely matches the
difference that was reported by `free` in my previous experiment.
Oddly, when trying the same thing again now, the difference as
reported by `free` is 423 MiB, so I'm not quite sure what's going on
there, but at least all of those numbers are considerably more
_plausible_, both in isolation and in relation to other relevant
measurements.

I'm honestly not sure how I went from the `ps` output (which wasn't
particularly dissimilar from the above, though of course I don't know
what the exact numbers were since I didn't record them) and the unit
of measurement being KiB, to going from KiB straight to GiB.


> That's not because the hello world program actually used half a
> megabyte worth of the C library; but rather, almost half a megabyte of
> the C library has been paged in to support all of the processes
> currently running in the system.  It also follows that every process
> using shared library is going to have an RSS which is at least 512k.

This is definitely a point on which I agree, and which I tried to
point out with the mention of shared libraries in my previous post.

-- 
Michael Kjörling • https://michael.kjorling.se • michael@kjorling.se
 “Remember when, on the Internet, nobody cared that you were a dog?”


^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] [tuhs] The Unix shell: a 50-year view
  2021-07-06 16:05                 ` Clem Cole
@ 2021-07-09 22:19                   ` Tomasz Rola
  0 siblings, 0 replies; 109+ messages in thread
From: Tomasz Rola @ 2021-07-09 22:19 UTC (permalink / raw)
  To: Clem Cole; +Cc: tuhs

On Tue, Jul 06, 2021 at 12:05:04PM -0400, Clem Cole wrote:
[...]
> Ouch!!! This is so much important that you have missed in this statement,
> as this is a great example of not seeing the forest because of all the
> trees.  You are right that C and UNIX's success was intertwined but I think
> you are missing the drivers for that.   They were successful because of
> other things and because they were successful, we have them today.
[...]

Clem, thanks a lot for writing this. I think I have connected few dots
now. Like, I was long aware that there were various groups of computer
users, but never before I had the thought about how this "divide"
contributed so much to creation and adoption of Unix.

I suppose I like Unix even more now.

[...]
> The idea of lots of little programs that cooperate with each other is not
> what IBM and the like wanted/was providing.  They were selling closed
> 'solutions' complete SW systems ("walled gardens" controlled by them or
> their programming minions) and yes needed the big iron they sold.  Note
> the IBM 'solutions were not sold to engineers, their products were sold on
> the golf course to the 'managers' of what we know call IT shops.

Ahem. I do not see myself selling or buying during golf
course... Especially if it has to do with computers...

[...]
> The important thing is that the latter group (not enterprise) did not have
> as much money but was a completely different population.   UNIX is 100% a
> 'Christiansen style Disruption'  ( read his book completely if you have
> not, please).

I will see if I can have my hand on it.

>  It's a 'lessor technology,' running on smaller equipment,
> targeting a new and different consumer who does not care that it is 'not as
> good' as the established products.  If you compare UNIX to IBM's OS or VMS
> for that matter, UNIX does not have the 'features' that are valued by the
> 'enterprise market.'
> 
> Ken/Dennis, *et al*, clearly do not care -- they were building something
> they (the engineers) needed and wanted (thankfully).  And yes it had to run
> on modest hardware because that is what they could afford and had access
> to.  But because since the HW was modest, that forces a mindset of what are
> we really doing on the SW?  How can we do it effectively.  The designers
> are asking an important research question? *Maybe some of these other
> schemes are not necessary*.

I definitely like this attitude.

> You are correct the C grew because of UNIX, but you kind of have it
> backward. I'm a perfect example of what happened.  These new
> microprocessors from Intel/Zilog/Moto/MOS Tech became available to us
> (engineers mind you).  Hey, I was at CMU in the mid-1970s, I even had
> access to the BLISS sources, but most people did not.  A BLISS cross
> compiler binary cost $5K per CPU!!!

Yikes. Five kilodollars, what a deal.

When I learned C, in middle/late 80-ties, it was my third language
after Basic and Pascal. I did not have computer at that time and I
wrote short programs in a notebook. I wonder if I could locate the
notebook and if any of this code would run. But, compared to the first
two, C had something fresh in it. Nowadays, however, this freshness is
a little bit trapped under a ton of supporting libraries. Back at the
time, the fact there was a C compiler on every computer I heard about,
and standard library gave it a bit of unified look, and I could write
C oneliners on less paper, more idiomatic (saves time! all those
begin/end words replaced by mere curly braces, very nice) - all of
this made C very interesting to me.

[...]
> The key point is that UNIX was inexpensive and worked on modest hardware.
> Yes C came with it and that is why I think we use it not BLISS, BCPL, or
> some flavor of PLx today.

I hope C would have been invented even without Unix.

-- 
Regards,
Tomasz Rola

--
** A C programmer asked whether computer had Buddha's nature.      **
** As the answer, master did "rm -rif" on the programmer's home    **
** directory. And then the C programmer became enlightened...      **
**                                                                 **
** Tomasz Rola          mailto:tomasz_rola@bigfoot.com             **

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] The Unix shell: a 50-year view
  2021-07-07 18:28                   ` Jon Steinhart
@ 2021-07-10 11:51                     ` Ralph Corderoy
  2021-07-10 13:54                       ` Henry Bent
  0 siblings, 1 reply; 109+ messages in thread
From: Ralph Corderoy @ 2021-07-10 11:51 UTC (permalink / raw)
  To: The Unix Heritage Society mailing list

Hi,

Jon Steinhart wrote:
> In lectures these days, I'm asking the question "We haven't managed to
> off more than a thousand or so people at a time with a software bug
> yet so what's going to be software's Union Carbide Bhopal moment?"

If buggy code rather than a single bug counts then the software model
written over fifteen years by Neil Ferguson of Imperial College, London,
which has been instrumental in poor UK Government policy decisions on
COVID-19 has easily topped more than a thousand deaths in the net tally.

It was a single 15,000-line file of C, written by a non-programmer.
Eventually, ic.ac.uk released a C++ version which had been worked on by
Microsoft and other volunteers for a month so it could face the public.

   ‘For me the code is not a mess, but it’s all in my head, completely
    undocumented.  Nobody would be able to use it... and I don’t have
    the bandwidth to support individual users.’ ― Neil Ferguson.

Politician Steve Baker MP, a former senior programmer, has been critical
of the public version and commissioned a review by Mike Hearn.  A path
to Hearn's paper starts at
https://threadreaderapp.com/thread/1323897771510943745.html

And another coder critique is at
https://lockdownsceptics.org/code-review-of-fergusons-model/

The numbers from Ferguson's original pre-release C program were
presented by him to Number 10 and were instrumental in setting the UK on
the path of lockdowns.  ‘...lockdowns are the single worst public health
mistake in the last 100 years’ ― Jay Bhattacharya, professor of medicine
at Stanford University.

-- 
Cheers, Ralph.

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] The Unix shell: a 50-year view
  2021-07-10 11:51                     ` [TUHS] " Ralph Corderoy
@ 2021-07-10 13:54                       ` Henry Bent
  2021-07-10 14:12                         ` Ralph Corderoy
  0 siblings, 1 reply; 109+ messages in thread
From: Henry Bent @ 2021-07-10 13:54 UTC (permalink / raw)
  To: Ralph Corderoy; +Cc: The Unix Heritage Society mailing list

[-- Attachment #1: Type: text/plain, Size: 2471 bytes --]

I'm going to come right out ahead of any path to the contrary and say that
I'm in favor of the lockdowns that were enacted in the US.  I very plainly
do not trust the population here to make reasonable decisions, even in the
face of clearly presented evidence to the contrary.  Furthermore, I have
not seen any evidence that US lawmakers acted according to any model
whatsoever.

The evidence being what it is, I applaud UK lawmakers for acting as they
did, and our hindsight evidence can only support increased funding for
statistical modeling.  It wasn't a widely regarded field before this and I
can only hope that its support improves after this.

-Henry

On Sat, 10 Jul 2021 at 08:02, Ralph Corderoy <ralph@inputplus.co.uk> wrote:

> Hi,
>
> Jon Steinhart wrote:
> > In lectures these days, I'm asking the question "We haven't managed to
> > off more than a thousand or so people at a time with a software bug
> > yet so what's going to be software's Union Carbide Bhopal moment?"
>
> If buggy code rather than a single bug counts then the software model
> written over fifteen years by Neil Ferguson of Imperial College, London,
> which has been instrumental in poor UK Government policy decisions on
> COVID-19 has easily topped more than a thousand deaths in the net tally.
>
> It was a single 15,000-line file of C, written by a non-programmer.
> Eventually, ic.ac.uk released a C++ version which had been worked on by
> Microsoft and other volunteers for a month so it could face the public.
>
>    ‘For me the code is not a mess, but it’s all in my head, completely
>     undocumented.  Nobody would be able to use it... and I don’t have
>     the bandwidth to support individual users.’ ― Neil Ferguson.
>
> Politician Steve Baker MP, a former senior programmer, has been critical
> of the public version and commissioned a review by Mike Hearn.  A path
> to Hearn's paper starts at
> https://threadreaderapp.com/thread/1323897771510943745.html
>
> And another coder critique is at
> https://lockdownsceptics.org/code-review-of-fergusons-model/
>
> The numbers from Ferguson's original pre-release C program were
> presented by him to Number 10 and were instrumental in setting the UK on
> the path of lockdowns.  ‘...lockdowns are the single worst public health
> mistake in the last 100 years’ ― Jay Bhattacharya, professor of medicine
> at Stanford University.
>
> --
> Cheers, Ralph.
>

[-- Attachment #2: Type: text/html, Size: 3223 bytes --]

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] The Unix shell: a 50-year view
  2021-07-10 13:54                       ` Henry Bent
@ 2021-07-10 14:12                         ` Ralph Corderoy
  2021-07-10 16:57                           ` [TUHS] Death by bug [formerly The Unix shell: a 50-year view] Jon Steinhart
  0 siblings, 1 reply; 109+ messages in thread
From: Ralph Corderoy @ 2021-07-10 14:12 UTC (permalink / raw)
  To: The Unix Heritage Society mailing list

Hi Henry,

> I'm going to come right out ahead of any path to the contrary and say
> that I'm in favor of the lockdowns that were enacted in the US.

Death-by-bug is OT enough for TUHS to begin with, though ‘entertaining’.
Can we try and keep the thread to that together with any argument for
why a bug caused net deaths.

-- 
Cheers, Ralph.

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Death by bug [formerly The Unix shell: a 50-year view]
  2021-07-10 14:12                         ` Ralph Corderoy
@ 2021-07-10 16:57                           ` Jon Steinhart
  2021-07-11  8:53                             ` [TUHS] Death by bug Ralph Corderoy
  0 siblings, 1 reply; 109+ messages in thread
From: Jon Steinhart @ 2021-07-10 16:57 UTC (permalink / raw)
  To: The Unix Heritage Society mailing list

Ralph Corderoy writes:
> Hi Henry,
>
> > I'm going to come right out ahead of any path to the contrary and say
> > that I'm in favor of the lockdowns that were enacted in the US.
>
> Death-by-bug is OT enough for TUHS to begin with, though ‘entertaining’.
> Can we try and keep the thread to that together with any argument for
> why a bug caused net deaths.
>
> -- 
> Cheers, Ralph.

C'mon folks, is it really that hard to have a relevant subject line?
Especially you Ralph, as I know that you're an nmh user so it's just
a text editor :-)

So while I agree with your example, I guess that I was contemplating
the sort of example that makes the public take notice.  Arguments
about statistical models are lost on 99% of the population.

Jon


^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Death by bug
  2021-07-10 16:57                           ` [TUHS] Death by bug [formerly The Unix shell: a 50-year view] Jon Steinhart
@ 2021-07-11  8:53                             ` Ralph Corderoy
  2021-07-11  9:04                               ` arnold
  2021-07-11 16:10                               ` Jon Steinhart
  0 siblings, 2 replies; 109+ messages in thread
From: Ralph Corderoy @ 2021-07-11  8:53 UTC (permalink / raw)
  To: The Unix Heritage Society mailing list

Hi Jon,

> So while I agree with your example, I guess that I was contemplating
> the sort of example that makes the public take notice.  Arguments
> about statistical models are lost on 99% of the population.

True.

I don't know of any but if I had to pick an area where they might be
occurring it would be medical devices.  I've been an external reviewer
of a company's embedded C code for a medical device in the past,
bare-metal, simple pre-emptive round-robin task-switcher, etc., though
my suggestion is nothing to do with their quality of work or products,
just that the level of detail they went to shows how slacker companies
could err.

The Therac-25 gave fatal radiation doses in a few cases.  These were
obvious when they occurred.  https://en.wikipedia.org/wiki/Therac-25

What if a medical device which sits in the background quietly doing its
continuous, but sometimes vital, thing had the occasional hiccough from
which it recovered?  If the blip is only fatal in patients which were
already touch and go and device fault left no obvious sign to suggest
death wasn't caused by the initial ailment then the cause of death would
be, quite reasonably, assumed.  

Given some devices are present in large numbers for many years in
hospitals, and there's a lot of hospitals, an unnoticed bug could be
steadily chipping away at its human overlords.

-- 
Cheers, Ralph.

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Death by bug
  2021-07-11  8:53                             ` [TUHS] Death by bug Ralph Corderoy
@ 2021-07-11  9:04                               ` arnold
  2021-07-12  1:42                                 ` Theodore Y. Ts'o
  2021-07-11 16:10                               ` Jon Steinhart
  1 sibling, 1 reply; 109+ messages in thread
From: arnold @ 2021-07-11  9:04 UTC (permalink / raw)
  To: tuhs, ralph

Ralph Corderoy <ralph@inputplus.co.uk> wrote:

> Given some devices are present in large numbers for many years in
> hospitals, and there's a lot of hospitals, an unnoticed bug could be
> steadily chipping away at its human overlords.

This is why I have purposely stayed away from jobs at companies doing
stuff like this. I know I don't write perfect code; I don't want to
be responsible for devices that can affect human life.  This is also
discussed in the new edition of "The Pragmatic Programmer", which I've
just finished reading. (Highly recommended.)

Arnold

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Death by bug
  2021-07-11  8:53                             ` [TUHS] Death by bug Ralph Corderoy
  2021-07-11  9:04                               ` arnold
@ 2021-07-11 16:10                               ` Jon Steinhart
  2021-07-12 10:37                                 ` Ralph Corderoy
  1 sibling, 1 reply; 109+ messages in thread
From: Jon Steinhart @ 2021-07-11 16:10 UTC (permalink / raw)
  To: tuhs

Ralph Corderoy writes:
> Hi Jon,
>
> > So while I agree with your example, I guess that I was contemplating
> > the sort of example that makes the public take notice.  Arguments
> > about statistical models are lost on 99% of the population.
>
> True.
>
> I don't know of any but if I had to pick an area where they might be
> occurring it would be medical devices.  I've been an external reviewer
> of a company's embedded C code for a medical device in the past,
> bare-metal, simple pre-emptive round-robin task-switcher, etc., though
> my suggestion is nothing to do with their quality of work or products,
> just that the level of detail they went to shows how slacker companies
> could err.
>
> The Therac-25 gave fatal radiation doses in a few cases.  These were
> obvious when they occurred.  https://en.wikipedia.org/wiki/Therac-25
>
> What if a medical device which sits in the background quietly doing its
> continuous, but sometimes vital, thing had the occasional hiccough from
> which it recovered?  If the blip is only fatal in patients which were
> already touch and go and device fault left no obvious sign to suggest
> death wasn't caused by the initial ailment then the cause of death would
> be, quite reasonably, assumed.  
>
> Given some devices are present in large numbers for many years in
> hospitals, and there's a lot of hospitals, an unnoticed bug could be
> steadily chipping away at its human overlords.
>
> -- 
> Cheers, Ralph.

Well, we agree once again.  I have done a lot of work on medical devices,
and was surprised at the lack of diligence on the part of some of the
people with whom I worked.  But your example is similar to the way that
people have trouble with COVID - there's too much of a lag between
exposure and symptoms for many folks to make the connection.  So I'm kind
of wondering what sort of bug may blow up a refinery or chemical plant and
cause enough loss of life and damage to make people take notice.

Jon

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Death by bug
  2021-07-11  9:04                               ` arnold
@ 2021-07-12  1:42                                 ` Theodore Y. Ts'o
  2021-07-12  2:57                                   ` Jon Steinhart
                                                     ` (2 more replies)
  0 siblings, 3 replies; 109+ messages in thread
From: Theodore Y. Ts'o @ 2021-07-12  1:42 UTC (permalink / raw)
  To: arnold; +Cc: tuhs

On Sun, Jul 11, 2021 at 03:04:53AM -0600, arnold@skeeve.com wrote:
> Ralph Corderoy <ralph@inputplus.co.uk> wrote:
> 
> > Given some devices are present in large numbers for many years in
> > hospitals, and there's a lot of hospitals, an unnoticed bug could be
> > steadily chipping away at its human overlords.
> 
> This is why I have purposely stayed away from jobs at companies doing
> stuff like this. I know I don't write perfect code; I don't want to
> be responsible for devices that can affect human life.  This is also
> discussed in the new edition of "The Pragmatic Programmer", which I've
> just finished reading. (Highly recommended.)

We should never be depending on a human being able to write "perfect
code".  Instead, we need to come up with processes so that imperfect
code doesn't escape into production *despite* the fact that humans are
fallible.  Such processes might include requiring unit tests,
integration tests, stress tests, etc., requiring code reivews by a
second pair of eyes, perhaps using formal proofs, having multiple
implementations of critical algorithms, cross-checking the results
from those independent implementations, and so on.

The space shuttle used a number of these techniques.  It did *not*
depend on super-human, Über-programmers.

						- Ted

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Death by bug
  2021-07-12  1:42                                 ` Theodore Y. Ts'o
@ 2021-07-12  2:57                                   ` Jon Steinhart
  2021-07-12  6:39                                   ` arnold
  2021-07-12  9:56                                   ` Ralph Corderoy
  2 siblings, 0 replies; 109+ messages in thread
From: Jon Steinhart @ 2021-07-12  2:57 UTC (permalink / raw)
  To: tuhs

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="UTF-8", Size: 2413 bytes --]

Theodore Y. Ts'o writes:
> On Sun, Jul 11, 2021 at 03:04:53AM -0600, arnold@skeeve.com wrote:
> > Ralph Corderoy <ralph@inputplus.co.uk> wrote:
> > 
> > > Given some devices are present in large numbers for many years in
> > > hospitals, and there's a lot of hospitals, an unnoticed bug could be
> > > steadily chipping away at its human overlords.
> > 
> > This is why I have purposely stayed away from jobs at companies doing
> > stuff like this. I know I don't write perfect code; I don't want to
> > be responsible for devices that can affect human life.  This is also
> > discussed in the new edition of "The Pragmatic Programmer", which I've
> > just finished reading. (Highly recommended.)
>
> We should never be depending on a human being able to write "perfect
> code".  Instead, we need to come up with processes so that imperfect
> code doesn't escape into production *despite* the fact that humans are
> fallible.  Such processes might include requiring unit tests,
> integration tests, stress tests, etc., requiring code reivews by a
> second pair of eyes, perhaps using formal proofs, having multiple
> implementations of critical algorithms, cross-checking the results
> from those independent implementations, and so on.
>
> The space shuttle used a number of these techniques.  It did *not*
> depend on super-human, Über-programmers.
>
> 						- Ted

In general I agree with you.  But the "we should never" ignores the fact
that we do.  It's a complicated problem.  A major component is that
our "legal" system allows companies to externalize their liabilities.
And it's difficult to come up a way to assign liability without severely
harming the open source world.  As I've said before, if your car crashed
as often as your Windows you probably wouldn't keep buying from the same
manufacturer.

Part of my preferred process is to have two software review teams.  One
has access to the source code, the other doesn't.  But of course this is
now often not possible when companies used closed-source components that
they purchase.

As someone who agrees with Feynman's dissent on the Challenger commission,
I think that incremental improvement is a good way to go.  But, we live in
an age where many folks would rather come up with something bloated and new
rather than build on what's come before.  That's making it more and more
difficult to distinguish between "full stack" and a landfill.

Jon

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Death by bug
  2021-07-12  1:42                                 ` Theodore Y. Ts'o
  2021-07-12  2:57                                   ` Jon Steinhart
@ 2021-07-12  6:39                                   ` arnold
  2021-07-12  9:56                                   ` Ralph Corderoy
  2 siblings, 0 replies; 109+ messages in thread
From: arnold @ 2021-07-12  6:39 UTC (permalink / raw)
  To: tytso, arnold; +Cc: tuhs

"Theodore Y. Ts'o" <tytso@mit.edu> wrote:

> On Sun, Jul 11, 2021 at 03:04:53AM -0600, arnold@skeeve.com wrote:
> > This is why I have purposely stayed away from jobs at companies doing
> > stuff like this. I know I don't write perfect code; I don't want to
> > be responsible for devices that can affect human life.  This is also
> > discussed in the new edition of "The Pragmatic Programmer", which I've
> > just finished reading. (Highly recommended.)
>
> We should never be depending on a human being able to write "perfect
> code".  Instead, we need to come up with processes so that imperfect
> code doesn't escape into production *despite* the fact that humans are
> fallible.  Such processes might include requiring unit tests,
> integration tests, stress tests, etc., requiring code reivews by a
> second pair of eyes, perhaps using formal proofs, having multiple
> implementations of critical algorithms, cross-checking the results
> from those independent implementations, and so on.
>
> The space shuttle used a number of these techniques.  It did *not*
> depend on super-human, Über-programmers.

I strongly agree with all that. But given that many places don't use such
practices (especially startups), I prefer not to put myself into
situations where safety of the product depends entirely on the skills
of the programming team.

Arnold

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Death by bug
  2021-07-12  1:42                                 ` Theodore Y. Ts'o
  2021-07-12  2:57                                   ` Jon Steinhart
  2021-07-12  6:39                                   ` arnold
@ 2021-07-12  9:56                                   ` Ralph Corderoy
  2 siblings, 0 replies; 109+ messages in thread
From: Ralph Corderoy @ 2021-07-12  9:56 UTC (permalink / raw)
  To: tuhs

Hi Ted,

> > This is why I have purposely stayed away from jobs at companies
> > doing stuff like this. I know I don't write perfect code; I don't
> > want to be responsible for devices that can affect human life.
>
> We should never be depending on a human being able to write "perfect
> code".

And no one has suggested we do; Arnold just pointed out he knows doesn't
which is a good first step to working on critical software.

> Instead, we need to come up with processes so that imperfect code
> doesn't escape into production *despite* the fact that humans are
> fallible.  Such processes might include requiring unit tests,
> integration tests, stress tests, etc., requiring code reivews by a
> second pair of eyes, perhaps using formal proofs, having multiple
> implementations of critical algorithms, cross-checking the results
> from those independent implementations, and so on.

Haven't you just pushed the need for perfection from coding to processes
to achieve ‘imperfect code doesn't escape into production’.  Perfection
doesn't exist there either.

-- 
Cheers, Ralph.

^ permalink raw reply	[flat|nested] 109+ messages in thread

* Re: [TUHS] Death by bug
  2021-07-11 16:10                               ` Jon Steinhart
@ 2021-07-12 10:37                                 ` Ralph Corderoy
  0 siblings, 0 replies; 109+ messages in thread
From: Ralph Corderoy @ 2021-07-12 10:37 UTC (permalink / raw)
  To: tuhs

Hi Jon,

> So I'm kind of wondering what sort of bug may blow up a refinery or
> chemical plant and cause enough loss of life and damage to make people
> take notice.

I see, a ‘big bang’ disaster, like Bhopal, rather than a gradual
sniping.

There's the Battery Management System software which controls the
charging and discharge of Lithium-ion batteries.  I don't mean a burning
laptop causing an airliner to go down, more a shipping container stuffed
full of the things going up.

There is an interesting recent paper on the flaws in ‘Battery Energy
Storage Systems’ and defending against the inevitable lithium metal
dendrites.  It assumes the BMS has no flaws but if it did then the same
end effect could occur.  Here's some extracts from the ‘Executive
Summary’.

    Safety of Grid Scale Lithium-ion Battery Energy Storage Systems
    https://www.researchgate.net/publication/352158070_Safety_of_Grid_Scale_Lithium-ion_Battery_Energy_Storage_Systems

    Li-ion batteries are dominant in large, grid-scale, Battery Energy
    Storage Systems (BESS) of several MWh and upwards in capacity.
    Several proposals for large-scale solar photovoltaic (PV) “energy
    farms” are current, incorporating very large capacity BESS.

    Despite storing electrochemical energy of many hundreds of tons of
    TNT equivalent, and several times the energy released in the August
    2020 Beirut explosion, these BESS are regarded as “articles” by the
    Health and Safety Executive (HSE), in defiance of the Control of
    Major Accident Hazards Regulations (COMAH) 2015, intended to
    safeguard public health...

    Li-ion batteries can fail by “thermal runaway” where overheating in
    a single faulty cell can propagate to neighbours with energy
    releases popularly known as “battery fires”.  These are not strictly
    “fires” at all, requiring no oxygen to propagate.  They are
    uncontrollable except by extravagant water cooling.  They evolve
    toxic gases such as Hydrogen Fluoride (HF) and highly inflammable
    gases including Hydrogen (H2), Methane (CH4), Ethylene (C2H4) and
    Carbon Monoxide (CO).  These in turn may cause further explosions or
    fires upon ignition.  The chemical energy then released can be up to
    20 times the stored electrochemical energy.

    “Battery fires” in grid scale BESS have occurred in South Korea,
    Belgium (2017), Arizona (2019) and in urban Liverpool (Sept 2020)...
    A report into the Liverpool “fire” though promised for New Year
    2021, has not yet been released...

    No existing engineering standards address thermal runaway
    adequately, or require measures (such as those already used in EV
    batteries) to pre-empt propagation of runaway events.

And BMS's get a few mentions, e.g.:

    Li-ion batteries are sensitive to mechanical damage and electrical
    surges, both in over-charging and discharging.  Most of this can
    however be safeguarded by an appropriate Battery Management System
    (BMS)...

-- 
Cheers, Ralph.

^ permalink raw reply	[flat|nested] 109+ messages in thread

end of thread, other threads:[~2021-07-12 10:38 UTC | newest]

Thread overview: 109+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-07-02 21:24 [TUHS] [tuhs] The Unix shell: a 50-year view Nelson H. F. Beebe
2021-07-02 21:36 ` Larry McVoy
2021-07-02 21:56   ` Henry Bent
2021-07-02 23:12     ` Steve Nickolas
2021-07-02 23:49       ` Steffen Nurpmeso
2021-07-03 13:34         ` Steffen Nurpmeso
2021-07-03 13:56           ` Richard Salz
2021-07-03 12:04       ` Thomas Paulsen
2021-07-03 13:20         ` Dan Cross
2021-07-03 17:37           ` Theodore Ts'o
2021-07-03 17:57             ` Warner Losh
2021-07-03 18:10               ` Theodore Ts'o
2021-07-03 20:02                 ` Dan Cross
2021-07-04  0:47           ` Tomasz Rola
2021-07-04  4:36             ` Larry McVoy
2021-07-04 14:56               ` Dan Cross
2021-07-04 16:07               ` Theodore Ts'o
2021-07-04 20:10               ` David Barto
2021-07-05  0:25                 ` Larry McVoy
2021-07-05  1:23                 ` John Cowan
2021-07-04 12:48             ` Dan Cross
2021-07-05  7:14               ` Tomasz Rola
2021-07-05 16:26                 ` John Cowan
2021-07-06 23:17                   ` Tomasz Rola
2021-07-06 23:47                     ` Steve Nickolas
2021-07-06 23:49                       ` Warner Losh
2021-07-06 23:48                     ` John Cowan
2021-07-07  0:46                     ` Theodore Ts'o
2021-07-07  0:58                       ` George Michaelson
2021-07-07  2:48                         ` Larry McVoy
2021-07-07 18:32                       ` Tomasz Rola
2021-07-07 20:50                         ` Michael Kjörling
2021-07-08  6:46                           ` [TUHS] Overgrown ffox (was: The Unix shell: a 50-year view) Tomasz Rola
2021-07-08 13:59                             ` Derek Fawcus
2021-07-08 19:25                               ` Steffen Nurpmeso
2021-07-08 19:37                                 ` Steffen Nurpmeso
2021-07-08 20:40                                 ` Steffen Nurpmeso
2021-07-08 22:23                             ` Kevin Bowling
2021-07-08 21:47                           ` [TUHS] [tuhs] The Unix shell: a 50-year view Theodore Ts'o
2021-07-09 20:14                             ` Michael Kjörling
2021-07-07 13:54                     ` Tony Finch
2021-07-06 16:05                 ` Clem Cole
2021-07-09 22:19                   ` Tomasz Rola
2021-07-04 20:10           ` Tony Finch
2021-07-05  3:59             ` Theodore Ts'o
2021-07-05 15:08               ` Steffen Nurpmeso
2021-07-05  3:52           ` Bakul Shah
2021-07-04 18:17     ` John Dow via TUHS
2021-07-04 19:46       ` Clem Cole
2021-07-05  1:33         ` Noel Hunt
2021-07-05  2:38           ` Clem Cole
2021-07-05  2:51             ` Warner Losh
2021-07-05  3:03               ` Clem Cole
2021-07-05  3:01             ` Clem Cole
2021-07-05  5:22             ` Noel Hunt
2021-07-06  5:10           ` Nevin Liber
2021-07-06 13:30             ` Clem Cole
2021-07-06 16:23               ` Theodore Ts'o
2021-07-07  1:57                 ` Dan Cross
2021-07-07  2:52                   ` Larry McVoy
2021-07-07  5:19                     ` Andrew Warkentin
2021-07-07 18:28                   ` Jon Steinhart
2021-07-10 11:51                     ` [TUHS] " Ralph Corderoy
2021-07-10 13:54                       ` Henry Bent
2021-07-10 14:12                         ` Ralph Corderoy
2021-07-10 16:57                           ` [TUHS] Death by bug [formerly The Unix shell: a 50-year view] Jon Steinhart
2021-07-11  8:53                             ` [TUHS] Death by bug Ralph Corderoy
2021-07-11  9:04                               ` arnold
2021-07-12  1:42                                 ` Theodore Y. Ts'o
2021-07-12  2:57                                   ` Jon Steinhart
2021-07-12  6:39                                   ` arnold
2021-07-12  9:56                                   ` Ralph Corderoy
2021-07-11 16:10                               ` Jon Steinhart
2021-07-12 10:37                                 ` Ralph Corderoy
2021-07-06 13:40             ` [TUHS] [tuhs] The Unix shell: a 50-year view John Cowan
2021-07-06 14:12             ` Chet Ramey
2021-07-07  0:53               ` Nevin Liber
2021-07-07 13:08                 ` Chet Ramey
2021-07-07 15:15                   ` Richard Salz
2021-07-03  0:09   ` Andrew Warkentin
2021-07-03 15:49   ` Andy Kosela
2021-07-04 23:24     ` [TUHS] Is C obsolete? (was Re: [tuhs] The Unix shell: a 50-year view) Derek Fawcus
2021-07-04 23:50       ` Nemo Nusquam
2021-07-05  0:15         ` Dan Stromberg
2021-07-05  0:21       ` Larry McVoy
2021-07-05  2:36         ` John Cowan
2021-07-05  2:59           ` Richard Salz
2021-07-05  3:47           ` Larry McVoy
2021-07-05  4:02             ` Dan Stromberg
2021-07-05 13:45               ` Steffen Nurpmeso
2021-07-05 20:15                 ` Dan Stromberg
2021-07-05 21:05                   ` Larry McVoy
2021-07-05 21:29                   ` Clem Cole
2021-07-05 22:22                     ` Brantley Coile
2021-07-06  4:35                     ` Dan Stromberg
2021-07-06  4:44                       ` Warner Losh
2021-07-06  5:58                       ` Rico Pajarola
2021-07-06 13:05                       ` Clem Cole
2021-07-05 12:11         ` Thomas Paulsen
2021-07-05  4:08       ` Dan Stromberg
2021-07-05  4:23         ` George Michaelson
2021-07-05 14:43           ` Larry McVoy
2021-07-05 15:17             ` Steffen Nurpmeso
2021-07-05 15:36               ` Steffen Nurpmeso
2021-07-05 15:53       ` Mike Markowski
2021-07-05 16:39       ` Warner Losh
2021-07-05 19:02         ` Clem Cole
2021-07-02 22:27 ` [TUHS] [tuhs] The Unix shell: a 50-year view Chet Ramey
2021-07-02 23:09 ` Steve Nickolas

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).