[TUHS] On Bloat and the Idea of Small Specialized Tools

[TUHS] On Bloat and the Idea of Small Specialized Tools

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 8116 bytes --]

While the idea of small tools that do one job well is the core tenant of
what I think of as the UNIX philosophy, this goes a bit beyond UNIX, so I
have moved this discussion to COFF and BCCing TUHS for now.

The key is that not all "bloat" is the same (really)—or maybe one person's
bloat is another person's preference.  That said, NIH leads to pure bloat
with little to recommend it, while multiple offerings are a choice. Maybe
the difference between the two may be one person's view over another.

On Fri, May 10, 2024 at 6:08 AM Rob Pike <robpike@gmail.com> wrote:

> Didn't recognize the command, looked it up. Sigh.
>
Like Rob -- this was a new one for me, too.
I looked, and it is on the SYS3 tape; see:
https://www.tuhs.org/cgi-bin/utree.pl?file=SysIII/usr/src/man/man1/nl.1


>   pr -tn <file>
>

> seems sufficient for me, but then that raises the question of your
> question.
>
Agreed, that has been burned into the ROMs in my  fingers since the
mid-1970s 😀
BTW: SYS3 has pr(1) with both switches too  (more in a minute)


> I've been developing a theory about how the existence of something leads
> to things being added to it that you didn't need at all and only thought of
> when the original thing was created.
>
That is a good point, and I generally agree with you.


> Bloat by example, if you will. I suspect it will not be a popular theory,
> however accurately it may describe the technological world.
>

Of course, sometimes the new features >>are<< easier (more natural *for
some people*).  And herein lies the core problem. The bloat is often
repetitive, and I suggest that it is often implemented in the wrong place -
and usually for the wrong reasons.

Bloat comes about because somebody thinks they need some feature and
probably doesn't understand that it is already there or how they can use
it. But they do know about it, their tool must be set up to exploit it - so
they do not need to reinvent it.  GUI-based tools are notorious for this
failure. Everyone seems to have a built-in (unique) editor, or a private
way to set up configuration options et al. But ... that walled garden is
comfortable for many users and >>can be<< useful sometimes.

Long ago, UNIX programmers learned that looking for $EDITOR in the
environment was way better than creating one.  Configuration was as ASCII
text, stored in /etc for system-wide and dot files in the home for users.
But it also means the >>output<< of each tool needs to be usable by each
other [*i.e.*, docx or xlx files are a no-no).

For example, for many things on my Mac, I do use the GUI-based tools --
there is no doubt they are better integrated with the core Mac system >>for
some tasks.<< But only if I obey a set of rules Apple decrees.  For
instance, this email read is easier much of the time than MH (or the HM
front end, for that matter), which I used for probably 25-30 years. But on
my Mac, I always have 4 or 5 iterm2(1) open running zsh(1) these days. And,
much of my typing (and everything I do as a programmer) is done in the shell
(including a simple text editor, not an 'IDE').  People who love IDEs swear
by them -- I'm just not impressed - there is nothing they do for me that
makes it easier, and I have learned yet another scheme.

That said, sadly, Apple is forcing me to learn yet another debugger since
none of the traditional UNIX-based ones still work on the M1-based systems.
But at least LLDB is in the same key as sdb/dbx/gdb *et al*., so it is a
PITA but not a huge thing as, in the end, LLDB is still based on the UNIX
idea of a single well-designed and specific to the task tool, to do each
job and can work with each other.

FWIW: I was recently a tad gob-smacked by the core idea of UNIX and its
tools, which I have taken for a fact since the 1970s.

It turns out that I've been helping with the PiDP-10 users (all of the
PiDPs are cool, BTW). Before I saw UNIX, I was paid to program a PDP-10. In
fact, my first UNIX job was helping move programs from the 10 to the UNIX.
Thus ... I had been thinking that doing a little PDP-10 hacking shouldn't
be too hard to dust off some of that old knowledge.  While some of it has,
of course, come back.  But daily, I am discovering small things that are so
natural with a few simple tools can be hard on those systems.

I am realizing (rediscovering) that the "build it into my tool" was the
norm in those days.   So instead of a pr(1) command, there was a tool that
created output to the lineprinter. You give it a file, and it is its job to
figure out what to do with it, so it has its set of features (switches) -
so "bloat" is that each tool (like many current GUI tools) has private ways
of doing things. If the maker of tool X decided to support some idea, they
would do it like tool Y.  The problem, of course, was that tools X and Y
had to 'know about' each type of file (in IBM terms, use its "access
method").  Yes, the engineers at DEC, in their wisdom, tried to
"standardize" those access methods/switches/features >>if you implemented
them<< -- but they are not all there.

This leads me back to the question Rob raises.  Years ago, I got into an
argument with Dave Cutler RE: UNIX *vs.* VMS. Dave's #1 complaint about
UNIX in those days was that it was not "standardized."  Every program was
different, and more to Dave's point, there was no attempt to make switches
or errors the same [getopt(3) had been introduced but was not being used by
most applications).  He hated that tar/tp used "keys" and tools like cpio
used switches.  Dave hated that I/O was so simple - in his world all user
programs should use his RMS access method of course [1].  VMS, TOPS, *etc.*,
tried to maintain a system-wide error scheme, and users could look things
like errors up in a system DB by error number, *etc*.  Simply put, VMS is
very "top-down."

My point with Dave was that by being "bottom-up," the best ideas in  UNIX
were able to rise. And yes, it did mean some rough edges and repeated
implementations of the same idea.  But UNIX offered a choice, and while Rob
and I like and find: pr -tn perfectly acceptable thank you, clearly someone
else desired the features that nl provides. The folks that put together
System 3 offer both solutions and let the user choose.

This, of course, comes as bloat, but maybe that is a type of bloat so bad?


My own thinking is this - get things down to the basics and simplest
privatives and then build back up.  It's okay to offer choices, as long as
the foundation is simple and clean.  To me, bloat becomes an issue when you
do the same thing over and over again, particularly because you can not
utilize what is there already, the worst example is NIH - which happens way
more than it should.


I think the kind of bloat that GUI tools and TOPS et al. created forces
recreation, not reuse. But offering choice and the expense of multiple
tools that do the same things strikes me as reasonable/probably a good
thing.


1.]  BTW: One of my favorite DEC stories WRT to VMS engineering has to do
with the RMS I/O system.  Supporting C using VMS was a bit of PITA.
 Eventually, the VMS engineers added Stream I/O - which simplified the C
runtime, but it was also made available for all technical languages.
Fairly soon after it was released, the DEC Marketing folks discovered
almost all new programs, regardless of language, had started to use Stream
I/O and many older programs were being rewritten by customers to use it. In
fact, inside of DEC itself, the languages group eventually rewrote things
like the FTN runtime to use streams, making it much smaller/easier to
maintain.   My line in the old days: "It's not so bad that ever I/O has
offer 1000 options, it's that Dave to check each one for every I/O. It's a
classic example of how you can easily build RMS I/O out of stream-based
I/O, but the other way around is much harder.   My point here is to *use
the right primitives*. RMS may have made it easier to build RDB, but it
impeded everything else.

[-- Attachment #2: Type: text/html, Size: 15955 bytes --]

[TUHS] Re: [COFF] Re: On Bloat and the Idea of Small Specialized Tools

[Adam Thornton]

[-- Attachment #1: Type: text/plain, Size: 2362 bytes --]

On Sat, May 11, 2024 at 2:35 PM Theodore Ts'o <tytso@mit.edu> wrote:

>
> I bet most of the young'uns would not be trying to do this as a shell
> script, but using the Cloud SDK with perl or python or Go, which is
> *way* more bloaty than using /bin/sh.
>
> So while some of us old farts might be bemoaning the death of the Unix
> philosophy, perhaps part of the reality is that the Unix philosophy
> were ideal for a simpler time, but might not be as good of a fit
> today


I'm finding myself in agreement.  I might well do this with jq, but as you
point out, you're using the jq DSL pretty extensively to pull out the
fields.  On the other hand, I don't think that's very different than piping
stuff through awk, and I don't think anyone feels like _that_ would be
cheating.  And jq -L is pretty much equivalent to awk -F, which is how I
would do this in practice, rather than trying to inline the whole jq bit.

But it does come down to the same argument as
https://www.microsoft.com/en-us/research/uploads/prod/2019/04/fork-hotos19.pdf

And it is true that while fork() is a great model for single-threaded
pipeline-looking tasks, it's not really what you want for an interactive
multithreaded application on your phone's GUI.

Oddly, I'd have a slightly different reason for reaching for Python (which
is probably how I'd do this anyway), and that's the batteries-included
bit.  If I write in Python, I've got the gcloud api available as a Python
module, and I've got a JSON parser also available as a Python module (but I
bet all the JSON unmarshalling is already handled in the gcloud library),
and I don't have to context-switch to the same degree that I would if I
were stringing it together in the shell.  Instead of "make an HTTP request
to get JSON text back, then parse that with repeated calls to jq", I'd just
get an object back from the instance fetch request, pick out the fields I
wanted, and I'd be done.

I'm afraid only old farts write anything in Perl anymore.  The kids just
mutter "OK, Boomer" when you try to tell them how much better CPAN was than
PyPi.  And it sure feels like all the cool kids have abandoned Go for Rust,
although Go would be a perfectly reasonable choice for this task as well
(and would look a lot like Python: get an object back, pick off the useful
fields).

Adam

[-- Attachment #2: Type: text/html, Size: 2961 bytes --]

[TUHS] Re: [COFF] Re: On Bloat and the Idea of Small Specialized Tools

[Larry McVoy]

On Sun, May 12, 2024 at 12:34:20PM -0700, Adam Thornton wrote:
> But it does come down to the same argument as
> https://www.microsoft.com/en-us/research/uploads/prod/2019/04/fork-hotos19.pdf
> 
> And it is true that while fork() is a great model for single-threaded
> pipeline-looking tasks, it's not really what you want for an interactive
> multithreaded application on your phone's GUI.

Perhaps a meaningless aside, but I agree on fork().  In the last major
project I did, which was cross platform {windows,macos, all the major
Unices, Linux}, we adopted spawn() rather than fork/exec.  There is no way
(that I know of) to fake fork() on Windows but it's easy to fake spawn().

--lm

[TUHS] Re: forking, Re: [COFF] Re: On Bloat and the Idea of Small Specialized Tools

[John Levine]

It appears that Larry McVoy <lm@mcvoy.com> said:
>Perhaps a meaningless aside, but I agree on fork().  In the last major
>project I did, which was cross platform {windows,macos, all the major
>Unices, Linux}, we adopted spawn() rather than fork/exec.  There is no way
>(that I know of) to fake fork() on Windows but it's easy to fake spawn().

The whole point of fork() is that it lets you get the effect of spawn with
a lot less internal mechanism.  Spawn is equivalent to:

  fork()
  ... do stuff to files and environment ...
  exec()

By separating the fork and the exec, they didn't have to put all of
the stuff in the 12 paragraphs in the spawn() man page into the the
tiny PDP-11 kernel.

These days now that programs include multi-megabyte shared libraries
just for fun, I agree that the argument is less persuasive.  On the
third hard, we now understand virtual memory and paging systems a lot
better so we don't need kludges like vfork().

R's,
John

[TUHS] Re: [COFF] Re: On Bloat and the Idea of Small Specialized Tools

[Dave Horsfall]

[-- Attachment #1: Type: text/plain, Size: 570 bytes --]

On Sun, 12 May 2024, Adam Thornton wrote:

> I'm afraid only old farts write anything in Perl anymore.  The kids just
> mutter "OK, Boomer" when you try to tell them how much better CPAN was than
> PyPi.  And it sure feels like all the cool kids have abandoned Go for Rust,
> although Go would be a perfectly reasonable choice for this task as well
> (and would look a lot like Python: get an object back, pick off the useful
> fields).

I must be an old fart then; the last language I used where white space was 
part of the syntax was FORTRAN...

-- Dave

[TUHS] Re: forking, Re: [COFF] Re: On Bloat and the Idea of Small Specialized Tools

[Dan Cross]

On Sun, May 12, 2024 at 4:14 PM John Levine <johnl@taugh.com> wrote:
> It appears that Larry McVoy <lm@mcvoy.com> said:
> >Perhaps a meaningless aside, but I agree on fork().  In the last major
> >project I did, which was cross platform {windows,macos, all the major
> >Unices, Linux}, we adopted spawn() rather than fork/exec.  There is no way
> >(that I know of) to fake fork() on Windows but it's easy to fake spawn().
>
> The whole point of fork() is that it lets you get the effect of spawn with
> a lot less internal mechanism.  Spawn is equivalent to:
>
>   fork()
>   ... do stuff to files and environment ...
>   exec()
>
> By separating the fork and the exec, they didn't have to put all of
> the stuff in the 12 paragraphs in the spawn() man page into the the
> tiny PDP-11 kernel.

Perhaps, but as I've written here before, `fork`/`exec` vs `spawn` is
a false dichotomy. Another alternative is a `proccreate`/`procrun`
pair, the former of which creates an unrunnable process, the latter of
which marks it runnable. Coupled with a set of primitives to
manipulate the state of an extant, but unrunnable, process and you
have the advantages of fork/exec without the downsides (which are
well-known; https://www.microsoft.com/en-us/research/uploads/prod/2019/04/fork-hotos19.pdf).
Similarly, this gives you the functionality of spawn, without the
downside of a singularly complicated interface. Could you have
implemented that in something as small as the PDP-7? Perhaps not, but
it does not follow that `fork` now remains a good primitive.

My spelunking in the original GENIE documentation leads me to believe
that its `fork` provided functionality similar to what I described.

        - Dan C.

[TUHS] Re: forking, Re: [COFF] Re: On Bloat and the Idea of Small Specialized Tools

[Larry McVoy]

On Sun, May 12, 2024 at 06:56:35PM -0400, Dan Cross wrote:
> Similarly, this gives you the functionality of spawn, without the
> downside of a singularly complicated interface. Could you have
> implemented that in something as small as the PDP-7? Perhaps not, but
> it does not follow that `fork` now remains a good primitive.

Our spawnvp() implmentation is 40 lines of code.  Worked fine everywhere.
I can post it if you like.

[TUHS] Re: forking, Re: [COFF] Re: On Bloat and the Idea of Small Specialized Tools

[Dave Horsfall]

On Sun, 12 May 2024, Larry McVoy wrote:

> Our spawnvp() implmentation is 40 lines of code.  Worked fine everywhere.
> I can post it if you like.

Pretty please...

-- Dave

[TUHS] Re: [COFF] Re: On Bloat and the Idea of Small Specialized Tools

[Alexis]

> On Sat, May 11, 2024 at 2:35 PM Theodore Ts'o <tytso@mit.edu> 
> wrote:
>
> So while some of us old farts might be bemoaning the death of 
> the
> Unix
> philosophy, perhaps part of the reality is that the Unix 
> philosophy
> were ideal for a simpler time, but might not be as good of a fit
> today

Hm .... i guess it might depend on the specific use-case(s) 
involved?

At one point i realised that a primary reason i enjoy using *n*x 
systems is that they're fundamentally 
_text-oriented_. (Unsurprisingly, of course, given the context in 
which Unix was developed.) i spend a lot of my time interacting 
and working with text, and *n*x systems provide me with many 
useful tools for this. Quoting the old "UNIX As Literature" piece, 
https://theody.net/elements.html:

"[T]he most recurrent complaint was that [Unix] was too 
text-oriented. People really hated the command line, with all the 
utilities, obscure flags, and arguments they had to memorize. They 
hated all the typing. One mislaid character and you had to start 
over. Interestingly, this complaint came most often from users of 
the GUI-laden Macintosh or Windows platforms. ...

"[A] suspiciously high proportion of my UNIX colleagues had 
already developed, in some prior career, a comfort and fluency 
with text and printed words. ...

"With UNIX, text — on the command line, STDIN, STDOUT, STDERR — is 
the primary interface mechanism: UNIX system utilities are a sort 
of Lego construction set for word-smiths. Pipes and filters 
connect one utility to the next, text flows invisibly 
between. Working with a shell, awk/lex derivatives, or the utility 
set is literally a word dance."

Perl, with its pervasive regex-based functionality and extensive 
Unicode support, fits neatly into this. i find regexes an 
_incredibly_ powerful tool for working with text, whether via 
Perl, sed, awk, or whatever. But my experience is that many people 
treat regexes as an anathema, with Zawinski's "Now you have two 
problems" regularly trotted out as a thought-terminating 
cliché. Sure, regexes can, and do, get used where they shouldn't 
be[a]; that doesn't mean the baby should be thrown out with the 
bathwater. 

But if one is only working with text under sufferance, trying to 
avoid it via substantially more graphically-oriented environments, 
the text-based "Unix philosophy" and the tools associated with it 
might feel (and actually be) much less appropriate and 
useful. Fair enough. The Unix construction set will still be there 
for those of us who find them very appropriate and tremendously 
useful.


Alexis.

[a] It seems unlikely that anyone on this list hasn't already seen 
this, but just in case:

https://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags/1732454#1732454

i'm looking forward to that comment sending OpenAI over the 
Mountains of Madness.

[TUHS] Re: [COFF] Re: On Bloat and the Idea of Small Specialized Tools

[Warner Losh]

[-- Attachment #1: Type: text/plain, Size: 4737 bytes --]

On Sun, May 12, 2024, 8:34 PM Alexis <flexibeast@gmail.com> wrote:

>
> > On Sat, May 11, 2024 at 2:35 PM Theodore Ts'o <tytso@mit.edu>
> > wrote:
> >
> > So while some of us old farts might be bemoaning the death of
> > the
> > Unix
> > philosophy, perhaps part of the reality is that the Unix
> > philosophy
> > were ideal for a simpler time, but might not be as good of a fit
> > today
>
> Hm .... i guess it might depend on the specific use-case(s)
> involved?
>

I created, years ago, a set of time legos. They were connected as a network
of producer / consumer interfaces. Each lego would do one thing and pass
the results to the next thing in the chain. A driver would read timing data
from the driver and convert it to a MI interface. Different other legos
would take time differences, compute phase or frequency differences and
these would feed into more sophisticated algorithms or output etc. All
locking was on yhe pipe's queues so all these algorithms were lock free
apart from the queueing or dequeueing of data.

Concrptually, this is just a bunch of pipe, with many to 1 or 1 to many
added. Each lego did one thing and passed the results along to the thing in
the chain... much like 'cmd | grep | awk | more'. Plus MI data
representations for almost everything so only the driver reader thread
cared about the hw. See also tty abstraction or ifnet abstraction in
unix....

So actually not a set of FDs passing data between process, but threads
doing the same sort of thing.

The whole data filtering paradigm works in lots of different ways. And it
still works really well by analogy.

Warner

ObComplaint: fork sucks for address spaces with 100s of threads. Forst
thing we created a child process we used to broker different threads
needing to run popen or system... having a create process / munge process /
start process API is kinda what we did behind the scenes though with "send
this data" and "receive that data". We iterated to this after the first
dozen attempts to closely broker fork/exec dance proved... unreliable.

At one point i realised that a primary reason i enjoy using *n*x
> systems is that they're fundamentally
> _text-oriented_. (Unsurprisingly, of course, given the context in
> which Unix was developed.) i spend a lot of my time interacting
> and working with text, and *n*x systems provide me with many
> useful tools for this. Quoting the old "UNIX As Literature" piece,
> https://theody.net/elements.html:
>
> "[T]he most recurrent complaint was that [Unix] was too
> text-oriented. People really hated the command line, with all the
> utilities, obscure flags, and arguments they had to memorize. They
> hated all the typing. One mislaid character and you had to start
> over. Interestingly, this complaint came most often from users of
> the GUI-laden Macintosh or Windows platforms. ...
>
> "[A] suspiciously high proportion of my UNIX colleagues had
> already developed, in some prior career, a comfort and fluency
> with text and printed words. ...
>
> "With UNIX, text — on the command line, STDIN, STDOUT, STDERR — is
> the primary interface mechanism: UNIX system utilities are a sort
> of Lego construction set for word-smiths. Pipes and filters
> connect one utility to the next, text flows invisibly
> between. Working with a shell, awk/lex derivatives, or the utility
> set is literally a word dance."
>
> Perl, with its pervasive regex-based functionality and extensive
> Unicode support, fits neatly into this. i find regexes an
> _incredibly_ powerful tool for working with text, whether via
> Perl, sed, awk, or whatever. But my experience is that many people
> treat regexes as an anathema, with Zawinski's "Now you have two
> problems" regularly trotted out as a thought-terminating
> cliché. Sure, regexes can, and do, get used where they shouldn't
> be[a]; that doesn't mean the baby should be thrown out with the
> bathwater.
>
> But if one is only working with text under sufferance, trying to
> avoid it via substantially more graphically-oriented environments,
> the text-based "Unix philosophy" and the tools associated with it
> might feel (and actually be) much less appropriate and
> useful. Fair enough. The Unix construction set will still be there
> for those of us who find them very appropriate and tremendously
> useful.
>
>
> Alexis.
>
> [a] It seems unlikely that anyone on this list hasn't already seen
> this, but just in case:
>
>
> https://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags/1732454#1732454
>
> i'm looking forward to that comment sending OpenAI over the
> Mountains of Madness.
>

[-- Attachment #2: Type: text/html, Size: 6111 bytes --]

[TUHS] Re: forking, Re: [COFF] Re: On Bloat and the Idea of Small Specialized Tools

[Andrew Warkentin]

On Sun, May 12, 2024 at 4:57 PM Dan Cross <crossd@gmail.com> wrote:
>l.
>
> Perhaps, but as I've written here before, `fork`/`exec` vs `spawn` is
> a false dichotomy. Another alternative is a `proccreate`/`procrun`
> pair, the former of which creates an unrunnable process, the latter of
> which marks it runnable. Coupled with a set of primitives to
> manipulate the state of an extant, but unrunnable, process and you
> have the advantages of fork/exec without the downsides (which are
> well-known; https://www.microsoft.com/en-us/research/uploads/prod/2019/04/fork-hotos19.pdf).
> Similarly, this gives you the functionality of spawn, without the
> downside of a singularly complicated interface. Could you have
> implemented that in something as small as the PDP-7? Perhaps not, but
> it does not follow that `fork` now remains a good primitive.
>
IMO something like that is the best model (although it probably would
have been a bit complicated for a PDP-7/PDP-11). That's basically what
I'm doing in the OS that I'm writing
<https://gitlab.com/uxrt/uxrt-toplevel>. Processes will basically just
be containers for hierarchical groups of threads, and will have pretty
much no other state besides the command line. All of the context
normally associated with a process (file descriptor space,
permissions/UID/GID, filesystem namespace, virtual address space) will
instead be in separate objects that are explicitly bound to threads.
Separate APIs for creating an empty process, creating threads within
it, manipulating context objects and binding threads to them, and
starting the process will be provided (all of these APIs will use a
file-based transport underneath; this will be the first OS I know of
where literally everything is a file). The base process APIs will be
general enough to allow an efficient copy-on-write fork() to be
implemented on top of them for backwards compatibility and the
remaining use cases where forking still makes sense (since even all
process memory will be implemented with files, this will be
implemented with a special in-memory "shadow filesystem" that creates
alternate mappings of other memory filesystems).

Really I'd say there are actually several design decisions in
conventional Unix that made sense on a PDP-7 or PDP-11, but no longer
make sense in the modern world. For instance, the rather inflexible
security model with its fixed set of root-only system calls rather
than some form of role-based access control, or the use of on-disk
device nodes bound by numbers rather than something like separate
special filesystems for each driver that get union mounted together,
or the lack of integrated support for userspace filesystem servers
(yes, there's FUSE, but it's kind of a poorly integrated hack that is
rarely used for anything important).

[TUHS] Re: [COFF] Re: On Bloat and the Idea of Small Specialized Tools

[markus schnalke]

Hoi.

> On Sat, May 11, 2024 at 2:35 PM Theodore Ts'o <tytso@mit.edu> wrote:
> 
>     I bet most of the young'uns would not be trying to do this as a shell
>     script, but using the Cloud SDK with perl or python or Go, which is
>     *way* more bloaty than using /bin/sh.
> 
>     So while some of us old farts might be bemoaning the death of the Unix
>     philosophy, perhaps part of the reality is that the Unix philosophy
>     were ideal for a simpler time, but might not be as good of a fit
>     today

It depends on what the Unix philosophy is seen to be. If it is
solving problems by reading text from standard in and printing to
standard out, then that might not be suitable anymore for many of
today's problems. But if it is prefering plain text to binary,
perfering simple solutions to complex ones, increasing the number
of operations one can perform by combining small generic parts,
... all because of good reasons ... Focussing on simplicity,
clarity, generality ... Omitting needless words! ... All this still
holds true, no matter if applied as shell scripts or within the
design of a new programming language or a programming interface.

It's not so much about the tools we use -- these should be suited
for the times you live in and the problems you have to solve --
but it's more about how you look at them and how you look at the
problems and what ideas for solutions you can imagine in your
mind. Here, Unix provides a continuing inspiration.

Only, like with every old book: when we read it today, we have to
read it within the background of the times back then and transfer
its message to today's times. The older the book, the more transfer
work has to be done, the more knowledgable the then younger and
more distant readers have to be, to really understand it.

Thus, in my oppinion, the Unix philosophy remains a good and very
relevant fit today, although not all of its applications from back
then still are.


meillo

[TUHS] Re: [COFF] Re: On Bloat and the Idea of Small Specialized Tools

[Andrew Warkentin]

On Sun, May 12, 2024 at 11:23 PM markus schnalke <meillo@marmaro.de> wrote:
>
>
> It depends on what the Unix philosophy is seen to be. If it is
> solving problems by reading text from standard in and printing to
> standard out, then that might not be suitable anymore for many of
> today's problems. But if it is prefering plain text to binary,
> perfering simple solutions to complex ones, increasing the number
> of operations one can perform by combining small generic parts,
> ... all because of good reasons ... Focussing on simplicity,
> clarity, generality ... Omitting needless words! ... All this still
> holds true, no matter if applied as shell scripts or within the
> design of a new programming language or a programming interface.
>
> It's not so much about the tools we use -- these should be suited
> for the times you live in and the problems you have to solve --
> but it's more about how you look at them and how you look at the
> problems and what ideas for solutions you can imagine in your
> mind. Here, Unix provides a continuing inspiration.
>
> Only, like with every old book: when we read it today, we have to
> read it within the background of the times back then and transfer
> its message to today's times. The older the book, the more transfer
> work has to be done, the more knowledgable the then younger and
> more distant readers have to be, to really understand it.
>
> Thus, in my oppinion, the Unix philosophy remains a good and very
> relevant fit today, although not all of its applications from back
> then still are.
>
I agree, but it seems that most Unix developers haven't really cared
since the side branches and clones effectively took over from Research
Unix in the early 80s. They've added system calls and ad-hoc socket
RPC interfaces with abandon instead of using generic filesystem-based
extensibility APIs, added options to various commands that should just
have been separate programs, and written desktop
environments/applications that have poor composability, extensibility
and modularity (I guess KDE's KParts kind of counts as a mechanism for
composing applications, but it's limited by being based on plugins
rather than an open IPC-based API). The only Unix desktop I can think
of that really tries to follow the Unix philosophy somewhat is the
now-abandoned Étoilé <http://etoileos.com/etoile/>. There's also the
desktops of the rather obscure BTRON family
<http://tronweb.super-nova.co.jp/btronproducts.html>, although those
OSes are only vaguely Unix-like. Both have an object-centric rather
than application-centric model with support for embedding applications
within each other and controlling them with RPC APIs.

IMO, the best practical realization of the Unix philosophy for the
modern era would be a QNX/Plan 9-like OS with an Étoilé/BTRON-like
desktop, hence why I'm working on one. Some of the specifics of the
original Unix philosophy may not be relevant to large parts of modern
computing, but I'd say the general ideas still are.

[TUHS] Re: forking, Re: [COFF] Re: On Bloat and the Idea of Small Specialized Tools

[Larry McVoy]

On Mon, May 13, 2024 at 11:34:38AM +1000, Dave Horsfall wrote:
> On Sun, 12 May 2024, Larry McVoy wrote:
> 
> > Our spawnvp() implmentation is 40 lines of code.  Worked fine everywhere.
> > I can post it if you like.
> 
> Pretty please...
> 
> -- Dave

/*
 * Copyright 1999-2002,2004-2006,2015-2016 BitMover, Inc
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

#include "system.h"

void	(*spawn_preHook)(int flags, char *av[]) = 0;

#ifndef WIN32
pid_t
bk_spawnvp(int flags, char *cmdname, char *av[])
{
	int	fd, status;
	pid_t	pid;
	char	*exec;

	/* Tell the calling process right away if there is no such program */
	unless (exec = which((char*)cmdname)) return (-1);

	if (spawn_preHook) spawn_preHook(flags, av);
	if (pid = fork()) {	/* parent */
		free(exec);
		if (pid == -1) return (pid);
		unless (flags & (_P_DETACH|_P_NOWAIT)) {
			if (waitpid(pid, &status, 0) != pid) status = -1;
			return (status);
		}
		return (pid);
	} else {		/* child */
		/*
		 * See win32/uwtlib/wapi_intf.c:spawnvp_ex()
		 * We leave nothing open on a detach, but leave
		 * in/out/err open on a normal fork/exec.
		 */
		if (flags & _P_DETACH) {
			unless (getenv("_NO_SETSID")) setsid();
			/* close everything to match winblows */
			for (fd = 0; fd < 100; fd++) (close)(fd);
		} else {
			/*
			 * Emulate having everything except in/out/err
			 * as being marked as close on exec to match winblows.
			 */
			for (fd = 3; fd < 100; fd++) (close)(fd);
		}
		execv(exec, av);
		perror(exec);
		_exit(19);
	}
}

#else /* ======== WIN32 ======== */

pid_t
bk_spawnvp(int flags, char *cmdname, char *av[])
{
	pid_t	pid;
	char	*exec;

	/* Tell the calling process right away if there is no such program */
	unless (exec = which((char*)cmdname)) return (-1);

	if (spawn_preHook) spawn_preHook(flags, av);
	/*
	 * We use our own version of spawn in uwtlib
	 * because the NT spawn() does not work well with tcl
	 */
	pid = _spawnvp_ex(flags, exec, av, 1);
	free(exec);
	return (pid);
}
#endif /* WIN32 */