The Unix Heritage Society mailing list
 help / color / mirror / Atom feed
* [TUHS] Systematic approach to command-line interfaces
@ 2021-08-02  2:42 Douglas McIlroy
  2021-08-02 14:58 ` Theodore Ts'o
  0 siblings, 1 reply; 20+ messages in thread
From: Douglas McIlroy @ 2021-08-02  2:42 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

> spawn() beats fork()[;] fork() should be deprecated

Spawn is a further complication of exec, which tells what signals and
file descriptors to inherit in addition to what arguments and
environment variables to pass.

Fork has a place. For example, Program 1 in
www.cs.dartmouth.edu/~doug/sieve/sieve.pdf forks like crazy and never
execs. To use spawn, the program would have to be split in three (or
be passed a switch setting).

While you may dismiss Program 1 as merely a neat demo, the same idea
applies in parallelizing code for use in a multiprocessor world.

Doug

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [TUHS] Systematic approach to command-line interfaces
  2021-08-02  2:42 [TUHS] Systematic approach to command-line interfaces Douglas McIlroy
@ 2021-08-02 14:58 ` Theodore Ts'o
  2021-08-02 18:15   ` Adam Thornton
  2021-08-03  7:19   ` arnold
  0 siblings, 2 replies; 20+ messages in thread
From: Theodore Ts'o @ 2021-08-02 14:58 UTC (permalink / raw)
  To: Douglas McIlroy; +Cc: The Eunuchs Hysterical Society

On Sun, Aug 01, 2021 at 10:42:53PM -0400, Douglas McIlroy wrote:
> > spawn() beats fork()[;] fork() should be deprecated
> 
> Spawn is a further complication of exec, which tells what signals and
> file descriptors to inherit in addition to what arguments and
> environment variables to pass.
> 
> Fork has a place. For example, Program 1 in
> www.cs.dartmouth.edu/~doug/sieve/sieve.pdf forks like crazy and never
> execs. To use spawn, the program would have to be split in three (or
> be passed a switch setting).
> 
> While you may dismiss Program 1 as merely a neat demo, the same idea
> applies in parallelizing code for use in a multiprocessor world.

It's certainly clear that some kind of primitive is needed to create
new threads.  An open question is whether if there exists some kind of
"new thread" primitve plus either spawn(2) or some kind of "create a
child process and then then frob like crazy using 'echo XXX >
/proc/<pid>/<magic files>'" whether there still is a need for a
fork(2) system call.

Obviously, as soon as we start going down this path, we're deviated
quite strongly from the "radical simplicity" of Unix Version 7 that
people have accused modern systems (whether they be Linux or FreeBSD)
of lacking.  It's rather interesting that we haven't heard complaints
about how people who dare to try come up with new API's are somehow
traitors to "The Unix Philosphy" that we've seen on other threads.  :-)

	    	      		      	    - Ted

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [TUHS] Systematic approach to command-line interfaces
  2021-08-02 14:58 ` Theodore Ts'o
@ 2021-08-02 18:15   ` Adam Thornton
  2021-08-02 18:24     ` Warner Losh
                       ` (2 more replies)
  2021-08-03  7:19   ` arnold
  1 sibling, 3 replies; 20+ messages in thread
From: Adam Thornton @ 2021-08-02 18:15 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 2099 bytes --]

It's a measure of Unix having been wounded by its own success.

fork() is a great model for a single-threaded text processing pipeline to
do automated typesetting.  (More generally, anything that is a
straightforward composition of filter/transform stages.)  Which is, y'know,
what Unix is *for*.

It's not so great for a responsive GUI in front of a multi-function
interactive program.

These days, the vast majority of Unix applications are "stuff people play
with on their phones."

Adam

On Mon, Aug 2, 2021 at 7:59 AM Theodore Ts'o <tytso@mit.edu> wrote:

> On Sun, Aug 01, 2021 at 10:42:53PM -0400, Douglas McIlroy wrote:
> > > spawn() beats fork()[;] fork() should be deprecated
> >
> > Spawn is a further complication of exec, which tells what signals and
> > file descriptors to inherit in addition to what arguments and
> > environment variables to pass.
> >
> > Fork has a place. For example, Program 1 in
> > www.cs.dartmouth.edu/~doug/sieve/sieve.pdf forks like crazy and never
> > execs. To use spawn, the program would have to be split in three (or
> > be passed a switch setting).
> >
> > While you may dismiss Program 1 as merely a neat demo, the same idea
> > applies in parallelizing code for use in a multiprocessor world.
>
> It's certainly clear that some kind of primitive is needed to create
> new threads.  An open question is whether if there exists some kind of
> "new thread" primitve plus either spawn(2) or some kind of "create a
> child process and then then frob like crazy using 'echo XXX >
> /proc/<pid>/<magic files>'" whether there still is a need for a
> fork(2) system call.
>
> Obviously, as soon as we start going down this path, we're deviated
> quite strongly from the "radical simplicity" of Unix Version 7 that
> people have accused modern systems (whether they be Linux or FreeBSD)
> of lacking.  It's rather interesting that we haven't heard complaints
> about how people who dare to try come up with new API's are somehow
> traitors to "The Unix Philosphy" that we've seen on other threads.  :-)
>
>                                             - Ted
>

[-- Attachment #2: Type: text/html, Size: 2860 bytes --]

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [TUHS] Systematic approach to command-line interfaces
  2021-08-02 18:15   ` Adam Thornton
@ 2021-08-02 18:24     ` Warner Losh
  2021-08-02 20:55     ` John Cowan
  2021-08-03  0:21     ` [TUHS] " Bakul Shah
  2 siblings, 0 replies; 20+ messages in thread
From: Warner Losh @ 2021-08-02 18:24 UTC (permalink / raw)
  To: Adam Thornton; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 2792 bytes --]

On Mon, Aug 2, 2021 at 12:16 PM Adam Thornton <athornton@gmail.com> wrote:

> It's a measure of Unix having been wounded by its own success.
>
> fork() is a great model for a single-threaded text processing pipeline to
> do automated typesetting.  (More generally, anything that is a
> straightforward composition of filter/transform stages.)  Which is, y'know,
> what Unix is *for*.
>

fork() dates from a time that demand paging wasn't a thing. Processes were
as cheap as it got. There were no threads.

All the different variations on a theme on fork() since then have been to
either make threads super cheap to create, to optimize the exec case (which
already has been discussed a bit:), and/or to control what the new process
inherits.

It's not so great for a responsive GUI in front of a multi-function
> interactive program.
>
> These days, the vast majority of Unix applications are "stuff people play
> with on their phones."
>

Ah, a thread-heavy environment that's not all that exec intensive (but
that's complicated enough you can no longer safely do a naive fork/exec
when you need to)...

But mostly, it's threads.

Warner


> Adam
>
> On Mon, Aug 2, 2021 at 7:59 AM Theodore Ts'o <tytso@mit.edu> wrote:
>
>> On Sun, Aug 01, 2021 at 10:42:53PM -0400, Douglas McIlroy wrote:
>> > > spawn() beats fork()[;] fork() should be deprecated
>> >
>> > Spawn is a further complication of exec, which tells what signals and
>> > file descriptors to inherit in addition to what arguments and
>> > environment variables to pass.
>> >
>> > Fork has a place. For example, Program 1 in
>> > www.cs.dartmouth.edu/~doug/sieve/sieve.pdf forks like crazy and never
>> > execs. To use spawn, the program would have to be split in three (or
>> > be passed a switch setting).
>> >
>> > While you may dismiss Program 1 as merely a neat demo, the same idea
>> > applies in parallelizing code for use in a multiprocessor world.
>>
>> It's certainly clear that some kind of primitive is needed to create
>> new threads.  An open question is whether if there exists some kind of
>> "new thread" primitve plus either spawn(2) or some kind of "create a
>> child process and then then frob like crazy using 'echo XXX >
>> /proc/<pid>/<magic files>'" whether there still is a need for a
>> fork(2) system call.
>>
>> Obviously, as soon as we start going down this path, we're deviated
>> quite strongly from the "radical simplicity" of Unix Version 7 that
>> people have accused modern systems (whether they be Linux or FreeBSD)
>> of lacking.  It's rather interesting that we haven't heard complaints
>> about how people who dare to try come up with new API's are somehow
>> traitors to "The Unix Philosphy" that we've seen on other threads.  :-)
>>
>>                                             - Ted
>>
>

[-- Attachment #2: Type: text/html, Size: 4253 bytes --]

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [TUHS] Systematic approach to command-line interfaces
  2021-08-02 18:15   ` Adam Thornton
  2021-08-02 18:24     ` Warner Losh
@ 2021-08-02 20:55     ` John Cowan
  2021-08-02 21:06       ` Jon Steinhart
  2021-08-03  0:21     ` [TUHS] " Bakul Shah
  2 siblings, 1 reply; 20+ messages in thread
From: John Cowan @ 2021-08-02 20:55 UTC (permalink / raw)
  To: Adam Thornton; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 950 bytes --]

On Mon, Aug 2, 2021 at 2:16 PM Adam Thornton <athornton@gmail.com> wrote:


> fork() is a great model for a single-threaded text processing pipeline to
> do automated typesetting.  (More generally, anything that is a
> straightforward composition of filter/transform stages.)  Which is, y'know,
> what Unix is *for*.
>

Indeed.  But it's also a very good model for "baking" web pages in the
background so that you can serve them up with a plain dumb web server,
maybe with a bit of JS to provide some auto-updating, especially if the
source data is stored not in a database but in the file system.  The result
is a page that displays (modulo network latency) as fast as you can hit the
Enter key in the address bar.

(The weak point is the lack of dependency management when the system is too
big to rebake all the pages each time.  Perhaps make(1), which Alex Shinn
described as "a beautiful little Prolog for the file system", is the Right
Thing.)

[-- Attachment #2: Type: text/html, Size: 1741 bytes --]

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [TUHS] Systematic approach to command-line interfaces
  2021-08-02 20:55     ` John Cowan
@ 2021-08-02 21:06       ` Jon Steinhart
  2021-08-02 21:25         ` Dan Cross
  0 siblings, 1 reply; 20+ messages in thread
From: Jon Steinhart @ 2021-08-02 21:06 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

John Cowan writes:
>
> > fork() is a great model for a single-threaded text processing pipeline to
> > do automated typesetting.  (More generally, anything that is a
> > straightforward composition of filter/transform stages.)  Which is, y'know,
> > what Unix is *for*.
> >
>
> Indeed.  But it's also a very good model for "baking" web pages in the
> background so that you can serve them up with a plain dumb web server,
> maybe with a bit of JS to provide some auto-updating, especially if the
> source data is stored not in a database but in the file system.  The result
> is a page that displays (modulo network latency) as fast as you can hit the
> Enter key in the address bar.
>
> (The weak point is the lack of dependency management when the system is too
> big to rebake all the pages each time.  Perhaps make(1), which Alex Shinn
> described as "a beautiful little Prolog for the file system", is the Right
> Thing.)

We have, of course, had similar discussions many times on this list.
I think that the root issue is the false equivalence of "I don't
understand this well enough to be able to use it effectively to solve
my problem" with "it's broken/obsolete/dated".


^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [TUHS] Systematic approach to command-line interfaces
  2021-08-02 21:06       ` Jon Steinhart
@ 2021-08-02 21:25         ` Dan Cross
  2021-08-02 21:59           ` Jon Steinhart
  2021-08-03  0:20           ` [TUHS] fork (Re: " Bakul Shah
  0 siblings, 2 replies; 20+ messages in thread
From: Dan Cross @ 2021-08-02 21:25 UTC (permalink / raw)
  To: Jon Steinhart; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 1664 bytes --]

On Mon, Aug 2, 2021 at 5:19 PM Jon Steinhart <jon@fourwinds.com> wrote:

> John Cowan writes:
> > > fork() is a great model for a single-threaded text processing pipeline
> to
> > > do automated typesetting.  (More generally, anything that is a
> > > straightforward composition of filter/transform stages.)  Which is,
> y'know,
> > > what Unix is *for*.
> > >
> >
> > Indeed.  But it's also a very good model for "baking" web pages in the
> > background so that you can serve them up with a plain dumb web server,
> > maybe with a bit of JS to provide some auto-updating, especially if the
> > source data is stored not in a database but in the file system.  The
> result
> > is a page that displays (modulo network latency) as fast as you can hit
> the
> > Enter key in the address bar.
> >
> > (The weak point is the lack of dependency management when the system is
> too
> > big to rebake all the pages each time.  Perhaps make(1), which Alex Shinn
> > described as "a beautiful little Prolog for the file system", is the
> Right
> > Thing.)
>
> We have, of course, had similar discussions many times on this list.
> I think that the root issue is the false equivalence of "I don't
> understand this well enough to be able to use it effectively to solve
> my problem" with "it's broken/obsolete/dated".
>

That's a bit unfair. One can understand something and see value in it and
still appreciate its limitations.

Fork has served us well for more than five decades; I've got no argument
with that. However, should we never question whether it continues to be the
right, or best, abstraction as the environment around it continues to
evolve?

        - Dan C.

[-- Attachment #2: Type: text/html, Size: 2200 bytes --]

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [TUHS] Systematic approach to command-line interfaces
  2021-08-02 21:25         ` Dan Cross
@ 2021-08-02 21:59           ` Jon Steinhart
  2021-08-02 22:33             ` John Cowan
  2021-08-03  0:20           ` [TUHS] fork (Re: " Bakul Shah
  1 sibling, 1 reply; 20+ messages in thread
From: Jon Steinhart @ 2021-08-02 21:59 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

Dan Cross writes:
> That's a bit unfair. One can understand something and see value in it and
> still appreciate its limitations.
>
> Fork has served us well for more than five decades; I've got no argument
> with that. However, should we never question whether it continues to be the
> right, or best, abstraction as the environment around it continues to
> evolve?

Oh, sorry, wasn't meaning to be categorical there.  Main reason that it came
to mind was John's web example; many have said in the past that the UNIX
model couldn't do that until they figured out that it actually could.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [TUHS] Systematic approach to command-line interfaces
  2021-08-02 21:59           ` Jon Steinhart
@ 2021-08-02 22:33             ` John Cowan
  0 siblings, 0 replies; 20+ messages in thread
From: John Cowan @ 2021-08-02 22:33 UTC (permalink / raw)
  To: Jon Steinhart; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 823 bytes --]

On Mon, Aug 2, 2021 at 6:00 PM Jon Steinhart <jon@fourwinds.com> wrote:>


> Oh, sorry, wasn't meaning to be categorical there.  Main reason that it
> came
> to mind was John's web example; many have said in the past that the UNIX
> model couldn't do that until they figured out that it actually could.
>

I sorta lost track of what I was saying there: spawn*() would work fine in
pipelines, since they involve fork-quickly-followed-by-exec.

Doug's nifty sieve example, on the other hand, would not: the Right Thing
there is Go, or else goroutines in C (either libmill or its successor
libdill as you prefer) since the sieve doesn't actually involve any sort of
global state for which processes are relevant.  Granted, the C libraries
have a little bit of x86_64 asm in them (but you are not expected to
understand this).

[-- Attachment #2: Type: text/html, Size: 1690 bytes --]

^ permalink raw reply	[flat|nested] 20+ messages in thread

* [TUHS] fork (Re:  Systematic approach to command-line interfaces
  2021-08-02 21:25         ` Dan Cross
  2021-08-02 21:59           ` Jon Steinhart
@ 2021-08-03  0:20           ` Bakul Shah
  2021-08-03  0:59             ` Adam Thornton
  1 sibling, 1 reply; 20+ messages in thread
From: Bakul Shah @ 2021-08-03  0:20 UTC (permalink / raw)
  To: Dan Cross; +Cc: The Eunuchs Hysterical Society

On Aug 2, 2021, at 2:25 PM, Dan Cross <crossd@gmail.com> wrote:
> 
> Fork has served us well for more than five decades; I've got no argument with that. However, should we never question whether it continues to be the right, or best, abstraction as the environment around it continues to evolve?

An os with no fork() can be used to reimplement fork() for backward 
compatibility. You will anyway need all the things fork() used to do
in this brave new world. For example, you will need to specify which
signals and file descriptors to inherit. Something like:

jmp_buf state;
pid = 0;
if (!setjmp(state) && !pid) {
	fd = openAt(hostfd, "/n/address-space/new", ...);
	<<copy /proc/self/mem to fd.>>
	pid = breathe_life(fd, state, signals, fds);
	longjmp(state, 1);
	return pid;
}

The /n/address-space filesystem yields a new address space on a
given host (as represented by the hostfd). You can then "copy"
to it to fill it up from the current proc's memory. Copy should
be done using COW if the new proc is on the same host (or even
remote but that opens up another area of discussion...).

breathe_life is given where everything is set up and it just has to
create a new thread, associated with the address, wire up signals
as file descriptors that are to be inherited and arrange things
so that on return from syscall in the child, process state will be
as per "state".

If signals and fds can be wired up to a remote host, this can be
used to "migrate" a process. There is likely much more process
state in the kernel mode which would have to be packaged up somehow.

There may be other breakage if the child & parent are on different
hosts.


^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [TUHS] Systematic approach to command-line interfaces
  2021-08-02 18:15   ` Adam Thornton
  2021-08-02 18:24     ` Warner Losh
  2021-08-02 20:55     ` John Cowan
@ 2021-08-03  0:21     ` Bakul Shah
  2021-08-03  1:49       ` Jon Steinhart
  2 siblings, 1 reply; 20+ messages in thread
From: Bakul Shah @ 2021-08-03  0:21 UTC (permalink / raw)
  To: Adam Thornton; +Cc: The Eunuchs Hysterical Society

Or perhaps the issue is not having graphics/GUI designers with the
creativity and sensibilities of the early Bell Labs crowd of researchers?
I keep thinking there ought to be something simpler/more elegant than
the current graphics subsystems....

> On Aug 2, 2021, at 11:16 AM, Adam Thornton <athornton@gmail.com> wrote:
> 
> 
> It's a measure of Unix having been wounded by its own success.
> 
> fork() is a great model for a single-threaded text processing pipeline to do automated typesetting.  (More generally, anything that is a straightforward composition of filter/transform stages.)  Which is, y'know, what Unix is *for*.
> 
> It's not so great for a responsive GUI in front of a multi-function interactive program.
> 
> These days, the vast majority of Unix applications are "stuff people play with on their phones."
> 
> Adam
> 
> On Mon, Aug 2, 2021 at 7:59 AM Theodore Ts'o <tytso@mit.edu> wrote:
> On Sun, Aug 01, 2021 at 10:42:53PM -0400, Douglas McIlroy wrote:
> > > spawn() beats fork()[;] fork() should be deprecated
> > 
> > Spawn is a further complication of exec, which tells what signals and
> > file descriptors to inherit in addition to what arguments and
> > environment variables to pass.
> > 
> > Fork has a place. For example, Program 1 in
> > www.cs.dartmouth.edu/~doug/sieve/sieve.pdf forks like crazy and never
> > execs. To use spawn, the program would have to be split in three (or
> > be passed a switch setting).
> > 
> > While you may dismiss Program 1 as merely a neat demo, the same idea
> > applies in parallelizing code for use in a multiprocessor world.
> 
> It's certainly clear that some kind of primitive is needed to create
> new threads.  An open question is whether if there exists some kind of
> "new thread" primitve plus either spawn(2) or some kind of "create a
> child process and then then frob like crazy using 'echo XXX >
> /proc/<pid>/<magic files>'" whether there still is a need for a
> fork(2) system call.
> 
> Obviously, as soon as we start going down this path, we're deviated
> quite strongly from the "radical simplicity" of Unix Version 7 that
> people have accused modern systems (whether they be Linux or FreeBSD)
> of lacking.  It's rather interesting that we haven't heard complaints
> about how people who dare to try come up with new API's are somehow
> traitors to "The Unix Philosphy" that we've seen on other threads.  :-)
> 
>                                             - Ted

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [TUHS] fork (Re: Systematic approach to command-line interfaces
  2021-08-03  0:20           ` [TUHS] fork (Re: " Bakul Shah
@ 2021-08-03  0:59             ` Adam Thornton
  2021-08-03  1:20               ` Steve Nickolas
  0 siblings, 1 reply; 20+ messages in thread
From: Adam Thornton @ 2021-08-03  0:59 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society



> On Aug 2, 2021, at 5:20 PM, Bakul Shah <bakul@iitbombay.org> wrote:
> 
> On Aug 2, 2021, at 2:25 PM, Dan Cross <crossd@gmail.com> wrote:
>> 
>> Fork has served us well for more than five decades; I've got no argument with that. However, should we never question whether it continues to be the right, or best, abstraction as the environment around it continues to evolve?
> 
> An os with no fork() can be used to reimplement fork() for backward 
> compatibility. You will anyway need all the things fork() used to do
> in this brave new world.

Oh man, now I’m having flashbacks to early Cygwin and what was that, DJGPP for OS/2?

Well, I picked a lousy week to give up sniffing glue anyway.

Adam

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [TUHS] fork (Re: Systematic approach to command-line interfaces
  2021-08-03  0:59             ` Adam Thornton
@ 2021-08-03  1:20               ` Steve Nickolas
  0 siblings, 0 replies; 20+ messages in thread
From: Steve Nickolas @ 2021-08-03  1:20 UTC (permalink / raw)
  To: Adam Thornton; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 773 bytes --]

On Mon, 2 Aug 2021, Adam Thornton wrote:

>
>
>> On Aug 2, 2021, at 5:20 PM, Bakul Shah <bakul@iitbombay.org> wrote:
>>
>> On Aug 2, 2021, at 2:25 PM, Dan Cross <crossd@gmail.com> wrote:
>>>
>>> Fork has served us well for more than five decades; I've got no argument with that. However, should we never question whether it continues to be the right, or best, abstraction as the environment around it continues to evolve?
>>
>> An os with no fork() can be used to reimplement fork() for backward
>> compatibility. You will anyway need all the things fork() used to do
>> in this brave new world.
>
> Oh man, now I’m having flashbacks to early Cygwin and what was that, DJGPP for OS/2?
>
> Well, I picked a lousy week to give up sniffing glue anyway.
>
> Adam

EMX.

-uso.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [TUHS] Systematic approach to command-line interfaces
  2021-08-03  0:21     ` [TUHS] " Bakul Shah
@ 2021-08-03  1:49       ` Jon Steinhart
  2021-08-03  3:21         ` Adam Thornton
  2021-08-03  3:27         ` Bakul Shah
  0 siblings, 2 replies; 20+ messages in thread
From: Jon Steinhart @ 2021-08-03  1:49 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

Bakul Shah writes:
> Or perhaps the issue is not having graphics/GUI designers with the
> creativity and sensibilities of the early Bell Labs crowd of researchers?
> I keep thinking there ought to be something simpler/more elegant than
> the current graphics subsystems....
>
> > On Aug 2, 2021, at 11:16 AM, Adam Thornton <athornton@gmail.com> wrote:
> > 
> > It's a measure of Unix having been wounded by its own success.
> > 
> > fork() is a great model for a single-threaded text processing pipeline to
> do automated typesetting.  (More generally, anything that is a straightforward
> composition of filter/transform stages.)  Which is, y'know, what Unix is *for*.
> > 
> > It's not so great for a responsive GUI in front of a multi-function interactive program.
> > 
> > These days, the vast majority of Unix applications are "stuff people play with on their phones."
> > 
> > Adam

I thought that I posted something about this recently when someone
was arguing for threads being unnecessary and bad.

My two cents is that GUIs were the big driver for threads.  I would
personally like to get rid of them as they're "the same but different"
with regards to processes.  My preference would be to solve the
heaviness of processes problem.  I'm not in the thick of that these
days, but I don't see it being solved in software alone; it's going
to take some serious hardware/software architecture work.  Might be
easier to accomplish now that the world has pretty much settled on
the process model.

Jon

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [TUHS] Systematic approach to command-line interfaces
  2021-08-03  1:49       ` Jon Steinhart
@ 2021-08-03  3:21         ` Adam Thornton
  2021-08-03  3:27         ` Bakul Shah
  1 sibling, 0 replies; 20+ messages in thread
From: Adam Thornton @ 2021-08-03  3:21 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society



> On Aug 2, 2021, at 6:49 PM, Jon Steinhart <jon@fourwinds.com> wrote:
> 
> My two cents is that GUIs were the big driver for threads.  I would
> personally like to get rid of them as they're "the same but different"
> with regards to processes.  My preference would be to solve the
> heaviness of processes problem.  I'm not in the thick of that these
> days, but I don't see it being solved in software alone; it's going
> to take some serious hardware/software architecture work.  Might be
> easier to accomplish now that the world has pretty much settled on
> the process model.


Has it, though?  Most of the stuff I’m working with right now is either asyncio and Python, or Javascript, both of which are very much about threading.

I think there’s a lot of stuff that does work better with multiple program counters for different things executing at the same time with shared access to memory, although of course reasoning about concurrency is always hard and having to manage locks on things that shouldn’t be shared _right now_ is a lot of work and easy to get wrong.

I like Go and goroutines and channels as an intra-process communication mechanism.

My problem with processes, in the traditional sense, is that there ARE often pieces of state you want to share between the things concurrently acting on that state.  A channel-like mechanism could work as IPC, but, equally well, why _not_ have multiple things—we can call them threads if you want—that can just see that shared state?

Adam

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [TUHS] Systematic approach to command-line interfaces
  2021-08-03  1:49       ` Jon Steinhart
  2021-08-03  3:21         ` Adam Thornton
@ 2021-08-03  3:27         ` Bakul Shah
  2021-08-03  3:51           ` Jon Steinhart
  1 sibling, 1 reply; 20+ messages in thread
From: Bakul Shah @ 2021-08-03  3:27 UTC (permalink / raw)
  To: Jon Steinhart; +Cc: The Eunuchs Hysterical Society

On Aug 2, 2021, at 6:49 PM, Jon Steinhart <jon@fourwinds.com> wrote:
> 
> Bakul Shah writes:
>> Or perhaps the issue is not having graphics/GUI designers with the
>> creativity and sensibilities of the early Bell Labs crowd of researchers?
>> I keep thinking there ought to be something simpler/more elegant than
>> the current graphics subsystems....
>> 
>>> On Aug 2, 2021, at 11:16 AM, Adam Thornton <athornton@gmail.com> wrote:
>>> 
>>> It's a measure of Unix having been wounded by its own success.
>>> 
>>> fork() is a great model for a single-threaded text processing pipeline to
>> do automated typesetting.  (More generally, anything that is a straightforward
>> composition of filter/transform stages.)  Which is, y'know, what Unix is *for*.
>>> 
>>> It's not so great for a responsive GUI in front of a multi-function interactive program.
>>> 
>>> These days, the vast majority of Unix applications are "stuff people play with on their phones."
>>> 
>>> Adam
> 
> I thought that I posted something about this recently when someone
> was arguing for threads being unnecessary and bad.
> 
> My two cents is that GUIs were the big driver for threads.  I would
> personally like to get rid of them as they're "the same but different"
> with regards to processes.  My preference would be to solve the
> heaviness of processes problem.  I'm not in the thick of that these
> days, but I don't see it being solved in software alone; it's going
> to take some serious hardware/software architecture work.  Might be
> easier to accomplish now that the world has pretty much settled on
> the process model.
> 
> Jon

AFAIK pretty much all GUI frameworks are (or were) single threaded.
At least I don't see the GUI as the main motivation for threads.

I was mainly complaining about the complexity of graphics subsystem,
including user interaction. Don't see what fork() has to do with it.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [TUHS] Systematic approach to command-line interfaces
  2021-08-03  3:27         ` Bakul Shah
@ 2021-08-03  3:51           ` Jon Steinhart
  0 siblings, 0 replies; 20+ messages in thread
From: Jon Steinhart @ 2021-08-03  3:51 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

Bakul Shah writes:
> On Aug 2, 2021, at 6:49 PM, Jon Steinhart <jon@fourwinds.com> wrote:
> > 
> > Bakul Shah writes:
> >> Or perhaps the issue is not having graphics/GUI designers with the
> >> creativity and sensibilities of the early Bell Labs crowd of researchers?
> >> I keep thinking there ought to be something simpler/more elegant than
> >> the current graphics subsystems....
> >> 
> >>> On Aug 2, 2021, at 11:16 AM, Adam Thornton <athornton@gmail.com> wrote:
> >>> 
> >>> It's a measure of Unix having been wounded by its own success.
> >>> 
> >>> fork() is a great model for a single-threaded text processing pipeline to
> >> do automated typesetting.  (More generally, anything that is a straightforward
> >> composition of filter/transform stages.)  Which is, y'know, what Unix is *for*.
> >>> 
> >>> It's not so great for a responsive GUI in front of a multi-function interactive program.
> >>> 
> >>> These days, the vast majority of Unix applications are "stuff people play with on their phones."
> >>> 
> >>> Adam
> > 
> > I thought that I posted something about this recently when someone
> > was arguing for threads being unnecessary and bad.
> > 
> > My two cents is that GUIs were the big driver for threads.  I would
> > personally like to get rid of them as they're "the same but different"
> > with regards to processes.  My preference would be to solve the
> > heaviness of processes problem.  I'm not in the thick of that these
> > days, but I don't see it being solved in software alone; it's going
> > to take some serious hardware/software architecture work.  Might be
> > easier to accomplish now that the world has pretty much settled on
> > the process model.
> > 
> > Jon
>
> AFAIK pretty much all GUI frameworks are (or were) single threaded.
> At least I don't see the GUI as the main motivation for threads.
>
> I was mainly complaining about the complexity of graphics subsystem,
> including user interaction. Don't see what fork() has to do with it.

Not my experience.  At least from what I was working on the 1980s
there was a need to be able to keep complicated state in graphical
programs which is when assembly language versions of threads came
into being.  Made it possible for whatever was generating output
to be interrupted so that input could be handled in a timely manner.
I seem to recall getting some code from Sun for this that I ported
to a different system.

Jon

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [TUHS] Systematic approach to command-line interfaces
  2021-08-02 14:58 ` Theodore Ts'o
  2021-08-02 18:15   ` Adam Thornton
@ 2021-08-03  7:19   ` arnold
  2021-08-03 23:12     ` Andrew Warkentin
  1 sibling, 1 reply; 20+ messages in thread
From: arnold @ 2021-08-03  7:19 UTC (permalink / raw)
  To: tytso, douglas.mcilroy; +Cc: tuhs

"Theodore Ts'o" <tytso@mit.edu> wrote:

> It's certainly clear that some kind of primitive is needed to create
> new threads.  An open question is whether if there exists some kind of
> "new thread" primitve plus either spawn(2) or some kind of "create a
> child process and then then frob like crazy using 'echo XXX >
> /proc/<pid>/<magic files>'" whether there still is a need for a
> fork(2) system call.

I haven't caught up yet in this thread. Apologies if this has been
discussed already.

The Plan 9 folks blazed this trail over 30 years ago with rfork, where
you specify what bits you wish to duplicate.  I don't remember details
anymore, but I think it was pretty elegant. IIRC Around that time Rob Pike
said "Threads are the lack of an idea", meaning, if you think you need
threads, you haven't thought about the problem hard enough. (Apologies
to Rob if I am misremembering and/or misrepresenting.)

Arnold

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [TUHS] Systematic approach to command-line interfaces
  2021-08-03  7:19   ` arnold
@ 2021-08-03 23:12     ` Andrew Warkentin
  2021-08-04 15:04       ` Paul Winalski
  0 siblings, 1 reply; 20+ messages in thread
From: Andrew Warkentin @ 2021-08-03 23:12 UTC (permalink / raw)
  To: tuhs

On 8/3/21, arnold@skeeve.com <arnold@skeeve.com> wrote:
>
> I haven't caught up yet in this thread. Apologies if this has been
> discussed already.
>
> The Plan 9 folks blazed this trail over 30 years ago with rfork, where
> you specify what bits you wish to duplicate.  I don't remember details
> anymore, but I think it was pretty elegant. IIRC Around that time Rob Pike
> said "Threads are the lack of an idea", meaning, if you think you need
> threads, you haven't thought about the problem hard enough. (Apologies
> to Rob if I am misremembering and/or misrepresenting.)
>
I've never really been a fan of the rfork()/clone() model, or at least
the Linux implementation of it that requires ugly library-level hacks
to share state between threads that the kernel doesn't support
sharing. Also, I don't really care for the large number of flags
required.

Up until now I was just planning on following the traditional
threading model of associating most state with processes with only
execution state being per-thread in the OS I'm working on, but now I'm
thinking I should reduce the state associated with a process to just
the PID, PPID, PGID, containing cgroup, command line, and list of
threads. All other state would be contained in various types of
context objects that are not tied to a particular process or thread
(other than being destroyed when no more threads are associated with
them). This would include:

Filesystem namespace
File descriptors
Address space
Security context (file permission list, UID, GID)
Signal handlers
Scheduling context

Each of these object types would be completely separate from the
others, allowing full control over which state is shared and which is
private. I'm using seL4 as a microkernel, and it already works like
this (it has no real concept of processes, only threads that are each
associated with an address space, a capability space, and a scheduling
context) so it's a good match for it.

exec() would still replace all threads within a process as on
traditional Unix, unless the exec is performed within a child process
that hasn't yet been started. Sending a signal to an entire process
would send it to every signal group within the process (similarly, it
would be possible to send a signal to an entire cgroup; basically,
processes will really just be a special kind of cgroup in this model).

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [TUHS] Systematic approach to command-line interfaces
  2021-08-03 23:12     ` Andrew Warkentin
@ 2021-08-04 15:04       ` Paul Winalski
  0 siblings, 0 replies; 20+ messages in thread
From: Paul Winalski @ 2021-08-04 15:04 UTC (permalink / raw)
  To: Andrew Warkentin; +Cc: tuhs

On 8/3/21, Andrew Warkentin <andreww591@gmail.com> wrote:
>
> Up until now I was just planning on following the traditional
> threading model of associating most state with processes with only
> execution state being per-thread in the OS I'm working on, but now I'm
> thinking I should reduce the state associated with a process to just
> the PID, PPID, PGID, containing cgroup, command line, and list of
> threads. All other state would be contained in various types of
> context objects that are not tied to a particular process or thread
> (other than being destroyed when no more threads are associated with
> them).

For what it's worth, this is the process model that Windows NT has.

This thread has discussed two very different ways to do command line
processing:  the TOPS-20 model, where the program image for the
command is activated immediately and then calls back to the OS to
process the command line elements, and the UNIX model, where the first
element of the command line is a path (full or partial) to the program
image for the command, and it's entirely up to that image to process
the command line elements as it sees fit.

VMS takes a third approach.  The VAX had four, hierarchical execution
modes:  user, supervisor, exec, and kernel, in increasing order of
privilege.  The VAX had "change mode" instructions (CHMU, CHMS, CHME,
CHMK) that ran the current process's change-mode interrupt handler to
process the request.  Applications and programs to process commands
ran in user mode.  The DCL command interpreter (VMS's equivalent of
the UNIX shell) ran in supervisor mode.  Exec mode was reserved for
record management services (RMS), VMS's equivalent of the file system
code in UNIX.  Kernel mode (the only mode where privileged
instructions can be executed) was, of course, for the kernel.  One
oddity of VMS is that each process had the supervisor, exec, and
kernel code and data mapped to its address space.

Commands in VMS are described using a meta-language that is compiled
into a set of data tables that the command line interpreter (CLI,
running in supervisor mode) uses to determine which program image
needs to be activated to execute the command.  The CLI also does a
full parse of the command line.  It then calls the kernel's image
activator to map the appropriate program image and switches to user
mode to start it running.  There is a runtime library of routines to
retrieve the various options and parameters from the command line (the
routines in this library do CHMS calls to achieve this).

So on VMS we have a situation where the command interpreter (shell) is
part of every process.  Processes are created by the "create process"
OS library routine, sys$creprc, which is a wrapper around the
appropriate CHMK instruction.  Unlike fork() on UNIX, the new process
inherits very little state from its parent.  Most significantly it
inherits neither address space nor open file context.  VMS process
creation is also notoriously expensive compared to fork() on UNIX.

-Paul W.

^ permalink raw reply	[flat|nested] 20+ messages in thread

end of thread, other threads:[~2021-08-04 15:05 UTC | newest]

Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-02  2:42 [TUHS] Systematic approach to command-line interfaces Douglas McIlroy
2021-08-02 14:58 ` Theodore Ts'o
2021-08-02 18:15   ` Adam Thornton
2021-08-02 18:24     ` Warner Losh
2021-08-02 20:55     ` John Cowan
2021-08-02 21:06       ` Jon Steinhart
2021-08-02 21:25         ` Dan Cross
2021-08-02 21:59           ` Jon Steinhart
2021-08-02 22:33             ` John Cowan
2021-08-03  0:20           ` [TUHS] fork (Re: " Bakul Shah
2021-08-03  0:59             ` Adam Thornton
2021-08-03  1:20               ` Steve Nickolas
2021-08-03  0:21     ` [TUHS] " Bakul Shah
2021-08-03  1:49       ` Jon Steinhart
2021-08-03  3:21         ` Adam Thornton
2021-08-03  3:27         ` Bakul Shah
2021-08-03  3:51           ` Jon Steinhart
2021-08-03  7:19   ` arnold
2021-08-03 23:12     ` Andrew Warkentin
2021-08-04 15:04       ` Paul Winalski

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).