* [TUHS] Systematic approach to command-line interfaces
@ 2021-08-02 2:42 Douglas McIlroy
2021-08-02 14:58 ` Theodore Ts'o
0 siblings, 1 reply; 20+ messages in thread
From: Douglas McIlroy @ 2021-08-02 2:42 UTC (permalink / raw)
To: The Eunuchs Hysterical Society
> spawn() beats fork()[;] fork() should be deprecated
Spawn is a further complication of exec, which tells what signals and
file descriptors to inherit in addition to what arguments and
environment variables to pass.
Fork has a place. For example, Program 1 in
www.cs.dartmouth.edu/~doug/sieve/sieve.pdf forks like crazy and never
execs. To use spawn, the program would have to be split in three (or
be passed a switch setting).
While you may dismiss Program 1 as merely a neat demo, the same idea
applies in parallelizing code for use in a multiprocessor world.
Doug
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [TUHS] Systematic approach to command-line interfaces 2021-08-02 2:42 [TUHS] Systematic approach to command-line interfaces Douglas McIlroy @ 2021-08-02 14:58 ` Theodore Ts'o 2021-08-02 18:15 ` Adam Thornton 2021-08-03 7:19 ` arnold 0 siblings, 2 replies; 20+ messages in thread From: Theodore Ts'o @ 2021-08-02 14:58 UTC (permalink / raw) To: Douglas McIlroy; +Cc: The Eunuchs Hysterical Society On Sun, Aug 01, 2021 at 10:42:53PM -0400, Douglas McIlroy wrote: > > spawn() beats fork()[;] fork() should be deprecated > > Spawn is a further complication of exec, which tells what signals and > file descriptors to inherit in addition to what arguments and > environment variables to pass. > > Fork has a place. For example, Program 1 in > www.cs.dartmouth.edu/~doug/sieve/sieve.pdf forks like crazy and never > execs. To use spawn, the program would have to be split in three (or > be passed a switch setting). > > While you may dismiss Program 1 as merely a neat demo, the same idea > applies in parallelizing code for use in a multiprocessor world. It's certainly clear that some kind of primitive is needed to create new threads. An open question is whether if there exists some kind of "new thread" primitve plus either spawn(2) or some kind of "create a child process and then then frob like crazy using 'echo XXX > /proc/<pid>/<magic files>'" whether there still is a need for a fork(2) system call. Obviously, as soon as we start going down this path, we're deviated quite strongly from the "radical simplicity" of Unix Version 7 that people have accused modern systems (whether they be Linux or FreeBSD) of lacking. It's rather interesting that we haven't heard complaints about how people who dare to try come up with new API's are somehow traitors to "The Unix Philosphy" that we've seen on other threads. :-) - Ted ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [TUHS] Systematic approach to command-line interfaces 2021-08-02 14:58 ` Theodore Ts'o @ 2021-08-02 18:15 ` Adam Thornton 2021-08-02 18:24 ` Warner Losh ` (2 more replies) 2021-08-03 7:19 ` arnold 1 sibling, 3 replies; 20+ messages in thread From: Adam Thornton @ 2021-08-02 18:15 UTC (permalink / raw) To: The Eunuchs Hysterical Society [-- Attachment #1: Type: text/plain, Size: 2099 bytes --] It's a measure of Unix having been wounded by its own success. fork() is a great model for a single-threaded text processing pipeline to do automated typesetting. (More generally, anything that is a straightforward composition of filter/transform stages.) Which is, y'know, what Unix is *for*. It's not so great for a responsive GUI in front of a multi-function interactive program. These days, the vast majority of Unix applications are "stuff people play with on their phones." Adam On Mon, Aug 2, 2021 at 7:59 AM Theodore Ts'o <tytso@mit.edu> wrote: > On Sun, Aug 01, 2021 at 10:42:53PM -0400, Douglas McIlroy wrote: > > > spawn() beats fork()[;] fork() should be deprecated > > > > Spawn is a further complication of exec, which tells what signals and > > file descriptors to inherit in addition to what arguments and > > environment variables to pass. > > > > Fork has a place. For example, Program 1 in > > www.cs.dartmouth.edu/~doug/sieve/sieve.pdf forks like crazy and never > > execs. To use spawn, the program would have to be split in three (or > > be passed a switch setting). > > > > While you may dismiss Program 1 as merely a neat demo, the same idea > > applies in parallelizing code for use in a multiprocessor world. > > It's certainly clear that some kind of primitive is needed to create > new threads. An open question is whether if there exists some kind of > "new thread" primitve plus either spawn(2) or some kind of "create a > child process and then then frob like crazy using 'echo XXX > > /proc/<pid>/<magic files>'" whether there still is a need for a > fork(2) system call. > > Obviously, as soon as we start going down this path, we're deviated > quite strongly from the "radical simplicity" of Unix Version 7 that > people have accused modern systems (whether they be Linux or FreeBSD) > of lacking. It's rather interesting that we haven't heard complaints > about how people who dare to try come up with new API's are somehow > traitors to "The Unix Philosphy" that we've seen on other threads. :-) > > - Ted > [-- Attachment #2: Type: text/html, Size: 2860 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [TUHS] Systematic approach to command-line interfaces 2021-08-02 18:15 ` Adam Thornton @ 2021-08-02 18:24 ` Warner Losh 2021-08-02 20:55 ` John Cowan 2021-08-03 0:21 ` [TUHS] " Bakul Shah 2 siblings, 0 replies; 20+ messages in thread From: Warner Losh @ 2021-08-02 18:24 UTC (permalink / raw) To: Adam Thornton; +Cc: The Eunuchs Hysterical Society [-- Attachment #1: Type: text/plain, Size: 2792 bytes --] On Mon, Aug 2, 2021 at 12:16 PM Adam Thornton <athornton@gmail.com> wrote: > It's a measure of Unix having been wounded by its own success. > > fork() is a great model for a single-threaded text processing pipeline to > do automated typesetting. (More generally, anything that is a > straightforward composition of filter/transform stages.) Which is, y'know, > what Unix is *for*. > fork() dates from a time that demand paging wasn't a thing. Processes were as cheap as it got. There were no threads. All the different variations on a theme on fork() since then have been to either make threads super cheap to create, to optimize the exec case (which already has been discussed a bit:), and/or to control what the new process inherits. It's not so great for a responsive GUI in front of a multi-function > interactive program. > > These days, the vast majority of Unix applications are "stuff people play > with on their phones." > Ah, a thread-heavy environment that's not all that exec intensive (but that's complicated enough you can no longer safely do a naive fork/exec when you need to)... But mostly, it's threads. Warner > Adam > > On Mon, Aug 2, 2021 at 7:59 AM Theodore Ts'o <tytso@mit.edu> wrote: > >> On Sun, Aug 01, 2021 at 10:42:53PM -0400, Douglas McIlroy wrote: >> > > spawn() beats fork()[;] fork() should be deprecated >> > >> > Spawn is a further complication of exec, which tells what signals and >> > file descriptors to inherit in addition to what arguments and >> > environment variables to pass. >> > >> > Fork has a place. For example, Program 1 in >> > www.cs.dartmouth.edu/~doug/sieve/sieve.pdf forks like crazy and never >> > execs. To use spawn, the program would have to be split in three (or >> > be passed a switch setting). >> > >> > While you may dismiss Program 1 as merely a neat demo, the same idea >> > applies in parallelizing code for use in a multiprocessor world. >> >> It's certainly clear that some kind of primitive is needed to create >> new threads. An open question is whether if there exists some kind of >> "new thread" primitve plus either spawn(2) or some kind of "create a >> child process and then then frob like crazy using 'echo XXX > >> /proc/<pid>/<magic files>'" whether there still is a need for a >> fork(2) system call. >> >> Obviously, as soon as we start going down this path, we're deviated >> quite strongly from the "radical simplicity" of Unix Version 7 that >> people have accused modern systems (whether they be Linux or FreeBSD) >> of lacking. It's rather interesting that we haven't heard complaints >> about how people who dare to try come up with new API's are somehow >> traitors to "The Unix Philosphy" that we've seen on other threads. :-) >> >> - Ted >> > [-- Attachment #2: Type: text/html, Size: 4253 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [TUHS] Systematic approach to command-line interfaces 2021-08-02 18:15 ` Adam Thornton 2021-08-02 18:24 ` Warner Losh @ 2021-08-02 20:55 ` John Cowan 2021-08-02 21:06 ` Jon Steinhart 2021-08-03 0:21 ` [TUHS] " Bakul Shah 2 siblings, 1 reply; 20+ messages in thread From: John Cowan @ 2021-08-02 20:55 UTC (permalink / raw) To: Adam Thornton; +Cc: The Eunuchs Hysterical Society [-- Attachment #1: Type: text/plain, Size: 950 bytes --] On Mon, Aug 2, 2021 at 2:16 PM Adam Thornton <athornton@gmail.com> wrote: > fork() is a great model for a single-threaded text processing pipeline to > do automated typesetting. (More generally, anything that is a > straightforward composition of filter/transform stages.) Which is, y'know, > what Unix is *for*. > Indeed. But it's also a very good model for "baking" web pages in the background so that you can serve them up with a plain dumb web server, maybe with a bit of JS to provide some auto-updating, especially if the source data is stored not in a database but in the file system. The result is a page that displays (modulo network latency) as fast as you can hit the Enter key in the address bar. (The weak point is the lack of dependency management when the system is too big to rebake all the pages each time. Perhaps make(1), which Alex Shinn described as "a beautiful little Prolog for the file system", is the Right Thing.) [-- Attachment #2: Type: text/html, Size: 1741 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [TUHS] Systematic approach to command-line interfaces 2021-08-02 20:55 ` John Cowan @ 2021-08-02 21:06 ` Jon Steinhart 2021-08-02 21:25 ` Dan Cross 0 siblings, 1 reply; 20+ messages in thread From: Jon Steinhart @ 2021-08-02 21:06 UTC (permalink / raw) To: The Eunuchs Hysterical Society John Cowan writes: > > > fork() is a great model for a single-threaded text processing pipeline to > > do automated typesetting. (More generally, anything that is a > > straightforward composition of filter/transform stages.) Which is, y'know, > > what Unix is *for*. > > > > Indeed. But it's also a very good model for "baking" web pages in the > background so that you can serve them up with a plain dumb web server, > maybe with a bit of JS to provide some auto-updating, especially if the > source data is stored not in a database but in the file system. The result > is a page that displays (modulo network latency) as fast as you can hit the > Enter key in the address bar. > > (The weak point is the lack of dependency management when the system is too > big to rebake all the pages each time. Perhaps make(1), which Alex Shinn > described as "a beautiful little Prolog for the file system", is the Right > Thing.) We have, of course, had similar discussions many times on this list. I think that the root issue is the false equivalence of "I don't understand this well enough to be able to use it effectively to solve my problem" with "it's broken/obsolete/dated". ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [TUHS] Systematic approach to command-line interfaces 2021-08-02 21:06 ` Jon Steinhart @ 2021-08-02 21:25 ` Dan Cross 2021-08-02 21:59 ` Jon Steinhart 2021-08-03 0:20 ` [TUHS] fork (Re: " Bakul Shah 0 siblings, 2 replies; 20+ messages in thread From: Dan Cross @ 2021-08-02 21:25 UTC (permalink / raw) To: Jon Steinhart; +Cc: The Eunuchs Hysterical Society [-- Attachment #1: Type: text/plain, Size: 1664 bytes --] On Mon, Aug 2, 2021 at 5:19 PM Jon Steinhart <jon@fourwinds.com> wrote: > John Cowan writes: > > > fork() is a great model for a single-threaded text processing pipeline > to > > > do automated typesetting. (More generally, anything that is a > > > straightforward composition of filter/transform stages.) Which is, > y'know, > > > what Unix is *for*. > > > > > > > Indeed. But it's also a very good model for "baking" web pages in the > > background so that you can serve them up with a plain dumb web server, > > maybe with a bit of JS to provide some auto-updating, especially if the > > source data is stored not in a database but in the file system. The > result > > is a page that displays (modulo network latency) as fast as you can hit > the > > Enter key in the address bar. > > > > (The weak point is the lack of dependency management when the system is > too > > big to rebake all the pages each time. Perhaps make(1), which Alex Shinn > > described as "a beautiful little Prolog for the file system", is the > Right > > Thing.) > > We have, of course, had similar discussions many times on this list. > I think that the root issue is the false equivalence of "I don't > understand this well enough to be able to use it effectively to solve > my problem" with "it's broken/obsolete/dated". > That's a bit unfair. One can understand something and see value in it and still appreciate its limitations. Fork has served us well for more than five decades; I've got no argument with that. However, should we never question whether it continues to be the right, or best, abstraction as the environment around it continues to evolve? - Dan C. [-- Attachment #2: Type: text/html, Size: 2200 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [TUHS] Systematic approach to command-line interfaces 2021-08-02 21:25 ` Dan Cross @ 2021-08-02 21:59 ` Jon Steinhart 2021-08-02 22:33 ` John Cowan 2021-08-03 0:20 ` [TUHS] fork (Re: " Bakul Shah 1 sibling, 1 reply; 20+ messages in thread From: Jon Steinhart @ 2021-08-02 21:59 UTC (permalink / raw) To: The Eunuchs Hysterical Society Dan Cross writes: > That's a bit unfair. One can understand something and see value in it and > still appreciate its limitations. > > Fork has served us well for more than five decades; I've got no argument > with that. However, should we never question whether it continues to be the > right, or best, abstraction as the environment around it continues to > evolve? Oh, sorry, wasn't meaning to be categorical there. Main reason that it came to mind was John's web example; many have said in the past that the UNIX model couldn't do that until they figured out that it actually could. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [TUHS] Systematic approach to command-line interfaces 2021-08-02 21:59 ` Jon Steinhart @ 2021-08-02 22:33 ` John Cowan 0 siblings, 0 replies; 20+ messages in thread From: John Cowan @ 2021-08-02 22:33 UTC (permalink / raw) To: Jon Steinhart; +Cc: The Eunuchs Hysterical Society [-- Attachment #1: Type: text/plain, Size: 823 bytes --] On Mon, Aug 2, 2021 at 6:00 PM Jon Steinhart <jon@fourwinds.com> wrote:> > Oh, sorry, wasn't meaning to be categorical there. Main reason that it > came > to mind was John's web example; many have said in the past that the UNIX > model couldn't do that until they figured out that it actually could. > I sorta lost track of what I was saying there: spawn*() would work fine in pipelines, since they involve fork-quickly-followed-by-exec. Doug's nifty sieve example, on the other hand, would not: the Right Thing there is Go, or else goroutines in C (either libmill or its successor libdill as you prefer) since the sieve doesn't actually involve any sort of global state for which processes are relevant. Granted, the C libraries have a little bit of x86_64 asm in them (but you are not expected to understand this). [-- Attachment #2: Type: text/html, Size: 1690 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
* [TUHS] fork (Re: Systematic approach to command-line interfaces 2021-08-02 21:25 ` Dan Cross 2021-08-02 21:59 ` Jon Steinhart @ 2021-08-03 0:20 ` Bakul Shah 2021-08-03 0:59 ` Adam Thornton 1 sibling, 1 reply; 20+ messages in thread From: Bakul Shah @ 2021-08-03 0:20 UTC (permalink / raw) To: Dan Cross; +Cc: The Eunuchs Hysterical Society On Aug 2, 2021, at 2:25 PM, Dan Cross <crossd@gmail.com> wrote: > > Fork has served us well for more than five decades; I've got no argument with that. However, should we never question whether it continues to be the right, or best, abstraction as the environment around it continues to evolve? An os with no fork() can be used to reimplement fork() for backward compatibility. You will anyway need all the things fork() used to do in this brave new world. For example, you will need to specify which signals and file descriptors to inherit. Something like: jmp_buf state; pid = 0; if (!setjmp(state) && !pid) { fd = openAt(hostfd, "/n/address-space/new", ...); <<copy /proc/self/mem to fd.>> pid = breathe_life(fd, state, signals, fds); longjmp(state, 1); return pid; } The /n/address-space filesystem yields a new address space on a given host (as represented by the hostfd). You can then "copy" to it to fill it up from the current proc's memory. Copy should be done using COW if the new proc is on the same host (or even remote but that opens up another area of discussion...). breathe_life is given where everything is set up and it just has to create a new thread, associated with the address, wire up signals as file descriptors that are to be inherited and arrange things so that on return from syscall in the child, process state will be as per "state". If signals and fds can be wired up to a remote host, this can be used to "migrate" a process. There is likely much more process state in the kernel mode which would have to be packaged up somehow. There may be other breakage if the child & parent are on different hosts. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [TUHS] fork (Re: Systematic approach to command-line interfaces 2021-08-03 0:20 ` [TUHS] fork (Re: " Bakul Shah @ 2021-08-03 0:59 ` Adam Thornton 2021-08-03 1:20 ` Steve Nickolas 0 siblings, 1 reply; 20+ messages in thread From: Adam Thornton @ 2021-08-03 0:59 UTC (permalink / raw) To: The Eunuchs Hysterical Society > On Aug 2, 2021, at 5:20 PM, Bakul Shah <bakul@iitbombay.org> wrote: > > On Aug 2, 2021, at 2:25 PM, Dan Cross <crossd@gmail.com> wrote: >> >> Fork has served us well for more than five decades; I've got no argument with that. However, should we never question whether it continues to be the right, or best, abstraction as the environment around it continues to evolve? > > An os with no fork() can be used to reimplement fork() for backward > compatibility. You will anyway need all the things fork() used to do > in this brave new world. Oh man, now I’m having flashbacks to early Cygwin and what was that, DJGPP for OS/2? Well, I picked a lousy week to give up sniffing glue anyway. Adam ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [TUHS] fork (Re: Systematic approach to command-line interfaces 2021-08-03 0:59 ` Adam Thornton @ 2021-08-03 1:20 ` Steve Nickolas 0 siblings, 0 replies; 20+ messages in thread From: Steve Nickolas @ 2021-08-03 1:20 UTC (permalink / raw) To: Adam Thornton; +Cc: The Eunuchs Hysterical Society [-- Attachment #1: Type: text/plain, Size: 773 bytes --] On Mon, 2 Aug 2021, Adam Thornton wrote: > > >> On Aug 2, 2021, at 5:20 PM, Bakul Shah <bakul@iitbombay.org> wrote: >> >> On Aug 2, 2021, at 2:25 PM, Dan Cross <crossd@gmail.com> wrote: >>> >>> Fork has served us well for more than five decades; I've got no argument with that. However, should we never question whether it continues to be the right, or best, abstraction as the environment around it continues to evolve? >> >> An os with no fork() can be used to reimplement fork() for backward >> compatibility. You will anyway need all the things fork() used to do >> in this brave new world. > > Oh man, now I’m having flashbacks to early Cygwin and what was that, DJGPP for OS/2? > > Well, I picked a lousy week to give up sniffing glue anyway. > > Adam EMX. -uso. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [TUHS] Systematic approach to command-line interfaces 2021-08-02 18:15 ` Adam Thornton 2021-08-02 18:24 ` Warner Losh 2021-08-02 20:55 ` John Cowan @ 2021-08-03 0:21 ` Bakul Shah 2021-08-03 1:49 ` Jon Steinhart 2 siblings, 1 reply; 20+ messages in thread From: Bakul Shah @ 2021-08-03 0:21 UTC (permalink / raw) To: Adam Thornton; +Cc: The Eunuchs Hysterical Society Or perhaps the issue is not having graphics/GUI designers with the creativity and sensibilities of the early Bell Labs crowd of researchers? I keep thinking there ought to be something simpler/more elegant than the current graphics subsystems.... > On Aug 2, 2021, at 11:16 AM, Adam Thornton <athornton@gmail.com> wrote: > > > It's a measure of Unix having been wounded by its own success. > > fork() is a great model for a single-threaded text processing pipeline to do automated typesetting. (More generally, anything that is a straightforward composition of filter/transform stages.) Which is, y'know, what Unix is *for*. > > It's not so great for a responsive GUI in front of a multi-function interactive program. > > These days, the vast majority of Unix applications are "stuff people play with on their phones." > > Adam > > On Mon, Aug 2, 2021 at 7:59 AM Theodore Ts'o <tytso@mit.edu> wrote: > On Sun, Aug 01, 2021 at 10:42:53PM -0400, Douglas McIlroy wrote: > > > spawn() beats fork()[;] fork() should be deprecated > > > > Spawn is a further complication of exec, which tells what signals and > > file descriptors to inherit in addition to what arguments and > > environment variables to pass. > > > > Fork has a place. For example, Program 1 in > > www.cs.dartmouth.edu/~doug/sieve/sieve.pdf forks like crazy and never > > execs. To use spawn, the program would have to be split in three (or > > be passed a switch setting). > > > > While you may dismiss Program 1 as merely a neat demo, the same idea > > applies in parallelizing code for use in a multiprocessor world. > > It's certainly clear that some kind of primitive is needed to create > new threads. An open question is whether if there exists some kind of > "new thread" primitve plus either spawn(2) or some kind of "create a > child process and then then frob like crazy using 'echo XXX > > /proc/<pid>/<magic files>'" whether there still is a need for a > fork(2) system call. > > Obviously, as soon as we start going down this path, we're deviated > quite strongly from the "radical simplicity" of Unix Version 7 that > people have accused modern systems (whether they be Linux or FreeBSD) > of lacking. It's rather interesting that we haven't heard complaints > about how people who dare to try come up with new API's are somehow > traitors to "The Unix Philosphy" that we've seen on other threads. :-) > > - Ted ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [TUHS] Systematic approach to command-line interfaces 2021-08-03 0:21 ` [TUHS] " Bakul Shah @ 2021-08-03 1:49 ` Jon Steinhart 2021-08-03 3:21 ` Adam Thornton 2021-08-03 3:27 ` Bakul Shah 0 siblings, 2 replies; 20+ messages in thread From: Jon Steinhart @ 2021-08-03 1:49 UTC (permalink / raw) To: The Eunuchs Hysterical Society Bakul Shah writes: > Or perhaps the issue is not having graphics/GUI designers with the > creativity and sensibilities of the early Bell Labs crowd of researchers? > I keep thinking there ought to be something simpler/more elegant than > the current graphics subsystems.... > > > On Aug 2, 2021, at 11:16 AM, Adam Thornton <athornton@gmail.com> wrote: > > > > It's a measure of Unix having been wounded by its own success. > > > > fork() is a great model for a single-threaded text processing pipeline to > do automated typesetting. (More generally, anything that is a straightforward > composition of filter/transform stages.) Which is, y'know, what Unix is *for*. > > > > It's not so great for a responsive GUI in front of a multi-function interactive program. > > > > These days, the vast majority of Unix applications are "stuff people play with on their phones." > > > > Adam I thought that I posted something about this recently when someone was arguing for threads being unnecessary and bad. My two cents is that GUIs were the big driver for threads. I would personally like to get rid of them as they're "the same but different" with regards to processes. My preference would be to solve the heaviness of processes problem. I'm not in the thick of that these days, but I don't see it being solved in software alone; it's going to take some serious hardware/software architecture work. Might be easier to accomplish now that the world has pretty much settled on the process model. Jon ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [TUHS] Systematic approach to command-line interfaces 2021-08-03 1:49 ` Jon Steinhart @ 2021-08-03 3:21 ` Adam Thornton 2021-08-03 3:27 ` Bakul Shah 1 sibling, 0 replies; 20+ messages in thread From: Adam Thornton @ 2021-08-03 3:21 UTC (permalink / raw) To: The Eunuchs Hysterical Society > On Aug 2, 2021, at 6:49 PM, Jon Steinhart <jon@fourwinds.com> wrote: > > My two cents is that GUIs were the big driver for threads. I would > personally like to get rid of them as they're "the same but different" > with regards to processes. My preference would be to solve the > heaviness of processes problem. I'm not in the thick of that these > days, but I don't see it being solved in software alone; it's going > to take some serious hardware/software architecture work. Might be > easier to accomplish now that the world has pretty much settled on > the process model. Has it, though? Most of the stuff I’m working with right now is either asyncio and Python, or Javascript, both of which are very much about threading. I think there’s a lot of stuff that does work better with multiple program counters for different things executing at the same time with shared access to memory, although of course reasoning about concurrency is always hard and having to manage locks on things that shouldn’t be shared _right now_ is a lot of work and easy to get wrong. I like Go and goroutines and channels as an intra-process communication mechanism. My problem with processes, in the traditional sense, is that there ARE often pieces of state you want to share between the things concurrently acting on that state. A channel-like mechanism could work as IPC, but, equally well, why _not_ have multiple things—we can call them threads if you want—that can just see that shared state? Adam ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [TUHS] Systematic approach to command-line interfaces 2021-08-03 1:49 ` Jon Steinhart 2021-08-03 3:21 ` Adam Thornton @ 2021-08-03 3:27 ` Bakul Shah 2021-08-03 3:51 ` Jon Steinhart 1 sibling, 1 reply; 20+ messages in thread From: Bakul Shah @ 2021-08-03 3:27 UTC (permalink / raw) To: Jon Steinhart; +Cc: The Eunuchs Hysterical Society On Aug 2, 2021, at 6:49 PM, Jon Steinhart <jon@fourwinds.com> wrote: > > Bakul Shah writes: >> Or perhaps the issue is not having graphics/GUI designers with the >> creativity and sensibilities of the early Bell Labs crowd of researchers? >> I keep thinking there ought to be something simpler/more elegant than >> the current graphics subsystems.... >> >>> On Aug 2, 2021, at 11:16 AM, Adam Thornton <athornton@gmail.com> wrote: >>> >>> It's a measure of Unix having been wounded by its own success. >>> >>> fork() is a great model for a single-threaded text processing pipeline to >> do automated typesetting. (More generally, anything that is a straightforward >> composition of filter/transform stages.) Which is, y'know, what Unix is *for*. >>> >>> It's not so great for a responsive GUI in front of a multi-function interactive program. >>> >>> These days, the vast majority of Unix applications are "stuff people play with on their phones." >>> >>> Adam > > I thought that I posted something about this recently when someone > was arguing for threads being unnecessary and bad. > > My two cents is that GUIs were the big driver for threads. I would > personally like to get rid of them as they're "the same but different" > with regards to processes. My preference would be to solve the > heaviness of processes problem. I'm not in the thick of that these > days, but I don't see it being solved in software alone; it's going > to take some serious hardware/software architecture work. Might be > easier to accomplish now that the world has pretty much settled on > the process model. > > Jon AFAIK pretty much all GUI frameworks are (or were) single threaded. At least I don't see the GUI as the main motivation for threads. I was mainly complaining about the complexity of graphics subsystem, including user interaction. Don't see what fork() has to do with it. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [TUHS] Systematic approach to command-line interfaces 2021-08-03 3:27 ` Bakul Shah @ 2021-08-03 3:51 ` Jon Steinhart 0 siblings, 0 replies; 20+ messages in thread From: Jon Steinhart @ 2021-08-03 3:51 UTC (permalink / raw) To: The Eunuchs Hysterical Society Bakul Shah writes: > On Aug 2, 2021, at 6:49 PM, Jon Steinhart <jon@fourwinds.com> wrote: > > > > Bakul Shah writes: > >> Or perhaps the issue is not having graphics/GUI designers with the > >> creativity and sensibilities of the early Bell Labs crowd of researchers? > >> I keep thinking there ought to be something simpler/more elegant than > >> the current graphics subsystems.... > >> > >>> On Aug 2, 2021, at 11:16 AM, Adam Thornton <athornton@gmail.com> wrote: > >>> > >>> It's a measure of Unix having been wounded by its own success. > >>> > >>> fork() is a great model for a single-threaded text processing pipeline to > >> do automated typesetting. (More generally, anything that is a straightforward > >> composition of filter/transform stages.) Which is, y'know, what Unix is *for*. > >>> > >>> It's not so great for a responsive GUI in front of a multi-function interactive program. > >>> > >>> These days, the vast majority of Unix applications are "stuff people play with on their phones." > >>> > >>> Adam > > > > I thought that I posted something about this recently when someone > > was arguing for threads being unnecessary and bad. > > > > My two cents is that GUIs were the big driver for threads. I would > > personally like to get rid of them as they're "the same but different" > > with regards to processes. My preference would be to solve the > > heaviness of processes problem. I'm not in the thick of that these > > days, but I don't see it being solved in software alone; it's going > > to take some serious hardware/software architecture work. Might be > > easier to accomplish now that the world has pretty much settled on > > the process model. > > > > Jon > > AFAIK pretty much all GUI frameworks are (or were) single threaded. > At least I don't see the GUI as the main motivation for threads. > > I was mainly complaining about the complexity of graphics subsystem, > including user interaction. Don't see what fork() has to do with it. Not my experience. At least from what I was working on the 1980s there was a need to be able to keep complicated state in graphical programs which is when assembly language versions of threads came into being. Made it possible for whatever was generating output to be interrupted so that input could be handled in a timely manner. I seem to recall getting some code from Sun for this that I ported to a different system. Jon ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [TUHS] Systematic approach to command-line interfaces 2021-08-02 14:58 ` Theodore Ts'o 2021-08-02 18:15 ` Adam Thornton @ 2021-08-03 7:19 ` arnold 2021-08-03 23:12 ` Andrew Warkentin 1 sibling, 1 reply; 20+ messages in thread From: arnold @ 2021-08-03 7:19 UTC (permalink / raw) To: tytso, douglas.mcilroy; +Cc: tuhs "Theodore Ts'o" <tytso@mit.edu> wrote: > It's certainly clear that some kind of primitive is needed to create > new threads. An open question is whether if there exists some kind of > "new thread" primitve plus either spawn(2) or some kind of "create a > child process and then then frob like crazy using 'echo XXX > > /proc/<pid>/<magic files>'" whether there still is a need for a > fork(2) system call. I haven't caught up yet in this thread. Apologies if this has been discussed already. The Plan 9 folks blazed this trail over 30 years ago with rfork, where you specify what bits you wish to duplicate. I don't remember details anymore, but I think it was pretty elegant. IIRC Around that time Rob Pike said "Threads are the lack of an idea", meaning, if you think you need threads, you haven't thought about the problem hard enough. (Apologies to Rob if I am misremembering and/or misrepresenting.) Arnold ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [TUHS] Systematic approach to command-line interfaces 2021-08-03 7:19 ` arnold @ 2021-08-03 23:12 ` Andrew Warkentin 2021-08-04 15:04 ` Paul Winalski 0 siblings, 1 reply; 20+ messages in thread From: Andrew Warkentin @ 2021-08-03 23:12 UTC (permalink / raw) To: tuhs On 8/3/21, arnold@skeeve.com <arnold@skeeve.com> wrote: > > I haven't caught up yet in this thread. Apologies if this has been > discussed already. > > The Plan 9 folks blazed this trail over 30 years ago with rfork, where > you specify what bits you wish to duplicate. I don't remember details > anymore, but I think it was pretty elegant. IIRC Around that time Rob Pike > said "Threads are the lack of an idea", meaning, if you think you need > threads, you haven't thought about the problem hard enough. (Apologies > to Rob if I am misremembering and/or misrepresenting.) > I've never really been a fan of the rfork()/clone() model, or at least the Linux implementation of it that requires ugly library-level hacks to share state between threads that the kernel doesn't support sharing. Also, I don't really care for the large number of flags required. Up until now I was just planning on following the traditional threading model of associating most state with processes with only execution state being per-thread in the OS I'm working on, but now I'm thinking I should reduce the state associated with a process to just the PID, PPID, PGID, containing cgroup, command line, and list of threads. All other state would be contained in various types of context objects that are not tied to a particular process or thread (other than being destroyed when no more threads are associated with them). This would include: Filesystem namespace File descriptors Address space Security context (file permission list, UID, GID) Signal handlers Scheduling context Each of these object types would be completely separate from the others, allowing full control over which state is shared and which is private. I'm using seL4 as a microkernel, and it already works like this (it has no real concept of processes, only threads that are each associated with an address space, a capability space, and a scheduling context) so it's a good match for it. exec() would still replace all threads within a process as on traditional Unix, unless the exec is performed within a child process that hasn't yet been started. Sending a signal to an entire process would send it to every signal group within the process (similarly, it would be possible to send a signal to an entire cgroup; basically, processes will really just be a special kind of cgroup in this model). ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [TUHS] Systematic approach to command-line interfaces 2021-08-03 23:12 ` Andrew Warkentin @ 2021-08-04 15:04 ` Paul Winalski 0 siblings, 0 replies; 20+ messages in thread From: Paul Winalski @ 2021-08-04 15:04 UTC (permalink / raw) To: Andrew Warkentin; +Cc: tuhs On 8/3/21, Andrew Warkentin <andreww591@gmail.com> wrote: > > Up until now I was just planning on following the traditional > threading model of associating most state with processes with only > execution state being per-thread in the OS I'm working on, but now I'm > thinking I should reduce the state associated with a process to just > the PID, PPID, PGID, containing cgroup, command line, and list of > threads. All other state would be contained in various types of > context objects that are not tied to a particular process or thread > (other than being destroyed when no more threads are associated with > them). For what it's worth, this is the process model that Windows NT has. This thread has discussed two very different ways to do command line processing: the TOPS-20 model, where the program image for the command is activated immediately and then calls back to the OS to process the command line elements, and the UNIX model, where the first element of the command line is a path (full or partial) to the program image for the command, and it's entirely up to that image to process the command line elements as it sees fit. VMS takes a third approach. The VAX had four, hierarchical execution modes: user, supervisor, exec, and kernel, in increasing order of privilege. The VAX had "change mode" instructions (CHMU, CHMS, CHME, CHMK) that ran the current process's change-mode interrupt handler to process the request. Applications and programs to process commands ran in user mode. The DCL command interpreter (VMS's equivalent of the UNIX shell) ran in supervisor mode. Exec mode was reserved for record management services (RMS), VMS's equivalent of the file system code in UNIX. Kernel mode (the only mode where privileged instructions can be executed) was, of course, for the kernel. One oddity of VMS is that each process had the supervisor, exec, and kernel code and data mapped to its address space. Commands in VMS are described using a meta-language that is compiled into a set of data tables that the command line interpreter (CLI, running in supervisor mode) uses to determine which program image needs to be activated to execute the command. The CLI also does a full parse of the command line. It then calls the kernel's image activator to map the appropriate program image and switches to user mode to start it running. There is a runtime library of routines to retrieve the various options and parameters from the command line (the routines in this library do CHMS calls to achieve this). So on VMS we have a situation where the command interpreter (shell) is part of every process. Processes are created by the "create process" OS library routine, sys$creprc, which is a wrapper around the appropriate CHMK instruction. Unlike fork() on UNIX, the new process inherits very little state from its parent. Most significantly it inherits neither address space nor open file context. VMS process creation is also notoriously expensive compared to fork() on UNIX. -Paul W. ^ permalink raw reply [flat|nested] 20+ messages in thread
end of thread, other threads:[~2021-08-04 15:05 UTC | newest] Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-08-02 2:42 [TUHS] Systematic approach to command-line interfaces Douglas McIlroy 2021-08-02 14:58 ` Theodore Ts'o 2021-08-02 18:15 ` Adam Thornton 2021-08-02 18:24 ` Warner Losh 2021-08-02 20:55 ` John Cowan 2021-08-02 21:06 ` Jon Steinhart 2021-08-02 21:25 ` Dan Cross 2021-08-02 21:59 ` Jon Steinhart 2021-08-02 22:33 ` John Cowan 2021-08-03 0:20 ` [TUHS] fork (Re: " Bakul Shah 2021-08-03 0:59 ` Adam Thornton 2021-08-03 1:20 ` Steve Nickolas 2021-08-03 0:21 ` [TUHS] " Bakul Shah 2021-08-03 1:49 ` Jon Steinhart 2021-08-03 3:21 ` Adam Thornton 2021-08-03 3:27 ` Bakul Shah 2021-08-03 3:51 ` Jon Steinhart 2021-08-03 7:19 ` arnold 2021-08-03 23:12 ` Andrew Warkentin 2021-08-04 15:04 ` Paul Winalski
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).