* [TUHS] Threads vs... not @ 2021-08-04 21:48 Lawrence Stewart 2021-08-04 22:02 ` Larry McVoy ` (2 more replies) 0 siblings, 3 replies; 11+ messages in thread From: Lawrence Stewart @ 2021-08-04 21:48 UTC (permalink / raw) To: TUHS main list What do folks think about event-driven programming as a substitute for threads in UI and process control settings? I wrote the service processor code for the Sicortex Machines using libevent.a and I thought it was very lightweight and fairly easy to think about. (This was a thing running on ucLinux on a tiny 16 MB coldfire that managed the consoles and power supplies and temp sensors and JTAG access and booting and so forth.) Tk (IIRC) has a straightforward event driven model for UI interactions. Meanwhile, the dropbox plugin for my Mac has 120 threads running. WTF? This was triggered by the fork/spawn discussion. -Larry (started with Unix at V6 on an 11/34) ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [TUHS] Threads vs... not 2021-08-04 21:48 [TUHS] Threads vs... not Lawrence Stewart @ 2021-08-04 22:02 ` Larry McVoy 2021-08-04 22:41 ` John Cowan 2021-08-04 22:13 ` Anthony Martin 2021-08-05 23:02 ` Bakul Shah 2 siblings, 1 reply; 11+ messages in thread From: Larry McVoy @ 2021-08-04 22:02 UTC (permalink / raw) To: Lawrence Stewart; +Cc: TUHS main list From http://mcvoy.com/lm/quotes.html A computer is a state machine. Threads are for people who can't program state machines. Alan Cox On Wed, Aug 04, 2021 at 05:48:46PM -0400, Lawrence Stewart wrote: > What do folks think about event-driven programming as a substitute for threads in UI and process control settings? > > I wrote the service processor code for the Sicortex Machines using libevent.a and I thought it was very lightweight and fairly easy to think about. (This was a thing running on ucLinux on a tiny 16 MB coldfire that managed the consoles and power supplies and temp sensors and JTAG access and booting and so forth.) > > Tk (IIRC) has a straightforward event driven model for UI interactions. > > Meanwhile, the dropbox plugin for my Mac has 120 threads running. WTF? > > This was triggered by the fork/spawn discussion. > > -Larry > (started with Unix at V6 on an 11/34) -- --- Larry McVoy lm at mcvoy.com http://www.mcvoy.com/lm ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [TUHS] Threads vs... not 2021-08-04 22:02 ` Larry McVoy @ 2021-08-04 22:41 ` John Cowan 2021-08-04 22:51 ` Larry McVoy 0 siblings, 1 reply; 11+ messages in thread From: John Cowan @ 2021-08-04 22:41 UTC (permalink / raw) To: Larry McVoy; +Cc: TUHS main list [-- Attachment #1: Type: text/plain, Size: 465 bytes --] On Wed, Aug 4, 2021 at 6:02 PM Larry McVoy <lm@mcvoy.com> wrote: > A computer is a state machine. Threads are for people who can't > program > state machines. > > Alan Cox Orly? Try embedding an LL(1) parser in an event loop that gives you a new event every time a block is read off the disk. Event loops are just manual CPS transformations of coroutines -- but why do the transformation manually instead of having your compiler do it for you? [-- Attachment #2: Type: text/html, Size: 1315 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [TUHS] Threads vs... not 2021-08-04 22:41 ` John Cowan @ 2021-08-04 22:51 ` Larry McVoy 2021-08-05 20:32 ` Clem Cole 2021-08-06 0:55 ` Dan Cross 0 siblings, 2 replies; 11+ messages in thread From: Larry McVoy @ 2021-08-04 22:51 UTC (permalink / raw) To: John Cowan; +Cc: TUHS main list On Wed, Aug 04, 2021 at 06:41:08PM -0400, John Cowan wrote: > On Wed, Aug 4, 2021 at 6:02 PM Larry McVoy <lm@mcvoy.com> wrote: > > > > A computer is a state machine. Threads are for people who can't > > program > > state machines. > > > > Alan Cox > > > Orly? Try embedding an LL(1) parser in an event loop that gives you a new > event every time a block is read off the disk. > > Event loops are just manual CPS transformations of coroutines -- but why do > the transformation manually instead of having your compiler do it for you? The counter to this is Solaris tried to allocate a thread for each 8K page on it's way to disk. The thread stack was 16K. This model, while seen as ever so elegant, means that 2/3rds of main memory were thread stacks. Sometimes threads make sense, there they did not. -- --- Larry McVoy lm at mcvoy.com http://www.mcvoy.com/lm ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [TUHS] Threads vs... not 2021-08-04 22:51 ` Larry McVoy @ 2021-08-05 20:32 ` Clem Cole 2021-08-06 0:55 ` Dan Cross 1 sibling, 0 replies; 11+ messages in thread From: Clem Cole @ 2021-08-05 20:32 UTC (permalink / raw) To: Larry McVoy; +Cc: TUHS main list [-- Attachment #1: Type: text/plain, Size: 1042 bytes --] On Wed, Aug 4, 2021 at 6:51 PM Larry McVoy <lm@mcvoy.com> wrote: > Sometimes threads make sense, there they did not. Right. The problem is too often people have a hammer and make everything a nail. Or they have an electric screwdriver and then try to use if for everything. The trick is to teach good taste and when certain solutions make more sense than others. Allocating a thread for every page being sent to disk, or needing 120 threads for a simple application -- somebody missed the boat. Funny, used to deliver nails in kegs. Hit with a hammer - wood fastened. Somebody develops a screw system. A 'better fastener.' Indeed it does work great for >>some jobs<< but not all. Frankly that sucks for framing. What do you see at any real job site. Not screws but a pneumatic hammer and nail cartridges? Changed the deliver system from key and forcing somebody to swing the hammer. Lesson learned to *use the proper tool *or update the tool if the >>technique<< is the right one. Kids these days ... get off my lawn... Clem [-- Attachment #2: Type: text/html, Size: 2722 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [TUHS] Threads vs... not 2021-08-04 22:51 ` Larry McVoy 2021-08-05 20:32 ` Clem Cole @ 2021-08-06 0:55 ` Dan Cross 1 sibling, 0 replies; 11+ messages in thread From: Dan Cross @ 2021-08-06 0:55 UTC (permalink / raw) To: Larry McVoy; +Cc: TUHS main list [-- Attachment #1: Type: text/plain, Size: 1859 bytes --] On Wed, Aug 4, 2021 at 6:51 PM Larry McVoy <lm@mcvoy.com> wrote: > On Wed, Aug 04, 2021 at 06:41:08PM -0400, John Cowan wrote: > > On Wed, Aug 4, 2021 at 6:02 PM Larry McVoy <lm@mcvoy.com> wrote: > > > A computer is a state machine. Threads are for people who can't > > > program state machines. > > > > > > Alan Cox > > > > Orly? Try embedding an LL(1) parser in an event loop that gives you a > new > > event every time a block is read off the disk. > > > > Event loops are just manual CPS transformations of coroutines -- but why > do > > the transformation manually instead of having your compiler do it for > you? > > The counter to this is Solaris tried to allocate a thread for each 8K page > on it's way to disk. The thread stack was 16K. This model, while seen as > ever so elegant, means that 2/3rds of main memory were thread stacks. > Honestly this sounds like a misapplication of threads. It makes more sense to me to have a small pool of workers working on a bounded queue or something. Sometimes threads make sense, there they did not. It strikes me that the Unix kernel we all know and love has been multithreaded since the early 1970s, when multiprogramming overlapped with IO was introduced; even in something as simple as 6th Edition, every process has a corresponding kernel thread and the fundamental mechanism to context switch from process $n$ to process $m$ is for n to trap into the kernel where it's running on n's kernel thread, invoke the scheduler to perform a thread switch to m's kernel thread, and then return to m's userspace. Various attempts at replacing multithreaded kernels with event-driven state machines have been proposed and tried, but in the end, I always come away with the sense that most of the time, when used judiciously, threads are wonderful concurrency primitives. - Dan C. [-- Attachment #2: Type: text/html, Size: 2550 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [TUHS] Threads vs... not 2021-08-04 21:48 [TUHS] Threads vs... not Lawrence Stewart 2021-08-04 22:02 ` Larry McVoy @ 2021-08-04 22:13 ` Anthony Martin 2021-08-05 23:02 ` Bakul Shah 2 siblings, 0 replies; 11+ messages in thread From: Anthony Martin @ 2021-08-04 22:13 UTC (permalink / raw) To: TUHS main list Lawrence Stewart <stewart@serissa.com> once said: > What do folks think about event-driven programming as a substitute for > threads in UI and process control settings? Why not both? https://swtch.com/~rsc/talks/threads07/ https://swtch.com/~rsc/thread/cws.pdf Cheers, Anthony ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [TUHS] Threads vs... not 2021-08-04 21:48 [TUHS] Threads vs... not Lawrence Stewart 2021-08-04 22:02 ` Larry McVoy 2021-08-04 22:13 ` Anthony Martin @ 2021-08-05 23:02 ` Bakul Shah 2021-08-06 14:19 ` Steffen Nurpmeso 2 siblings, 1 reply; 11+ messages in thread From: Bakul Shah @ 2021-08-05 23:02 UTC (permalink / raw) To: Lawrence Stewart; +Cc: TUHS main list On Aug 4, 2021, at 2:48 PM, Lawrence Stewart <stewart@serissa.com> wrote: > > What do folks think about event-driven programming as a substitute for threads in UI and process control settings? The equivalence here is with coroutines (sometimes called green-threads), not true threads. True threads can take advantage of multiple cores. Your event-driven program won't benefit from multiple cores. > I wrote the service processor code for the Sicortex Machines using libevent.a and I thought it was very lightweight and fairly easy to think about. (This was a thing running on ucLinux on a tiny 16 MB coldfire that managed the consoles and power supplies and temp sensors and JTAG access and booting and so forth.) I wrote the RealNetworks media server code using an event-driven model. It was efficient but the control flow got quite messy as effectively my code had to do explicit continuation passing. Given that each media stream was independent of other streams, a thread based model was far simpler. That is what a colleague did to test the server code! As a thought experiment consider implementing a Unix style pipeline but without an OS that provides the illusion of independent processes or threads and where each stage in the pipeline is a separate function. Try implementing this in Go which has support for threads vs C (but without pthreads). I have used coroutines for hardware simulation a number of times, the first time was in 1982 for simulating bus transactions to see the impact of adding an ethernet controller (5.6Mhz clock, memory access 4 clocks). The last time was to simulate an ethernet switch with complex i/o logic. One can think of each hardware module as a separate state machine but smushing them into one giant state machine would not been easy as the code was being developed and debugging it would be very painful. One would then be tempted to write a "preprocessor" language of some sort to hide all this.... > Tk (IIRC) has a straightforward event driven model for UI interactions. > > Meanwhile, the dropbox plugin for my Mac has 120 threads running. WTF? Any system can be abused but the abuse-case should not be mistaken for the normal use-case. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [TUHS] Threads vs... not 2021-08-05 23:02 ` Bakul Shah @ 2021-08-06 14:19 ` Steffen Nurpmeso 2021-08-06 14:43 ` John Cowan 0 siblings, 1 reply; 11+ messages in thread From: Steffen Nurpmeso @ 2021-08-06 14:19 UTC (permalink / raw) To: Bakul Shah; +Cc: TUHS main list Bakul Shah wrote in <74E8349E-95A4-40C9-B429-11A4E396BE12@iitbombay.org>: |On Aug 4, 2021, at 2:48 PM, Lawrence Stewart <stewart@serissa.com> wrote: |> |> What do folks think about event-driven programming as a substitute \ |> for threads in UI and process control settings? | |The equivalence here is with coroutines (sometimes called |green-threads), not true threads. True threads can take |advantage of multiple cores. Your event-driven program won't |benefit from multiple cores. | |> I wrote the service processor code for the Sicortex Machines using \ |> libevent.a and I thought it was very lightweight and fairly easy \ |> to think about. (This was a thing running on ucLinux on a tiny 16 \ |> MB coldfire that managed the consoles and power supplies and temp \ |> sensors and JTAG access and booting and so forth.) | |I wrote the RealNetworks media server code using an |event-driven model. It was efficient but the control flow got |quite messy as effectively my code had to do explicit |continuation passing. Given that each media stream was |independent of other streams, a thread based model was far |simpler. That is what a colleague did to test the server |code! Only twenty years ago but i was under the impression that i got good (better) performance by having a single event loop object (thread) doing the select(2) aka the OS interaction, as the driver under the hood of an IOEvent thing, which then were emitted. These then dispatched to worker threads, or not. It may be different today with those highly refined mechanisms poll, epoll, kqueue. It may anyway be different with whatever operating system specific things, for example FreeBSD had or has some network ring where user and kernel space could simply exchange buffers, ?, well i remember it was written by an Italien (Professor?) living in Pisa, in sight of the lopsided tower. And then the audioring technology Creative Labs shipped with X-Fi ~2005. --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [TUHS] Threads vs... not 2021-08-06 14:19 ` Steffen Nurpmeso @ 2021-08-06 14:43 ` John Cowan 2021-08-06 15:12 ` Steffen Nurpmeso 0 siblings, 1 reply; 11+ messages in thread From: John Cowan @ 2021-08-06 14:43 UTC (permalink / raw) To: Bakul Shah, Lawrence Stewart, TUHS main list [-- Attachment #1: Type: text/plain, Size: 1848 bytes --] On Fri, Aug 6, 2021 at 10:20 AM Steffen Nurpmeso <steffen@sdaoden.eu> wrote: > Bakul Shah wrote in > <74E8349E-95A4-40C9-B429-11A4E396BE12@iitbombay.org>: > |It was efficient but the control flow got > |quite messy as effectively my code had to do explicit > |continuation passing. > > Only twenty years ago but i was under the impression that i got > good (better) performance by having a single event loop object > (thread) doing the select(2) aka the OS interaction, as the driver > under the hood of an IOEvent thing, which then were emitted. > It sounds like you are in violent agreement: you get more performance with an event loop. So perhaps Cox's claim is true after all: threads are for programmers who can't deal with events. And then my claim is that at a certain not-very-high level of complexity no one can deal with events. After all, the interface to all modern operating systems is as a coroutine: when we call read() we give up control to the kernel, who does the read for us and delivers it to a buffer in our space, then surrenders control back. (How it looks from inside the kernel is another story.) Only when we request signal handling does the kernel actually interrupt the ordered flow within our program, but the restrictions on what a signal handler can do in Posix are so many that it's hardly safe to do more than set a flag to be polled later or to do a longjmp() to invoke a continuation further up the stack. Historic operating systems worked quite differently: they had the equivalent of signals that meant "disk block has been read" (use the buffer, and fast, before the OS starts to write the next block into it) and "disk block has been written" (the buffer is available for reuse). And programming for them was, in someone's memorable phrase, like moving dead whales along the beach by kicking them. [-- Attachment #2: Type: text/html, Size: 3052 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [TUHS] Threads vs... not 2021-08-06 14:43 ` John Cowan @ 2021-08-06 15:12 ` Steffen Nurpmeso 0 siblings, 0 replies; 11+ messages in thread From: Steffen Nurpmeso @ 2021-08-06 15:12 UTC (permalink / raw) To: John Cowan; +Cc: Bakul Shah, TUHS main list John Cowan wrote in <CAD2gp_SH5qt2G0oyHeDYnQbjmHeejz9A8FJhCvBFOKq21aAVkg@mail.gmail.com>: |On Fri, Aug 6, 2021 at 10:20 AM Steffen Nurpmeso <steffen@sdaoden.eu> \ |wrote: |> Bakul Shah wrote in |> <74E8349E-95A4-40C9-B429-11A4E396BE12@iitbombay.org>: |>|It was efficient but the control flow got |>|quite messy as effectively my code had to do explicit |>|continuation passing. |> |> Only twenty years ago but i was under the impression that i got |> good (better) performance by having a single event loop object |> (thread) doing the select(2) aka the OS interaction, as the driver |> under the hood of an IOEvent thing, which then were emitted. |> | |It sounds like you are in violent agreement: you get more performance with |an event loop. So perhaps Cox's claim is true after all: threads are for |programmers who can't deal with events. And then my claim is that at a |certain not-very-high level of complexity no one can deal with events. Well i am just too old and lazy to really dig into the asynchronous hype you see "everywhere". I know just for fun i did some MacOS Audio thing around 2009/10 (when i bought the single Apple i once wanted), nothing but a simple OggVorbis player (because Quicktime just did not like free Audio, and because i was interested), and those callbacks i had to deal with just drove me insane. The Linux Jack Audio Server also does that, i think. I do not understand. The CPU usage was just _grazy_, even though i used the biggest buffers i could use (iirc). I mean, i could drive audio play and recording side-by-side (with programming work) on a Cyrix 166+ with a 3 MB/sec (30 MB/sec with caching, there was some I/O test on Linux, forgotten the name) hard disk on FreeBSD 4.7 or what, it was nice(1)d, but other than that, and i think under 1 or 3 percent CPU. That is how it has to be! And not to forget most async and maybe even many coroutine things just internally base on threads, i think OpenSSL for example does, so what looks so cheap and "tickless" is in effect a tremendous monster. I hope they at least use pools. And that the pools do not leak resources to wherever. No, by chance, i am simple-minded and drove non-blocking event loops, which nicely bundled I/O events and gave me timers for free. This scaled wonderful. And for multi-core and super efficiency it scales easily, too: either by having worker threads (or sandboxed, isolated processes), picked out of a pool, doing work as it happens, or maybe, today, scaled by X different event loops. (Like, for networking, having one accepting new connections, and several others handling them.) But this seems so "hyper", i wonder whether any of the actually used "super servers", say postfix, dovecot, exim, postfix, *sql, apache, nginx, let me add lighttpd now that i said postfix, as it works so nice, uses _so_ much scaling, in the actual everyday work. --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2021-08-06 15:12 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-08-04 21:48 [TUHS] Threads vs... not Lawrence Stewart 2021-08-04 22:02 ` Larry McVoy 2021-08-04 22:41 ` John Cowan 2021-08-04 22:51 ` Larry McVoy 2021-08-05 20:32 ` Clem Cole 2021-08-06 0:55 ` Dan Cross 2021-08-04 22:13 ` Anthony Martin 2021-08-05 23:02 ` Bakul Shah 2021-08-06 14:19 ` Steffen Nurpmeso 2021-08-06 14:43 ` John Cowan 2021-08-06 15:12 ` Steffen Nurpmeso
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).