From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=0.7 required=5.0 tests=DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,MAILING_LIST_MULTI, MISSING_HEADERS autolearn=no autolearn_force=no version=3.4.4 Received: from minnie.tuhs.org (minnie.tuhs.org [50.116.15.146]) by inbox.vuxu.org (Postfix) with ESMTP id B76FB23ED7 for ; Sun, 19 May 2024 10:31:01 +0200 (CEST) Received: from minnie.tuhs.org (localhost [IPv6:::1]) by minnie.tuhs.org (Postfix) with ESMTP id 2903643682; Sun, 19 May 2024 18:30:55 +1000 (AEST) Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by minnie.tuhs.org (Postfix) with ESMTPS id A109443680 for ; Sun, 19 May 2024 18:30:46 +1000 (AEST) Received: by mail-wr1-x431.google.com with SMTP id ffacd0b85a97d-34d7d04808bso1103102f8f.0 for ; Sun, 19 May 2024 01:30:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1716107445; x=1716712245; darn=tuhs.org; h=cc:subject:message-id:date:from:in-reply-to:references:mime-version :from:to:cc:subject:date:message-id:reply-to; bh=kArZr73Ts9zhGauQTm7ygWF6HzQ71KQyxXTABw7lb1Q=; b=JpS7nCZccfGDaNmqnuORvXvD2361zhxMrLF424FfYARtJOM1Xusvz0q3IU8ZOjqSGk Xw8FCkoPc0dhosX+02BxEMOT2ODPA51tw0V/Tcgvx0+LdKc+HXdzIv/fFgJTzRfSIhX0 VAUHp0h41yI3Eg7ZrlYr9YnUc4W8Xi1P8WAOJyiqdag7mY5lN3nE71TNtHx9VpCt7dhG hbodOxvoxaCyVXUVWtmt6ateZ8qCCnT7Q6gGm8jmWEpTkpI2bBACHAUzNnllWYNB6TRH MxKKwlAORoyxsaQYgvMhXIr5uEnaPuO2E27atZaiZzSXnFU/fUecaqkBVSIxAYGOWQDH sDMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716107445; x=1716712245; h=cc:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=kArZr73Ts9zhGauQTm7ygWF6HzQ71KQyxXTABw7lb1Q=; b=dr78L2e4Dh31tAW4T/qwzDFZALrff/brH1ArhzdvH88W5I8tlVHPdzvmY2G3ksY9h4 +Tr2lFjaLgq4XACJz2KLAfHgC9CVtSf+BZPmR/yVAVYGxLFmohu5UGSp89kkJjiwNr6J 1HyhUvIyH/klXNIK8znplpl6tZ5PuXZ+eKTs1pnQdfknG2xEExD3ejvLyID5g0iYEf+X Gmx9nlW1A8eAFawqFIUojH8jtTJw7lzr+Hg15wW0hEfaDjvHxwyzYPHFpPc5A080SEEU B+mP1fwSnqh+M84oCt686MSqXsQyNpUu0flPApDlvtXy+N+gL05Br5C3Lgt4nsAq2GyT vFpg== X-Gm-Message-State: AOJu0YwIyUPTFSrXxyhFcd3/oifpVIpryO91inccdvQLut5TbhHhgNFo d5XLNPvGL2RkEOLTXgqWIZJc8GCtCLBifx6SKjv/KPUYOc8QZS8sk5jiHAchn5vQrBpcCn3hlkb SUXN/FnRBPKCGYbsVDLKv0NItbFsb X-Google-Smtp-Source: AGHT+IE9Hy8DoKnprBlLbbVfTh8U3JuAwVqoc+CdbIXM1VJRMF55Gl6hCeYlF+Zf49HjUm86JloaJ0KQPpUn82MVIZM= X-Received: by 2002:a5d:61cd:0:b0:34c:1c1e:a322 with SMTP id ffacd0b85a97d-3504a631154mr24274986f8f.22.1716107444484; Sun, 19 May 2024 01:30:44 -0700 (PDT) MIME-Version: 1.0 References: <20240514111032.2kotrrjjv772h5f4@illithid> <20240515164212.beswgy4h2nwvbdck@illithid> <8D556958-0C7F-43F3-8694-D7391E9D89DA@iitbombay.org> <20240519012114.GU9216@mcvoy.com> <767E78C5-E6E7-4CB5-889D-B4E0E5FBA085@iitbombay.org> <20240519020256.GV9216@mcvoy.com> In-Reply-To: From: Marc Rochkind Date: Sun, 19 May 2024 11:30:32 +0300 Message-ID: Cc: The Unix Heritage Society mailing list Content-Type: multipart/alternative; boundary="000000000000a002790618ca662a" Message-ID-Hash: UVW42IRP5NR7UWNMUIOT46SXRSJYHMF5 X-Message-ID-Hash: UVW42IRP5NR7UWNMUIOT46SXRSJYHMF5 X-MailFrom: mrochkind@gmail.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.6b1 Precedence: list Subject: [TUHS] Re: If forking is bad, how about buffering? List-Id: The Unix Heritage Society mailing list Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --000000000000a002790618ca662a Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Yes, many classic commands -- cat, cp, and others -- were sleekly and succinctly written. In part because they were devoid of error checking. I recall how annoying it was one time in the early 70s to cp a bunch of files to a file system that was out of space. As I grew older, my concept of what constituted elegant programming changed= . UNIX was a *research* project, not a production system! At one of the first UNIX meetings, somebody from an OSS (operations support system) was talking about the limitations of UNIX when Doug asked, "Why are you using UNIX?" Marc On Sun, May 19, 2024, 5:54=E2=80=AFAM Andrew Warkentin wrote: > On Sat, May 18, 2024 at 8:03=E2=80=AFPM Larry McVoy wrote: > > > > On Sat, May 18, 2024 at 06:40:42PM -0700, Bakul Shah wrote: > > > On May 18, 2024, at 6:21???PM, Larry McVoy wrote: > > > > > > > > On Sat, May 18, 2024 at 06:04:23PM -0700, Bakul Shah via TUHS wrote= : > > > >> [1] This brings up a separate point: in a microkernel even a simpl= e > > > >> thing like "foo | bar" would require a third process - a "pipe > > > >> service", to buffer up the output of foo! You may have reduced > > > >> the overhead of individual syscalls but you will have more of > > > >> cross-domain calls! > > > > > > > > Do any micro kernels do address space to address space bcopy()? > > > > > > mmapping the same page in two processes won't be hard but now > > > you have complicated cat (or some iolib)! > > > > I recall asking Linus if that could be done to save TLB entries, as in > > multiple processes map a portion of their address space (at the same > > virtual location) and then they all use the same TLB entries for that > > part of their address space. He said it couldn't be done because the > > process ID concept was hard wired into the TLB. I don't know if TLB > > tech has evolved such that a single process could have multiple "proces= s" > > IDs associated with it in the TLB. > > > > I wanted it because if you could share part of your address space with > > another process, using the same TLB entries, then motivation for thread= s > > could go away (I've never been a threads fan but I acknowledge why > > you might need them). I was channeling Rob's "If you think you need > > threads, your processes are too fat". > > > > The idea of using processes instead of threads falls down when you > > consider TLB usage. And TLB usage, when you care about performance, is > > an issue. I could craft you some realistic benchmarks, mirroring real > > world work loads, that would kill the idea of replacing threads with > > processes unless they shared TLB entries. Think of a N-way threaded > > application, lots of address space used, that application uses all of t= he > > TLB. Now do that with N processes and your TLB is N times less > effective. > > > > This was a conversation decades ago so maybe TLB tech now has solved > this. > > I doubt it, if this was a solved problem I think every OS would say scr= ew > > threads, just use processes and mmap(). The nice part of that model > > is you can choose what parts of your address space you want to share. > > That cuts out a HUGE swath of potential problems where another thread > > can go poke in a part of your address space that you don't want poked. > > > > I've never been a fan of the rfork()/clone() model. With the OS I'm > working on, rather than using processes that share state as threads, a > process will more or less just be a collection of threads that share a > command line and get replaced on exec(). All of the state usually > associated with a process (e.g. file descriptor space, filesystem > namespace, virtual address space, memory allocations) will instead be > stored in separate container objects that can be shared between > threads. It will be possible to share any of these containers between > processes, or use different combinations between threads within a > process. This would allow more control over what gets shared between > threads/processes than rfork()/clone() because the state containers > will appear in the filesystem and be explicitly bound to threads > rather than being anonymous and only transferred on rfork()/clone(). > Emulating rfork()/clone on top of this will be easy enough though. > --000000000000a002790618ca662a Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Yes, many classic commands -- cat, cp, and others -- were= sleekly and succinctly written.

In part because they were devoid of error checking.

I recall how annoying it was one time in the= early 70s to cp a bunch of files to a file system that was out of space.

As I grew older, my conce= pt of what constituted elegant programming changed.
=
UNIX was a research project, not a produ= ction system!

At one of = the first UNIX meetings, somebody from an OSS (operations support system) w= as talking about the limitations of UNIX when Doug asked, "Why are you= using UNIX?"

Marc<= /div>

On Sun, May 19, 2024, 5:54=E2=80=AFAM Andrew Warkentin <andreww591@gmail.com> wrote:
<= blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px= #ccc solid;padding-left:1ex">On Sat, May 18, 2024 at 8:03=E2=80=AFPM Larry= McVoy <lm@mcvoy.com> wrote:
>
> On Sat, May 18, 2024 at 06:40:42PM -0700, Bakul Shah wrote:
> > On May 18, 2024, at 6:21???PM, Larry McVoy <lm@mcvoy.com> wro= te:
> > >
> > > On Sat, May 18, 2024 at 06:04:23PM -0700, Bakul Shah via TUH= S wrote:
> > >> [1] This brings up a separate point: in a microkernel ev= en a simple
> > >> thing like "foo | bar" would require a third p= rocess - a "pipe
> > >> service", to buffer up the output of foo! You may h= ave reduced
> > >> the overhead of individual syscalls but you will have mo= re of
> > >> cross-domain calls!
> > >
> > > Do any micro kernels do address space to address space bcopy= ()?
> >
> > mmapping the same page in two processes won't be hard but now=
> > you have complicated cat (or some iolib)!
>
> I recall asking Linus if that could be done to save TLB entries, as in=
> multiple processes map a portion of their address space (at the same > virtual location) and then they all use the same TLB entries for that<= br> > part of their address space.=C2=A0 He said it couldn't be done bec= ause the
> process ID concept was hard wired into the TLB.=C2=A0 I don't know= if TLB
> tech has evolved such that a single process could have multiple "= process"
> IDs associated with it in the TLB.
>
> I wanted it because if you could share part of your address space with=
> another process, using the same TLB entries, then motivation for threa= ds
> could go away (I've never been a threads fan but I acknowledge why=
> you might need them).=C2=A0 I was channeling Rob's "If you th= ink you need
> threads, your processes are too fat".
>
> The idea of using processes instead of threads falls down when you
> consider TLB usage.=C2=A0 And TLB usage, when you care about performan= ce, is
> an issue.=C2=A0 I could craft you some realistic benchmarks, mirroring= real
> world work loads, that would kill the idea of replacing threads with > processes unless they shared TLB entries.=C2=A0 Think of a N-way threa= ded
> application, lots of address space used, that application uses all of = the
> TLB.=C2=A0 Now do that with N processes and your TLB is N times less e= ffective.
>
> This was a conversation decades ago so maybe TLB tech now has solved t= his.
> I doubt it, if this was a solved problem I think every OS would say sc= rew
> threads, just use processes and mmap().=C2=A0 The nice part of that mo= del
> is you can choose what parts of your address space you want to share.<= br> > That cuts out a HUGE swath of potential problems where another thread<= br> > can go poke in a part of your address space that you don't want po= ked.
>

I've never been a fan of the rfork()/clone() model. With the OS I'm=
working on, rather than using processes that share state as threads, a
process will more or less just be a collection of threads that share a
command line and get replaced on exec(). All of the state usually
associated with a process (e.g. file descriptor space, filesystem
namespace, virtual address space, memory allocations) will instead be
stored in separate container objects that can be shared between
threads. It will be possible to share any of these containers between
processes, or use different combinations between threads within a
process. This would allow more control over what gets shared between
threads/processes than rfork()/clone() because the state containers
will appear in the filesystem and be explicitly bound to threads
rather than being anonymous and only transferred on rfork()/clone().
Emulating rfork()/clone on top of this will be easy enough though.
--000000000000a002790618ca662a--