From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, HTML_MESSAGE,MAILING_LIST_MULTI,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 3414 invoked from network); 15 Dec 2023 16:25:30 -0000 Received: from minnie.tuhs.org (2600:3c01:e000:146::1) by inbox.vuxu.org with ESMTPUTF8; 15 Dec 2023 16:25:30 -0000 Received: from minnie.tuhs.org (localhost [IPv6:::1]) by minnie.tuhs.org (Postfix) with ESMTP id A1CAA43E93; Sat, 16 Dec 2023 02:25:24 +1000 (AEST) Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com [IPv6:2a00:1450:4864:20::635]) by minnie.tuhs.org (Postfix) with ESMTPS id 6CFB843E8E for ; Sat, 16 Dec 2023 02:25:17 +1000 (AEST) Received: by mail-ej1-x635.google.com with SMTP id a640c23a62f3a-a1e35c2807fso101796966b.3 for ; Fri, 15 Dec 2023 08:25:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1702657516; x=1703262316; darn=tuhs.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=ptnTGlVsvshjj9IxD5s9EiHgVqKdYcTuK32NNQMw0tI=; b=jwQtYCeCg2WyZlvRxgLlUSanDTU+dSUhMwexmtbKPgctgYN9we1jz3aqnL/VGHLsTR MthKM5riouHGV6SLO1zWl/dCJsNIDwmS0sZ4g5m4YqNmGfWZPCOEVn/xhJ0Iwds992nZ Bs2rSq/yCQPBMsM6jJDs7nxPZeN6JhwIQBo3LD/3GElYMEfaHH07sNxEB7N8eYXIcXaI xzoP6620SjAso0hzaXXuFNlRNCTH8yw8LQqOSNafEKlGWFBKzjaX/DKhyzGeudtv1EtQ c/ZCEazdgPISoQiEgcXkZyC4B68GyI/H3CR51VpqyqPfgn6UhMcrspX68N2xKVasrE4u yJ4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702657516; x=1703262316; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ptnTGlVsvshjj9IxD5s9EiHgVqKdYcTuK32NNQMw0tI=; b=wRRq701X+5Pw1Vw3Q789gdZDOVOtSIXsLtCUcnFZ6MdMk3YIbbgddnyD+BK2sQDGiB JLUoTXSubpNvaLqRUToexy1i5Nr5S9Uz/tnoshDz7a6Lvc0ULpNeQqZ+rLuGqSM3YFLA md1YJYF28N/KxXX15gNHjRG6lWUAfRCJzU3kjbQYIaQkU0kZryB+eXAt5L+tQQM41Ryg ynloA46Ah4UNlLBljng/WNzsEKwUghiz5lwaNb7GC7IPWPu7NhGy2+u3Jb49r/M7Yq+9 HHcJ0GHJR/qcETYS+oXWA4mmMAG6K4+6hkYW4OjIA30FPlzBStmi8wX47DWZHNcVGSya mI2w== X-Gm-Message-State: AOJu0YxKsOlsidMuzt2qhcVHZNkV/F/Bn93ZVSTIVu5fSjLPfd4WDuEj BoJbLUGCTj5N0qt2JRppwsaHdb0j6+fpwQK01E3DqVKE+xNbVMB6aJo= X-Google-Smtp-Source: AGHT+IGgaT3o3b17y5QOT5SwElt3/FRBP7VnfQceqAg97J/MOUEb8MX0XjG7sZIS5Ikv+dqrjFn5wlEAMFo2dkOq9yM= X-Received: by 2002:a17:906:188:b0:a1d:e5a:3034 with SMTP id 8-20020a170906018800b00a1d0e5a3034mr5558143ejb.31.1702657515674; Fri, 15 Dec 2023 08:25:15 -0800 (PST) MIME-Version: 1.0 References: <20231214214805.81B2618C08F@mercury.lcs.mit.edu> In-Reply-To: From: Warner Losh Date: Fri, 15 Dec 2023 09:25:04 -0700 Message-ID: To: Dan Cross Content-Type: multipart/alternative; boundary="0000000000006577c7060c8ed8dd" Message-ID-Hash: RMCAM3OOBP5DJL76OCFACDREQFQECFUY X-Message-ID-Hash: RMCAM3OOBP5DJL76OCFACDREQFQECFUY X-MailFrom: wlosh@bsdimp.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Noel Chiappa , coff@tuhs.org X-Mailman-Version: 3.3.6b1 Precedence: list Subject: [COFF] Re: Terminology query - 'system process'? List-Id: Computer Old Farts Forum Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --0000000000006577c7060c8ed8dd Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, Dec 15, 2023 at 7:21=E2=80=AFAM Dan Cross wrote: > On Thu, Dec 14, 2023 at 5:17=E2=80=AFPM Clem Cole wrote: > > I don't know of a standard name. We used to call the kernel processes > or kernel threads also. > > I've heard all combinations of (system|kernel) (thread|task|process), > all of which mean more or less the same thing: something the kernel > can schedule and run that doesn't have a userspace component, reusing > the basic concurrency primitives in the kernel for its own internal > purposes. I'm not sure I've heard "kernel daemon" before, but > intuitively I'd lump it into the same category unless I was told > otherwise (as Noel mentioned, of course Berkeley had the "pageout > daemon" which ran only in the kernel). > FreeBSD (and likely others) have extended this to allow kernel threads that we loosely call daemons as well. FreeBSD has APIs for creating a kernel-only threads and processes... > > For instance, in the original Masscomp EFS code, we had a handful of > processes that got forked after the pager using kernel code. Since the > basic UNIX read/write from the user space scheme is synchronous, the > premade pool of kernel processes was dispatched as needed when we listene= d > for asynchronous remote requests for I/O. This is similar to the fact tha= t > asynchronous devices from serial or network interfaces need a pool of > memory to stuff things into since you never know ahead of time when it wi= ll > come. > > =E1=90=A7 > > I remember reading a paper on the design of NFS (it may have been the > BSD paper) and there was a note about how the NFS server process ran > mostly in the kernel; user code created it, but pretty much all it did > was invoke a system call that implemented the server. That was kind of > neat. I recall discussions with the kernel people at Solbourne who were bringing up SunOS 4.0 on Solbourne hardware about this. nfsd was little more than an N way fork followed by the system call. It provided a process context to sleep in, which couldn't be created in the kernel at the time. I've not gone to the available SunOS sources to confirm this is what's going on. I'd thought, though, that this was the second nfsd implementation. The first one would decode the requests off the wire and schedule the I/O. It was only when there were issues with this approach that it moved into the kernel. This was mostly a context switch thing, but as more security measures were added to the system that root couldn't bypass, NFS needed to move into the kernel so it could bypass them. See getfh and similar system calls. I'm not sure how much of this was done in SunOS, I'm only familiar with the post 4.4BSD work... Warner > - Dan C. > > > On Thu, Dec 14, 2023 at 4:48=E2=80=AFPM Noel Chiappa > wrote: > >> So Lars Brinkhoff and I were chatting about daemons: > >> > >> https://gunkies.org/wiki/Talk:Daemon > >> > >> and I pointed out that in addition to 'standard' daemons (e.g. the > printer > >> spooler daemon, email daemon, etc, etc) there are some other things > that are > >> daemon-like, but are fundamentally different in major ways (explained > later > >> below). I dubbed them 'system processes', but I'm wondering if ayone > knows if > >> there is a standard term for them? (Or, failing that, if they have a > >> suggestion for a better name?) > >> > >> > >> Early UNIX is one of the first systems to have one (process 0, the > "scheduling (swapping) > >> process"), but the CACM "The UNIX Time-Sharing System" paper: > >> > >> https://people.eecs.berkeley.edu/~brewer/cs262/unix.pdf > >> > >> doesn't even mention it, so no guidance there. Berkeley UNIX also has > one, > >> mentioned in "Design and Implementation of the Berkeley Virtual Memory > >> Extensions to the UNIX Operating System": > >> > >> http://roguelife.org/~fujita/COOKIES/HISTORY/3BSD/design.pdf > >> > >> where it is called the "pageout daemon".("During system initialization= , > just > >> before the init process is created, the bootstrapping code creates > process 2 > >> which is known as the pageout daemon. It is this process that .. > writ[es] > >> back modified pages. The process leaves its normal dormant state upon > being > >> waken up due to the memory free list size dropping below an upper > >> threshold.") However, I think there are good reasons to dis-favour the > term > >> 'daemon' for them. > >> > >> > >> For one thing, typical daemons look (to the kernel) just like 'normal' > >> processes: their object code is kept in a file, and is loaded into the > >> daemon's process when it starts, using the same mechanism that 'normal= ' > >> processes use for loading their code; daemons are often started long > after > >> the kernel itself is started, and there is usually not a special > mechanism in > >> the kernel to start daemons (on early UNIXes, /etc/rc is run by the > 'init' > >> process, not the kernel); daemons interact with the kernel through > system > >> calls, just like 'ordinary' processes; the daemon's process runs in > 'user' > >> CPU mode (using the same standard memory mapping mechanisms, just like > >> blah-blah). > >> > >> 'System processes' do none of these things: their object code is linke= d > into > >> the monolithic kernel, and is thus loaded by the bootstrap; the kernel > >> contains special provision for starting the system process, which star= t > as > >> the kernel is starting; they don't do system calls, just call kernel > routines > >> directly; they run in kernel mode, using the same memory mapping as th= e > >> kernel itself; etc, etc. > >> > >> Another important point is that system processes are highly intertwine= d > with > >> the operation of the kernel; without the system process(es) operating > >> correctly, the operation of the system will quickly grind to a halt. > The loss > >> of ordinary' daemons is usually not fatal; if the email daemon dies, t= he > >> system will keep running indefinitely. Not so, for the swapping > process, or > >> the pageout daemon > >> > >> > >> Anyway, is there a standard term for these things? If not, a better > name than > >> 'system process'? > >> > >> Noel > --0000000000006577c7060c8ed8dd Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Fri, Dec 15, 2023 at 7:21=E2=80=AF= AM Dan Cross <crossd@gmail.com&g= t; wrote:
On Thu= , Dec 14, 2023 at 5:17=E2=80=AFPM Clem Cole <clemc@ccc.com> wrote:
> I don't know of a standard name.=C2=A0 =C2=A0We used to call the k= ernel processes or kernel threads also.

I've heard all combinations of (system|kernel) (thread|task|process), all of which mean more or less the same thing: something the kernel
can schedule and run that doesn't have a userspace component, reusing the basic concurrency primitives in the kernel for its own internal
purposes. I'm not sure I've heard "kernel daemon" before,= but
intuitively I'd lump it into the same category unless I was told
otherwise (as Noel mentioned, of course Berkeley had the "pageout
daemon" which ran only in the kernel).

=
FreeBSD (and likely=C2=A0others) have extended this to allow kernel
threads that we loosely call daemons as well. FreeBSD has APIs
for creating a kernel-only threads and processes...
=C2=A0<= /div>
> For instance, in the original Masscomp EFS code, we had a handful of p= rocesses that got forked after the pager using kernel code.=C2=A0 Since the= basic UNIX read/write from the user space scheme is synchronous, the prema= de pool of kernel processes was dispatched as needed when we listened for a= synchronous remote requests for I/O. This is similar to the fact that async= hronous devices from serial or network interfaces need a pool of memory to = stuff things into since you never know ahead of time when it will come.
> =E1=90=A7

I remember reading a paper on the design of NFS (it may have been the
BSD paper) and there was a note about how the NFS server process ran
mostly in the kernel; user code created it, but pretty much all it did
was invoke a system call that implemented the server. That was kind of
neat.

I recall discussions with the kernel = people at Solbourne who were bringing up
SunOS 4.0 on Solbourne h= ardware about this. nfsd was little more than an N way
fork follo= wed by the system call. It provided a process context to sleep in, which
couldn't be created in the kernel at the time. I've not gon= e to the available SunOS
sources to confirm this is what's go= ing on.

I'd thought, though, that this was the= second nfsd implementation. The first one
would decode the reque= sts off the wire and schedule the I/O. It was only when
there wer= e issues with this approach that it moved into the kernel. This was mostly<= /div>
a context switch thing, but as more security measures were added = to the system
that root couldn't bypass, NFS needed to move i= nto the kernel so it could bypass
them. See getfh and similar sys= tem calls. I'm not sure how much of this was
done in SunOS, I= 'm only familiar with the post 4.4BSD work...

= Warner
=C2=A0
=C2=A0 =C2=A0 =C2=A0 =C2=A0 - Dan C.

> On Thu, Dec 14, 2023 at 4:48=E2=80=AFPM Noel Chiappa <jnc@mercury.lcs.mit.edu= > wrote:
>> So Lars Brinkhoff and I were chatting about daemons:
>>
>>=C2=A0 =C2=A0https://gunkies.org/wiki/Talk:Daemon<= br> >>
>> and I pointed out that in addition to 'standard' daemons (= e.g. the printer
>> spooler daemon, email daemon, etc, etc) there are some other thing= s that are
>> daemon-like, but are fundamentally different in major ways (explai= ned later
>> below). I dubbed them 'system processes', but I'm wond= ering if ayone knows if
>> there is a standard term for them? (Or, failing that, if they have= a
>> suggestion for a better name?)
>>
>>
>> Early UNIX is one of the first systems to have one (process 0, the= "scheduling (swapping)
>> process"), but the CACM "The UNIX Time-Sharing System&qu= ot; paper:
>>
>>=C2=A0 =C2=A0https://people.eecs.berk= eley.edu/~brewer/cs262/unix.pdf
>>
>> doesn't even mention it, so no guidance there. Berkeley UNIX a= lso has one,
>> mentioned in "Design and Implementation of the Berkeley Virtu= al Memory
>> Extensions to the UNIX Operating System":
>>
>>=C2=A0 =C2=A0http://roguelife.or= g/~fujita/COOKIES/HISTORY/3BSD/design.pdf
>>
>> where it is called the "pageout daemon".("During sy= stem initialization, just
>> before the init process is created, the bootstrapping code creates= process 2
>> which is known as the pageout daemon. It is this process that .. w= rit[es]
>> back modified pages. The process leaves its normal dormant state u= pon being
>> waken up due to the memory free list size dropping below an upper<= br> >> threshold.") However, I think there are good reasons to dis-f= avour the term
>> 'daemon' for them.
>>
>>
>> For one thing, typical daemons look (to the kernel) just like '= ;normal'
>> processes: their object code is kept in a file, and is loaded into= the
>> daemon's process when it starts, using the same mechanism that= 'normal'
>> processes use for loading their code; daemons are often started lo= ng after
>> the kernel itself is started, and there is usually not a special m= echanism in
>> the kernel to start daemons (on early UNIXes, /etc/rc is run by th= e 'init'
>> process, not the kernel); daemons interact with the kernel through= system
>> calls, just like 'ordinary' processes; the daemon's pr= ocess runs in 'user'
>> CPU mode (using the same standard memory mapping mechanisms, just = like
>> blah-blah).
>>
>> 'System processes' do none of these things: their object c= ode is linked into
>> the monolithic kernel, and is thus loaded by the bootstrap; the ke= rnel
>> contains special provision for starting the system process, which = start as
>> the kernel is starting; they don't do system calls, just call = kernel routines
>> directly; they run in kernel mode, using the same memory mapping a= s the
>> kernel itself; etc, etc.
>>
>> Another important point is that system processes are highly intert= wined with
>> the operation of the kernel; without the system process(es) operat= ing
>> correctly, the operation of the system will quickly grind to a hal= t. The loss
>> of ordinary' daemons is usually not fatal; if the email daemon= dies, the
>> system will keep running indefinitely. Not so, for the swapping pr= ocess, or
>> the pageout daemon
>>
>>
>> Anyway, is there a standard term for these things? If not, a bette= r name than
>> 'system process'?
>>
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Noel
--0000000000006577c7060c8ed8dd--