From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, HTML_MESSAGE,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 2224 invoked from network); 25 Jul 2021 12:55:52 -0000 Received: from minnie.tuhs.org (45.79.103.53) by inbox.vuxu.org with ESMTPUTF8; 25 Jul 2021 12:55:52 -0000 Received: by minnie.tuhs.org (Postfix, from userid 112) id AA9879CABC; Sun, 25 Jul 2021 22:55:48 +1000 (AEST) Received: from minnie.tuhs.org (localhost [127.0.0.1]) by minnie.tuhs.org (Postfix) with ESMTP id 2EB419CAA5; Sun, 25 Jul 2021 22:54:40 +1000 (AEST) Authentication-Results: minnie.tuhs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=planet.nl header.i=@planet.nl header.b="x5ge1DYh"; dkim-atps=neutral Received: by minnie.tuhs.org (Postfix, from userid 112) id 2405D9CAA5; Sun, 25 Jul 2021 22:54:32 +1000 (AEST) Received: from cpsmtpb-ews02.kpnxchange.com (cpsmtpb-ews02.kpnxchange.com [213.75.39.5]) by minnie.tuhs.org (Postfix) with ESMTP id 33E1D9CAA4 for ; Sun, 25 Jul 2021 22:54:24 +1000 (AEST) Received: from cpsps-ews28.kpnxchange.com ([10.94.84.194]) by cpsmtpb-ews02.kpnxchange.com with Microsoft SMTPSVC(8.5.9600.16384); Sun, 25 Jul 2021 14:54:14 +0200 X-Brand: 7abm2Q== X-KPN-SpamVerdict: e1=0;e2=0;e3=0;e4=;e6=(e1=10;e3=10;e2=11;e4=10;e6=1 0);EVW:White;BM:NotScanned;FinalVerdict:Clean X-CMAE-Analysis: v=2.4 cv=N6hKq0xB c=1 sm=1 tr=0 ts=60fd5ef6 cx=a_idp_e a=WB5lYbMG1jvHJ1f8o08CVQ==:117 a=soxbC+bCkqwFbqeW/W/r+Q==:17 a=x1i13A_MHe4A:10 a=e_q4qTt1xDgA:10 a=KtQxTXL8aguQWbg8txwA:9 a=QEXdDO2ut3YA:10 a=_OpWXrcsl4eECPA8:21 a=_W_S_7VecoQA:10 X-CM-AcctID: kpn@feedback.cloudmark.com Received: from smtp.kpnmail.nl ([195.121.84.14]) by cpsps-ews28.kpnxchange.com over TLS secured channel with Microsoft SMTPSVC(8.5.9600.16384); Sun, 25 Jul 2021 14:54:14 +0200 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=planet.nl; s=planet01; h=to:date:message-id:subject:mime-version:content-type:from; bh=7uLKsfC3Hd5kbmThstZHbdu0Ub1reytN9jw8xRGBfRc=; b=x5ge1DYhaF9ZjMo8CoedvWyOEpZwYeTUWbhRuKJdH7URrc2TIvQo3iIdwBCAtjqqMZwft2XFuaYlh XduDFOehOR0ZhecSsYRE3B4U/EO2+PND5d5hligh7DPsORFoinVyIzB5EkeUBqcDRWzxtUFTmYKepA hxrB9tByI1lm0dUQ= X-KPN-MID: 33|7PZO2+5xk0igwqcqRlJ3rwdtYOhOaAthkU6RlIlYyibE6RPQZkdkrcH7wZ1nJqm +6hQQB/Pvlf4YEt0OYtJNCygCtGa4oUs5ZwtqbqrRKtA= X-KPN-VerifiedSender: Yes X-CMASSUN: 33|PjMk6hJq+L6UApMXQwCG3lS0/+o2bgpphYedAulPsJyKi+NrEOFu5kG4tR12uo+ wv8idETGZgPYE8MDmuZ5z+w== X-Originating-IP: 80.101.112.122 Received: from mba1.fritz.box (sqlite.xs4all.nl [80.101.112.122]) by smtp.kpnmail.nl (Halon) with ESMTPSA id 69cb6544-ed47-11eb-96c1-00505699d6e5; Sun, 25 Jul 2021 14:54:14 +0200 (CEST) From: Paul Ruizendaal Content-Type: multipart/alternative; boundary="Apple-Mail=_9A502110-2EE5-4871-8485-CC65FD1E6AA5" Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.7\)) Message-Id: <28452B35-1321-4EC2-8F83-07E9D78BEC8B@planet.nl> Date: Sun, 25 Jul 2021 14:54:13 +0200 To: TUHS main list X-Mailer: Apple Mail (2.3445.9.7) X-OriginalArrivalTime: 25 Jul 2021 12:54:14.0173 (UTC) FILETIME=[2BBC84D0:01D78154] X-RcptDomain: minnie.tuhs.org Subject: [TUHS] Wy 32V R3 was fast X-BeenThere: tuhs@minnie.tuhs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: The Unix Heritage Society mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: tuhs-bounces@minnie.tuhs.org Sender: "TUHS" --Apple-Mail=_9A502110-2EE5-4871-8485-CC65FD1E6AA5 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > TUHS list (Rob Pike, Aug 2019) >=20 > I think it was slightly later. I joined mid-1980 and VAXes to replace = the > 11/70 were being discussed but had not arrived. We needed to convert a = lab > into a VAX machine room and decide between BSD and Reiser, all of = which > happened in the second half of 1980. >=20 > Reiser Unix got demand paging a little later, and it was spectacularly > fast. I remember being gobsmacked when I saw a demo in early 1981. >=20 > Dead ends everywhere. I think I have figured out why 32V R3 was so fast (assuming my current = understanding of how 32V R3 must have worked is correct). Its VM subsystem tags each memory frame with its disk mirror location, = be it in swap or in the file system. A page can quickly be found as they = are hashed on device and block no. This is true both for pages in the = working set and pages on the 2nd chance list. Effectively, most core is = a disk cache. In a unified buffer design, the buffer code would first look for an = existing buffer header for the requested disk block, as in V7. If not = found, it would check the page frame list for that block and if found it = would connect the frame to an empty buffer header, increment its use = count and move it to the working set. If not found there either, it = would be loaded from disk as per usual. When a buffer is released, the = frame use count would be decremented and if zero the page frame would be = put back on the 2nd chance list and the buffer header would be marked = empty. With this approach, up to 4MB of the disk could be cached in RAM. Early in 1981 most binaries and files were a few dozen KB in size. All = of the shell, editor, compiler tool chain, library files, intermediate = files, etc. would have fitted in RAM all at once. In a developer focused = demo and once memory was primed, the system would effectively run from = RAM, barely hitting the disk, even with tens of concurrent logins. Also = something like =E2=80=9Cls -l /bin=E2=80=9D would have been much faster = on its second run. It puts a comment from JFR in a clearer context: Strict LRU on 8,192 pages, plus Copy-On-Write, made the second reference = to a page "blindingly fast". So far I read this in context of the paging algorithm, and then it is = hard to understand (is LRU really that much better than NRU?). In the = context of a unified buffer and disk pages, it makes a lot more sense. = Even the CoW part: as the (clean) data segment of executables would = still be in core, they could start without reloading from disk - CoW = would create copies as needed. =3D=3D=3D The interesting question now is: if this buffer unification was so = impressive, why was it abandoned in SVr2-vax? I can think of 3 reasons: 1. Maybe there was a subtle bug that was hard to diagnose. =E2=80=9CResear= ch" opting for the BSD memory system =E2=80=9Cas it did not want to do = the maintenance=E2=80=9D suggests that there may have been lingering = issues. 1b. A variation of this: JFR mentioned that his implementation of = unified buffers broke conceptual layering. USG do not strike me as = purists, but maybe they thought the code was too messy to maintain. 2. Maybe there was an unintended semantic issue (e.g. you can lock a = buffer, but not a mmap =E2=80=98ed page). 3. Maybe it was hard to come up with a good sync() policy, making = database work risky (and system crashes more devastating to the file = system). JFR mentioned that he did the design and implementation for 32V R3 in = about 3 months, with 3 more months for bug fixing and polishing. That is = not a lot of time for such a big and complex kernel mod (for its time). --Apple-Mail=_9A502110-2EE5-4871-8485-CC65FD1E6AA5 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 
TUHS list =
(Rob Pike, Aug 2019)

I think it was slightly later. I joined mid-1980 and VAXes to replace =
the
11/70 were being discussed but had not arrived. We needed to convert a =
lab
into a VAX machine room and decide between BSD and Reiser, all of which
happened in the second half of 1980.

Reiser Unix got demand paging a little later, and it was spectacularly
fast. I remember being gobsmacked when I saw a demo in early 1981.

Dead ends everywhere.

I think I have figured out why 32V R3 = was so fast (assuming my current understanding of how 32V R3 must have = worked is correct).

Its VM subsystem tags each memory frame with its disk mirror = location, be it in swap or in the file system. A page can quickly be = found as they are hashed on device and block no. This is true both for = pages in the working set and pages on the 2nd chance list. Effectively, = most core is a disk cache.

In a unified buffer design, the buffer code would first look = for an existing buffer header for the requested disk block, as in V7. If = not found, it would check the page frame list for that block and if = found it would connect the frame to an empty buffer header, increment = its use count and move it to the working set. If not found there either, = it would be loaded from disk as per usual. When a buffer is released, = the frame use count would be decremented and if zero the page frame = would be put back on the 2nd chance list and the buffer header would be = marked empty. With this approach, up to 4MB of the disk could be cached = in RAM.

Early = in 1981 most binaries and files were a few dozen KB in size. All of the = shell, editor, compiler tool chain, library files, intermediate files, = etc. would have fitted in RAM all at once. In a developer focused demo = and once memory was primed, the system would effectively run from RAM, = barely hitting the disk, even with tens of concurrent logins. Also = something like =E2=80=9Cls -l /bin=E2=80=9D would have been much faster = on its second run.

It puts a comment from JFR in a clearer context:

<quote>
Strict LRU on 8,192 pages, plus Copy-On-Write, made the second reference to a page "blindingly fast".
<unquote>

So far I read this in = context of the paging algorithm, and then it is hard to understand (is = LRU really that much better than NRU?). In the context of a unified = buffer and disk pages, it makes a lot more sense. Even the CoW part: as = the (clean) data segment of executables would still be in core, they = could start without reloading from disk - CoW would create copies as = needed.

=3D=3D=3D

The interesting question now is: if this buffer unification = was so impressive, why was it abandoned in SVr2-vax? I can think of 3 = reasons:

1. = Maybe there was a subtle bug that was hard to diagnose. =E2=80=9CResearch"= opting for the BSD memory system =E2=80=9Cas it did not want to do the = maintenance=E2=80=9D suggests that there may have been lingering = issues.

1b. A = variation of this: JFR mentioned that his implementation of unified = buffers broke conceptual layering. USG do not strike me as purists, but = maybe they thought the code was too messy to maintain.

2. Maybe there was an = unintended semantic issue (e.g. you can lock a buffer, but not a mmap = =E2=80=98ed page).

3. Maybe it was hard to come up with a good sync() policy, = making database work risky (and system crashes more devastating to the = file system).

JFR mentioned that he did the design and implementation for = 32V R3 in about 3 months, with 3 more months for bug fixing and = polishing. That is not a lot of time for such a big and complex kernel = mod (for its time).




= --Apple-Mail=_9A502110-2EE5-4871-8485-CC65FD1E6AA5--