From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 17964 invoked from network); 3 Jun 2021 23:22:46 -0000 Received: from minnie.tuhs.org (45.79.103.53) by inbox.vuxu.org with ESMTPUTF8; 3 Jun 2021 23:22:46 -0000 Received: by minnie.tuhs.org (Postfix, from userid 112) id 1F1329C885; Fri, 4 Jun 2021 09:22:44 +1000 (AEST) Received: from minnie.tuhs.org (localhost [127.0.0.1]) by minnie.tuhs.org (Postfix) with ESMTP id 224229C82B; Fri, 4 Jun 2021 09:22:06 +1000 (AEST) Authentication-Results: minnie.tuhs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=planet.nl header.i=@planet.nl header.b="OdpLGSuo"; dkim-atps=neutral Received: by minnie.tuhs.org (Postfix, from userid 112) id 94A499C82B; Fri, 4 Jun 2021 09:21:57 +1000 (AEST) Received: from cpsmtpb-ews08.kpnxchange.com (unknown [213.75.39.13]) by minnie.tuhs.org (Postfix) with ESMTP id F19599C829 for ; Fri, 4 Jun 2021 09:21:46 +1000 (AEST) Received: from cpsps-ews01.kpnxchange.com ([10.94.84.168]) by cpsmtpb-ews08.kpnxchange.com with Microsoft SMTPSVC(8.5.9600.16384); Fri, 4 Jun 2021 01:21:26 +0200 X-Brand: 7abm2Q== X-KPN-SpamVerdict: e1=0;e2=0;e3=0;e4=;e6=(e1=10;e3=10;e2=11;e4=10;e6=1 0);EVW:White;BM:NotScanned;FinalVerdict:Clean X-CMAE-Analysis: v=2.4 cv=L+d+/8f8 c=1 sm=1 tr=0 ts=60b963f6 cx=a_idp_e a=dZ5u/0G9QtS9WKCcNUBnHQ==:117 a=soxbC+bCkqwFbqeW/W/r+Q==:17 a=x1i13A_MHe4A:10 a=IkcTkHD0fZMA:10 a=r6YtysWOX24A:10 a=C3I3ZF1iAAAA:8 a=OensOVJAAAAA:8 a=AoeSMSUmAAAA:8 a=J918yP9rxylfLjCW-zoA:9 a=QEXdDO2ut3YA:10 a=IA4qIw_M-6cA:10 a=7YsK12cni5sJQXfPmEYQ:22 a=2UY7SMgi64q-0UtCmZ5F:22 X-CM-AcctID: kpn@feedback.cloudmark.com Received: from smtp.kpnmail.nl ([195.121.84.13]) by cpsps-ews01.kpnxchange.com over TLS secured channel with Microsoft SMTPSVC(8.5.9600.16384); Fri, 4 Jun 2021 01:21:26 +0200 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=planet.nl; s=planet01; h=to:date:message-id:subject:mime-version:content-type:from; bh=f8KyYb24mGugaLRvae8gmUfEsMF2Lv0t9GX+rFtrPMA=; b=OdpLGSuoneuqr6+0DczXr0k9PtQTVIQ7HcmlCIs/NO68BAbiDbDlqVrFQzY2jxm21QnJgck97+U0a TjOVFAnDCe6I+k5kDT8EPLFhO52GkLNTHmn6G/q3T/HSuEAU5PKMsMm/Z9Ari0oDCvZb52OHPsRi4R CJIDwg2oeb0FqwRk= X-KPN-VerifiedSender: Yes X-CMASSUN: 33|aYH7tjqNXb5p8dgJa2aWeSILRfNpVYBxbxIouwfyW362jUzXOHaoouaAUjcYonr bvr1DMfXWvQYXiducSCf1Pw== X-Originating-IP: 80.101.112.122 Received: from mba2.fritz.box (sqlite.xs4all.nl [80.101.112.122]) by smtp.kpnmail.nl (Halon) with ESMTPSA id 6aafffb7-c4c2-11eb-8208-005056998788; Fri, 04 Jun 2021 01:21:26 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.4\)) Message-Id: Date: Fri, 4 Jun 2021 01:21:25 +0200 To: TUHS main list X-Mailer: Apple Mail (2.3608.120.23.2.4) X-OriginalArrivalTime: 03 Jun 2021 23:21:26.0403 (UTC) FILETIME=[2CD01D30:01D758CF] X-RcptDomain: minnie.tuhs.org Subject: [TUHS] 32V memory management: not quite V7 style swapping X-BeenThere: tuhs@minnie.tuhs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: The Unix Heritage Society mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Paul Ruizendaal via TUHS Reply-To: Paul Ruizendaal Errors-To: tuhs-bounces@minnie.tuhs.org Sender: "TUHS" Received wisdom is that 32V used V7 style swapping for memory = management. Over the past days I=E2=80=99ve come to the conclusion that = this is a very partial truth and only holds true for 32V as it existed = in the first half of 1978. In the second half of =E2=80=9978 it = implemented something that is referred to as =E2=80=9Cscatter loading=E2=80= =9D which is an interesting halfway house on the road to demand paging. = It was also used in the VAX version of Sys III (and presumably in SysV = R1 and early R2). In the 32V report from July 1978 it says: "Like the UNIX system for the PDP-11, the current implementation for the = VAX-11/780 maintains each process in contiguous physical memory and = swaps processes to disk when there is not enough physical memory to = contain them all. Reducing external memory fragmentation to zero by = utilizing the VAX- 11/780 memory mapping hardware for scatter loading is = high on the list of things to do in the second implementation pass. To = simplify kernel memory allocation, the size of the user-segment memory = map is an assembly parameter which currently allows three pages of page = table or 192K bytes total for text, data, and stack.=E2=80=9D = (https://www.bell-labs.com/usr/dmr/www/otherports/32v.pdf). It turns out that scatter loading was added in the next months, and it = was this version that was used as the basis for 3BSD and SysIII. Babaoglu & Joy write: "Except for the machine-dependent sections of code, UNIX for the VAX was = quite similar to that for the PDP-11 which has a 16-bit address space = and no paging hardware. It made no use of the memory-management hardware = available on the VAX aside from simulating the PDP-11 segment registers = with VAX page table entries. The main-memory management schemes employed = by this first version of the system were identical to their PDP-11 = counterparts -- processes were allocated contiguous blocks of real = memory on a first-fit basis and were swapped in their entirety. A = subsequent version of the system was capable of loading processes into = noncontiguous real memory locations, called scatter loading, and was = able to swap only portions of a process, called partial swapping, as = deemed necessary by the memory contention. This would become the basis = for the paging system development discussed in this paper.=E2=80=9D = (https://www.researchgate.net/publication/247924813_Converting_a_swap-base= d_system_to_do_paging_in_an_architecture_lacking_page-referenced_bits) The 32V code on the TUHS website (e.g. here = https://www.tuhs.org/cgi-bin/utree.pl?file=3D32V) is actually this later = scatter loading code, and not the early 1978 code that used V7 style = memory management. The 32-bit Sys III code is closely related (see = https://www.tuhs.org/cgi-bin/utree.pl?file=3DSysIII/usr/src/uts/vax). =3D=3D=3D My current understanding of how scatter loading worked (based on a brief = code review) is as follows: (Note that on the VAX pages/frames are 512 bytes and the page list is = essentially single level; page lists grow quickly. It is also unusual in = the sense that user page table entries point to kernel virtual memory, = but kernel page table entries point to physical memory). - Each process keeps a list of pages in its u-area (a page table = prototype, if you will). This list is fixed size and allows up to 512KB = per process in 32V and ~2.5MB per process in Sys III (i.e up to 1024 = resp. 5120 pages). - The kernel keeps a bitmap of free/used frames in physical memory. - When a process loads/grows, the bitmap is scanned for free frames, = these are marked as in-use, and added to the u-area list. If there are = not enough free pages a process is selected for swapping out. Swapping = out is gradual, in 8KB chunks in 32V and 32KB chunks in SysIII. When a = process shrinks or completes, its pages are added back to the bitmap. - When a partially swapped out process needs to run, the swapped out = part is loaded back similar to the above. Partial swap-outs truncate the = process, so everything above the remaining size needs to re-load. - The user process page table is not swapped, but recreated from the = u-area data instead. - When switching between user processes, the kernel needs to update 16 = (32V) or 40 (SysIII) kernel page table entries to update the user memory = map. Scatter loading and partial swapping seem to be a major improvement over = V7 style swapping, although it of course falls short of demand paging. = So far I have not seen bits of code that suggest =E2=80=98lazy = loading=E2=80=99 or copy-on-write functionality in 32V or Sys III, but = these things would not seem impossible to do in this memory management = scheme. In short, the view that =E2=80=9C32V used V7 style swapping=E2=80=9D = seems to be an oversimplification.