* Re: [TUHS] Memory management in Dennis Ritchie's C Compiler @ 2020-08-17 19:27 Noel Chiappa 2020-08-17 19:30 ` Larry McVoy 2020-08-18 0:52 ` Rob Gingell 0 siblings, 2 replies; 7+ messages in thread From: Noel Chiappa @ 2020-08-17 19:27 UTC (permalink / raw) To: tuhs; +Cc: jnc > From: Jim Geist > When did mmap(2) come about? Pretty sure it's a Berserkleyism. I think it came in with the VM stuff that DARPA mandated for VAX Unix (for the research project they funded). Noel ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [TUHS] Memory management in Dennis Ritchie's C Compiler 2020-08-17 19:27 [TUHS] Memory management in Dennis Ritchie's C Compiler Noel Chiappa @ 2020-08-17 19:30 ` Larry McVoy 2020-08-17 19:44 ` Dan Halbert 2020-08-17 19:50 ` Paul Winalski 2020-08-18 0:52 ` Rob Gingell 1 sibling, 2 replies; 7+ messages in thread From: Larry McVoy @ 2020-08-17 19:30 UTC (permalink / raw) To: Noel Chiappa; +Cc: tuhs On Mon, Aug 17, 2020 at 03:27:15PM -0400, Noel Chiappa wrote: > > From: Jim Geist > > > When did mmap(2) come about? > > Pretty sure it's a Berserkleyism. I think it came in with the VM stuff that > DARPA mandated for VAX Unix (for the research project they funded). Bill Joy imagined it, the prototype is in one the 4.x BSD releases. Sun (Joe Moran) actually implemented it first in any Unix variant. It's possible the concept existed in some other OS but I'm not aware of it. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [TUHS] Memory management in Dennis Ritchie's C Compiler 2020-08-17 19:30 ` Larry McVoy @ 2020-08-17 19:44 ` Dan Halbert 2020-08-17 19:50 ` Paul Winalski 1 sibling, 0 replies; 7+ messages in thread From: Dan Halbert @ 2020-08-17 19:44 UTC (permalink / raw) To: tuhs On 8/17/20 3:30 PM, Larry McVoy wrote: > On Mon, Aug 17, 2020 at 03:27:15PM -0400, Noel Chiappa wrote: >> > From: Jim Geist >> >> > When did mmap(2) come about? >> >> Pretty sure it's a Berserkleyism. I think it came in with the VM stuff that >> DARPA mandated for VAX Unix (for the research project they funded). > Bill Joy imagined it, the prototype is in one the 4.x BSD releases. > Sun (Joe Moran) actually implemented it first in any Unix variant. > It's possible the concept existed in some other OS but I'm not aware > of it. I have a clear memory of having a discussion with Bill Joy about the idea of vread(2) and vwrite(2) when we were both grad students at Berkeley. I remember we were eating sausages from Top Dog and sitting outside Etcheverry Hall on or near the grassy plaza. I think vfork may have already existed, and we were talking about adding some kind of memory-mapped file I/O. I think I suggested the actual names, as a parallel to vfork. Some of my thinking might have come from my experience with TENEX while working summers at BBN. I was just a sounding board; I wasn't implementing any of this. vread and vwrite were in BSD4.1, but dropped in 4.2 (so my Googling says). I think it became clear later that mmap was an easier concept than the initial vread and vwrite ideas. Dan H. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [TUHS] Memory management in Dennis Ritchie's C Compiler 2020-08-17 19:30 ` Larry McVoy 2020-08-17 19:44 ` Dan Halbert @ 2020-08-17 19:50 ` Paul Winalski 2020-08-17 22:05 ` Arthur Krewat 1 sibling, 1 reply; 7+ messages in thread From: Paul Winalski @ 2020-08-17 19:50 UTC (permalink / raw) To: Larry McVoy; +Cc: tuhs, Noel Chiappa On 8/17/20, Larry McVoy <lm@mcvoy.com> wrote: > On Mon, Aug 17, 2020 at 03:27:15PM -0400, Noel Chiappa wrote: [regarding mmap(2)] > > Bill Joy imagined it, the prototype is in one the 4.x BSD releases. > Sun (Joe Moran) actually implemented it first in any Unix variant. > It's possible the concept existed in some other OS but I'm not aware > of it. > VAX/VMS had the equivalent of mmap(2) back in 1978. You can specify a range of contiguous pages in virtual memory and associate that with a (page-aligned) range of blocks in a file. The blocks in the file act as backing store for the virtual memory. VMS also has a system call $CRETVA (create virtual address space) that lets you associate a VA range using the system page file as backing store. The VMS image activator (runtime loader in Unix-speak) used these primitives to load program images into virtual memory. More than one process can map the same region of a file. This is how sharing of read-only program segments such as .text is implemented. I think Burroughs OSes had this concept even before VMS. There is also $EXPREG (expand address region), which is more or less equivalent to sbrk(). -Paul W. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [TUHS] Memory management in Dennis Ritchie's C Compiler 2020-08-17 19:50 ` Paul Winalski @ 2020-08-17 22:05 ` Arthur Krewat 0 siblings, 0 replies; 7+ messages in thread From: Arthur Krewat @ 2020-08-17 22:05 UTC (permalink / raw) To: tuhs On 8/17/2020 3:50 PM, Paul Winalski wrote: > VAX/VMS had the equivalent of mmap(2) back in 1978. You can specify a > range of contiguous pages in virtual memory and associate that with a > (page-aligned) range of blocks in a file. The blocks in the file act > as backing store for the virtual memory. I, and my boss, took advantage of that while converting his database software from TOPS-10 to VMS. His TOPS-10 implementation used various ways to optimize I/O from/to disk. On VMS, our benchmarks showed that as long as we understood the paging size, performance was about the same. RP06 vs. I think something bigger. Maybe RP07. But even still, it proved the operating system virtual memory system was up to the task. I seem to remember there was a way to sync a page after you wrote it just to make sure it was committed to disk. It worked so well that I also seem to recall that we used some of that file space as heap. art k. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [TUHS] Memory management in Dennis Ritchie's C Compiler 2020-08-17 19:27 [TUHS] Memory management in Dennis Ritchie's C Compiler Noel Chiappa 2020-08-17 19:30 ` Larry McVoy @ 2020-08-18 0:52 ` Rob Gingell 2021-01-29 23:52 ` [TUHS] mmap() (was: Memory management in Dennis Ritchie's C Compiler) Greg A. Woods 1 sibling, 1 reply; 7+ messages in thread From: Rob Gingell @ 2020-08-18 0:52 UTC (permalink / raw) To: tuhs On 8/17/2020 12:27 PM, Noel Chiappa wrote: > > From: Jim Geist > > > When did mmap(2) come about? > > Pretty sure it's a Berserkleyism. I think it came in with the VM stuff that > DARPA mandated for VAX Unix (for the research project they funded). [What follows verges on being COFF-ish]. mmap() is descended from TENEX's PMAP, which is a simplified descendant of the file/process memory integration provided by Multics. (Other antecedents I leave to more informed genealogists than I.) A Multics process had an address space made up of segments (files) which were paged (vs. swapped). Segments were what programs dealt with, paging was done to manage the physical resources of the system. TENEX simplified everything to pages, address spaces were composed by mapping pages of the address space to files. Programs operated on pages, which were also what the system used to manage memory. Files were described by page tables, address spaces were described by page tables, lots of symmetry around things of size 512 (9 bits) (page size, page table size, page-tables of page tables for long files). File size and file system system capacities were announced in pages. Very harmonious, but uniquely so. Multics was most definitely not portable due to its unique hardware constraints (Intel's 432 being an exception). TENEX only required paging support but was not portable from PDP-10s which were not keeping up with the world's movement to computers that were increasingly 4-bits shy of a word (i.e., not 36 bits). Many more considerations apply but to keep it only verging on COFF-ish will stop there -- going deeper would be best done in person with tankards of ale. TENEX systems were ARPA's workhorse systems in the 1970s, supporting numerous research programs and sites. The file/process memory integration was a valued capability among the community (though, honestly, it was one of these things everyone said should exist though few actually used directly). Even though DEC provided commercial support by adopting TENEX as TOPS-20 and selling 1000s of them (vs. the 10s of them made via BBN + PARC's MAXC), as the 70's progressed it was apparent that things were going to shift elsewhere. DEC was hesitant about further 36-bit development and UNIX's arrival provided an intellectual migration path for the community that commercial alternatives didn't. If only UNIX could just accumulate some of the capabilities expected in the ARPA world, like file/process memory integration (ironically, one of the Multics complexities from which UNIX had departed.) mmap() is a simplified, portable form of PMAP. It was written into the specifications for ARPA via Bill Joy as Larry McVoy and Noel Chiappa noted. It existed only as a specification for quite a while. First implementations were as a device-driver level function for things like frame-buffers for use by window systems and perhaps for other devices I'm not remembering at the moment. The general file/process memory integration aspect didn't happen prior to SunOS 4.0 though I seem to recall some special-case subsets being tried by other UNIX implementations and also more generally with Mach, I believe. The gestalt of file/process memory integration in 4.0 was very TENEX-like in how it was used in fork()/exec()/brk()/shared libraries. 4.0 differed from TENEX in being portable, launched on heterogeneous hardware (VAX, 68010 (Sun-2's), 68020 (Sun-3's), 386 (386i), SPARC (Sun-4) all except the VAX being actively supported Sun products at the time), and a general-networking view of how coherency should work (vs. a clustered-network such as TOPS-20's CFS.) (As a hysterical note, every release of (BSD-based) SunOS through 4.0 was always ported to the VAX as a portability check. It also provied us utility as a convenient way to quickly check for "but it works on the VAX" problems with new things gotten from UCB. Prior to 4.0, given that the BSD-based implementation basically tried to mimic a VAX internally, that wasn't too bad. But the VAXectomy that occurred in 4.0 made that much more daunting, and so the 4.0 VAX port was a pretty half-hearted attempt and the last new task for the 11/750 that faded deeper and deeper into the back of the lab over time.) ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [TUHS] mmap() (was: Memory management in Dennis Ritchie's C Compiler) 2020-08-18 0:52 ` Rob Gingell @ 2021-01-29 23:52 ` Greg A. Woods 0 siblings, 0 replies; 7+ messages in thread From: Greg A. Woods @ 2021-01-29 23:52 UTC (permalink / raw) To: The Unix Heritage Society mailing list [-- Attachment #1: Type: text/plain, Size: 2438 bytes --] At Mon, 17 Aug 2020 17:52:08 -0700, Rob Gingell <gingell@computer.org> wrote: Subject: Re: [TUHS] Memory management in Dennis Ritchie's C Compiler > > mmap() is descended from TENEX's PMAP, which is a simplified > descendant of the file/process memory integration provided by > Multics. (Other antecedents I leave to more informed genealogists than > I.) Indeed. I see mmap() as just a poor-man's version of Multics segments, and with it being bolted on after the fact, and without integrated language and compiler support, that makes it far less useful. > Multics was most definitely not portable due to its unique hardware > constraints (Intel's 432 being an exception). Well the basic design of Multics would/does fit reasonably well onto the iAPX-386 / IA-32 architecture, though a lone x86 CPU would have been a bit more limited in some capabilities than the Multics hardware was. I think that if more than just OS/2 had made good use of all of the segmentation features of iAPX-386 then possibly CPU designers (including beyond Intel) would have had some incentive to continue to improve it and make it more efficient and effective and indeed to carry it on in full-fledged manner to the x86-64 architecture as well. I think the failure of the iAPX-432 to meet performance expectations, especially with silicon limitations of the day (along with the i960MX never really getting full public exposure), put OS people in general off the idea of building anything better than Unix, which of course only needs a single flat address space and thus ran just fine on the much better performing silicon already available at the same time. I personally also think another problem that limited the porting and evolution of Multics, and the creation of new systems using much more of its design (and thus which favoured the growth of Unix in many ways) was that a lot of commercial OS people were put off by the idea of going with an object oriented model all the way from the languages on top and down through the OS and into the hardware. Too many users were (are) still running applications written in FORTRAN and Cobol and similar, and these were already implemented to use more direct read/write I/O models for data access that Unix supports well. -- Greg A. Woods <gwoods@acm.org> Kelowna, BC +1 250 762-7675 RoboHack <woods@robohack.ca> Planix, Inc. <woods@planix.com> Avoncote Farms <woods@avoncote.ca> [-- Attachment #2: OpenPGP Digital Signature --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2021-01-29 23:53 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-08-17 19:27 [TUHS] Memory management in Dennis Ritchie's C Compiler Noel Chiappa 2020-08-17 19:30 ` Larry McVoy 2020-08-17 19:44 ` Dan Halbert 2020-08-17 19:50 ` Paul Winalski 2020-08-17 22:05 ` Arthur Krewat 2020-08-18 0:52 ` Rob Gingell 2021-01-29 23:52 ` [TUHS] mmap() (was: Memory management in Dennis Ritchie's C Compiler) Greg A. Woods
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).