* [TUHS] Question about BSD disklabel history @ 2023-12-31 17:30 Grant Taylor via TUHS 2023-12-31 17:38 ` [TUHS] " arnold 2023-12-31 21:31 ` Clem Cole 0 siblings, 2 replies; 44+ messages in thread From: Grant Taylor via TUHS @ 2023-12-31 17:30 UTC (permalink / raw) To: The Unix Heritage Society [-- Attachment #1: Type: text/plain, Size: 649 bytes --] Hi, I've found myself wondering about partitions inside of BSD disk labels. Specifically, when and where was the convention that "a" is root, "b" is swap, etc? I also understand the "c" partition to be the entire disk, unless it isn't, at which point it's the entire slice (BIOS / MBR partition) containing the BSD disklabel and "d" is the entire disk. I also found something last night that indicated that OpenBSD uses disk labels somewhat differently than FreeBSD. Aside: This is one of the dangers of wondering how something curious came to be and why it came to be when working on 10-15 year old FreeBSD systems. -- Grant. . . . [-- Attachment #2: S/MIME Cryptographic Signature --] [-- Type: application/pkcs7-signature, Size: 4033 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2023-12-31 17:30 [TUHS] Question about BSD disklabel history Grant Taylor via TUHS @ 2023-12-31 17:38 ` arnold 2023-12-31 20:07 ` Warner Losh ` (2 more replies) 2023-12-31 21:31 ` Clem Cole 1 sibling, 3 replies; 44+ messages in thread From: arnold @ 2023-12-31 17:38 UTC (permalink / raw) To: tuhs, gtaylor The different overlapping partitions predates disk labels. Up to and including 4.3 BSD, to change the size of partitions on a particular disk, you had to recompile the kernel. They were that way so that if you had multiple disks, you could use one for root + swap + some thing small and use another whole disk for a single filesystem. It was also helpful, if you had the drives, to nightly dd your real root to the "a" partition on another, identical drive, so that you could boot the backup root in an emergency. I don't remember for sure, but I think that Ultrix may have been the first BSD-style system to have disk labels, followed by some version of SunOS. All of that is way in the distant past though: mid- to late 80's. I am guessing that the original conventions date back to V7 or 32V, but one would have to go looking at code to be sure. HTH, Arnold Grant Taylor via TUHS <tuhs@tuhs.org> wrote: > Hi, > > I've found myself wondering about partitions inside of BSD disk labels. > > Specifically, when and where was the convention that "a" is root, "b" is > swap, etc? > > I also understand the "c" partition to be the entire disk, unless it > isn't, at which point it's the entire slice (BIOS / MBR partition) > containing the BSD disklabel and "d" is the entire disk. > > I also found something last night that indicated that OpenBSD uses disk > labels somewhat differently than FreeBSD. > > Aside: This is one of the dangers of wondering how something curious > came to be and why it came to be when working on 10-15 year old FreeBSD > systems. > > > > -- > Grant. . . . ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2023-12-31 17:38 ` [TUHS] " arnold @ 2023-12-31 20:07 ` Warner Losh 2024-01-01 0:13 ` Bakul Shah 2023-12-31 20:27 ` Phil Budne 2024-01-01 0:26 ` Grant Taylor via TUHS 2 siblings, 1 reply; 44+ messages in thread From: Warner Losh @ 2023-12-31 20:07 UTC (permalink / raw) To: arnold; +Cc: tuhs, gtaylor [-- Attachment #1: Type: text/plain, Size: 3924 bytes --] On Sun, Dec 31, 2023 at 10:39 AM <arnold@skeeve.com> wrote: > The different overlapping partitions predates disk labels. > Up to and including 4.3 BSD, to change the size of partitions > on a particular disk, you had to recompile the kernel. > Yes. Disk partitions were implemented in the disk driver dating back at least to the 5th edition. Lack of sources makes it harder to know for sure. The man pages for each of the disk drivers included the partition layout (IIRC, I didn't cross verify). > They were that way so that if you had multiple disks, you > could use one for root + swap + some thing small and use another > whole disk for a single filesystem. > Yes. Unlike today, the partitions covered the disk in different, overlapping ways. And allowed for some parts of the disk to be uncovered by a partition. You could then patch the offset and length into the kernel with adb and use that area of the disk for swap space. > It was also helpful, if you had the drives, to nightly dd > your real root to the "a" partition on another, identical > drive, so that you could boot the backup root in an emergency. > > I don't remember for sure, but I think that Ultrix may have > been the first BSD-style system to have disk labels, followed > by some version of SunOS. All of that is way in the distant > past though: mid- to late 80's. > When I looked into it years ago, I convinced myself that SunOS was the first to have it (since the very first version of SunOS 1.0 had disk labels) and that all the other vendors followed suit within a couple of years. Ultrix-11 had the fixed labels through its EOL. I didn't see any disklable stuff in the Ultrix-32M that we have, but it was admittedly a quick look. > I am guessing that the original conventions date back to > V7 or 32V, but one would have to go looking at code to be sure. > Yes. The conventions are as old as Unix. And did change at least between V5 and V6 and I think maybe again for some drivers between V6 and V7 (though I'm less sure of this). BSD4.3 Tahoe, the first release after BSD4.3, fixed the disklabel format. Sun's vtoc format (aka sunlabel) was different from that, but quite similar. This is why it's called 4.3BSD format in many places, even though it wasn't in the original 4.3BSD release. OpenBSD and FreeBSD's disklabels are the same format. However, OpenBSD's have twice as many entries than FreeBSD's. And there's a number of other fussy details about what different labels mean based on the shared history and divergent evolution of the two systems. OpenBSD also supports both endians, while FreeBSD only supports native endian. So these changes and different decisions lead to the situation today where it's hard to read them on each-other's systems. Plus with the new, larger disks FreeBSD has effectively abandoned them in lue of GPT partitioning. There is a disklabel64 format too, but I'm unsure how widely that's deployed. All the commercial Unixes also had their own format. AIX, HPUX, DG, etc. That's a topic for another time... Warner > Grant Taylor via TUHS <tuhs@tuhs.org> wrote: > > > Hi, > > > > I've found myself wondering about partitions inside of BSD disk labels. > > > > Specifically, when and where was the convention that "a" is root, "b" is > > swap, etc? > > > > I also understand the "c" partition to be the entire disk, unless it > > isn't, at which point it's the entire slice (BIOS / MBR partition) > > containing the BSD disklabel and "d" is the entire disk. > > > > I also found something last night that indicated that OpenBSD uses disk > > labels somewhat differently than FreeBSD. > > > > Aside: This is one of the dangers of wondering how something curious > > came to be and why it came to be when working on 10-15 year old FreeBSD > > systems. > > > > > > > > -- > > Grant. . . . > [-- Attachment #2: Type: text/html, Size: 5514 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2023-12-31 20:07 ` Warner Losh @ 2024-01-01 0:13 ` Bakul Shah 0 siblings, 0 replies; 44+ messages in thread From: Bakul Shah @ 2024-01-01 0:13 UTC (permalink / raw) To: Warner Losh; +Cc: tuhs, gtaylor [-- Attachment #1: Type: text/plain, Size: 1990 bytes --] On Dec 31, 2023, at 12:07 PM, Warner Losh <imp@bsdimp.com> wrote: > > Yes. Unlike today, the partitions covered the disk in different, overlapping > ways. And allowed for some parts of the disk to be uncovered by a > partition. You could then patch the offset and length into the kernel with > adb and use that area of the disk for swap space. > >> It was also helpful, if you had the drives, to nightly dd >> your real root to the "a" partition on another, identical >> drive, so that you could boot the backup root in an emergency. >> >> I don't remember for sure, but I think that Ultrix may have >> been the first BSD-style system to have disk labels, followed >> by some version of SunOS. All of that is way in the distant >> past though: mid- to late 80's. > > When I looked into it years ago, I convinced myself that SunOS > was the first to have it (since the very first version of SunOS 1.0 > had disk labels) and that all the other vendors followed suit within > a couple of years. Ultrix-11 had the fixed labels through its EOL. > I didn't see any disklable stuff in the Ultrix-32M that we have, but > it was admittedly a quick look. I wrote the first 2-3 HD drivers for Fortune Systems. I had the first one up and running[1] by late 1981. IIRC we used an ioctl to read/write sector 0 of a disk, which is where we stored partition info. I think by 1983 we were using some 4.1a bits (or at least influenced by it) so likely disklabel was used by then. The first disk drive was 5MB and cost $1700. But 5 1/4" disk capacities were growing fast so there was no question of hardwiring a disktab in source code. I even had a program that would try to "step" through cylinders until it ran into errors, to find at usable capacity! [1] Well, more like walking! Initially DMA didn't work on the first wirewrap boards so had to use PIO (programmed IO), at 25KB/s. A quick hack doubled that performance, while an ST506 disk could do 5Mbits/sec. [-- Attachment #2: Type: text/html, Size: 7052 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2023-12-31 17:38 ` [TUHS] " arnold 2023-12-31 20:07 ` Warner Losh @ 2023-12-31 20:27 ` Phil Budne 2023-12-31 21:02 ` Warner Losh 2024-01-01 0:26 ` Grant Taylor via TUHS 2 siblings, 1 reply; 44+ messages in thread From: Phil Budne @ 2023-12-31 20:27 UTC (permalink / raw) To: tuhs, gtaylor, arnold arnold@skeeve.com wrote: > I am guessing that the original conventions date back to > V7 or 32V, but one would have to go looking at code to be sure. Looking at the hp (RP-04/5/6) man page in V7, 32/v, and 3bsd the convention was rpN, where 0 <= N <= 7 were partitions on the first drive, and rp8 was the first partition on the second drive. rp0 was root, rp1 was swap, rp2 and rp3 (in the v7 man page) are unassigned. The rpNx convention, where N is the drive number, with "c" as the whole disk is present in the 4.1BSD hp(4) man page. ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2023-12-31 20:27 ` Phil Budne @ 2023-12-31 21:02 ` Warner Losh 0 siblings, 0 replies; 44+ messages in thread From: Warner Losh @ 2023-12-31 21:02 UTC (permalink / raw) To: Phil Budne; +Cc: tuhs, gtaylor [-- Attachment #1: Type: text/plain, Size: 961 bytes --] On Sun, Dec 31, 2023 at 1:27 PM Phil Budne <phil@ultimate.com> wrote: > arnold@skeeve.com wrote: > > I am guessing that the original conventions date back to > > V7 or 32V, but one would have to go looking at code to be sure. > > Looking at the hp (RP-04/5/6) man page in V7, 32/v, and 3bsd the > convention was rpN, where 0 <= N <= 7 were partitions on the first > drive, and rp8 was the first partition on the second drive. rp0 was > root, rp1 was swap, rp2 and rp3 (in the v7 man page) are unassigned. > > The rpNx convention, where N is the drive number, with "c" as the > whole disk is present in the 4.1BSD hp(4) man page. > This was just a change to MAKEDEV. The kernel used the same minor number encoding. 4BSD still used the V7 make script in /dev, but at some point before 4.1cBSD (I don't have the 4.1BSD exploded locally) it changed, both in name from MAKE to MAKEDEV and to start the [a-h] suffix for partitions. Warner [-- Attachment #2: Type: text/html, Size: 1444 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2023-12-31 17:38 ` [TUHS] " arnold 2023-12-31 20:07 ` Warner Losh 2023-12-31 20:27 ` Phil Budne @ 2024-01-01 0:26 ` Grant Taylor via TUHS 2024-01-01 2:22 ` Warner Losh 2 siblings, 1 reply; 44+ messages in thread From: Grant Taylor via TUHS @ 2024-01-01 0:26 UTC (permalink / raw) To: tuhs [-- Attachment #1: Type: text/plain, Size: 3545 bytes --] On 12/31/23 11:38, arnold@skeeve.com wrote: > The different overlapping partitions predates disk labels. Okay. That in and of itself doesn't surprise me much that a convention of overlapping partitions was carried forward from the driver based partitioning into label based partitioning. > Up to and including 4.3 BSD, to change the size of partitions on a > particular disk, you had to recompile the kernel. So I've learned over the last couple of years as I read more about Unix history. > They were that way so that if you had multiple disks, you could use > one for root + swap + some thing small and use another whole disk > for a single filesystem. I'm not understanding how /overlapping/ partitions helps make use of portions of disks. Maybe I should back up and ask for clarification. What /overlapping/ partitions means in this context? My naive assumption was that partition -- I use that term loosely -- "c" overlaps / contains / all other partitions on the disk; "a", "b", and maybe "d". I'm eliding the "c" MBR partition vs "d" entire drive" distinction for the moment. I see some value in the "c" partition being the entire disk as used by BSD so that it's possible to point backup / restore / copy utilities at the entire disk. But I don't understand value in having partitions overlap / contain each other's blocks, save for backup via "c". I do see some value in extending the "c" is the entire MBR partition methodology to "d" is the entire disk containing multiple MBR partitions. Again, the value seems to be in backup and recovery. But I still simply do not understand why I would ever want partition "e" to be blocks 100-199, partition "f" to be blocks 195-299, and partition "g" to be blocks 295-399. What value is there in having partitions e, f, and g overlap each other? I get dd if=/dev/<something>0c of=/dev/rmt. Or even /dev/<something>0d. I fail to understand why I'd ever want other partitions to overlap. > It was also helpful, if you had the drives, to nightly dd your real > root to the "a" partition on another, identical drive, so that you > could boot the backup root in an emergency. Sure. But I don't see what that has to do with overlapping partitions. I'd naively think that I could do something like the following: dd if=/dev/<something>0a of=/dev/<something>1a And get the same effect. > I am guessing that the original conventions date back to V7 or 32V, > but one would have to go looking at code to be sure. "a" for root makes some intuitive sense as the root file system is required to do anything else. Then when you want swap, the next partition is "b". Wanting another partition that is the entire disk (as seen by BSD) makes some logical sense to me too, so "c". Were subsequent partitions sort of used as needed and had less consistency? Especially when "d" because the entire disk containing multiple MBR partitions when "c" was restricted to the MBR partition the label was in? Aside: Would that mean that the following "d" partitions would be the same thing, as in the entire /dev/ad0 disk? /dev/ad0s0d /dev/ad0s1d Wherein I'm borrowing the FreeBSD slice nomenclature -- as I understand it -- to identify the first (zero) and second (one) MBR partition on /dev/ad0 History and how we got to where we are today can be both very confusing and even more enlightening once you understand it. What's more is that once you understand it, things start making more intuitive sense when you look at them. -- Grant. . . . [-- Attachment #2: S/MIME Cryptographic Signature --] [-- Type: application/pkcs7-signature, Size: 4033 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2024-01-01 0:26 ` Grant Taylor via TUHS @ 2024-01-01 2:22 ` Warner Losh 2024-01-01 3:24 ` Grant Taylor via TUHS 0 siblings, 1 reply; 44+ messages in thread From: Warner Losh @ 2024-01-01 2:22 UTC (permalink / raw) To: Grant Taylor; +Cc: The Eunuchs Hysterical Society [-- Attachment #1: Type: text/plain, Size: 4425 bytes --] 8 On Sun, Dec 31, 2023, 5:26 PM Grant Taylor via TUHS <tuhs@tuhs.org> wrote: > On 12/31/23 11:38, arnold@skeeve.com wrote: > > The different overlapping partitions predates disk labels. > > Okay. That in and of itself doesn't surprise me much that a convention > of overlapping partitions was carried forward from the driver based > partitioning into label based partitioning. > Many editors dont let you do this... > Up to and including 4.3 BSD, to change the size of partitions on a > > particular disk, you had to recompile the kernel. > > So I've learned over the last couple of years as I read more about Unix > history. > > > They were that way so that if you had multiple disks, you could use > > one for root + swap + some thing small and use another whole disk > > for a single filesystem. > > I'm not understanding how /overlapping/ partitions helps make use of > portions of disks. > Maybe I should back up and ask for clarification. What /overlapping/ > partitions means in this context? > It means you either use one set of non overlapping partitions or another set. They were setup in clever ways My naive assumption was that partition -- I use that term loosely -- "c" > overlaps / contains / all other partitions on the disk; "a", "b", and > maybe "d". > > I'm eliding the "c" MBR partition vs "d" entire drive" distinction for > the moment. > > I see some value in the "c" partition being the entire disk as used by > BSD so that it's possible to point backup / restore / copy utilities at > the entire disk. > > But I don't understand value in having partitions overlap / contain each > other's blocks, save for backup via "c". > > I do see some value in extending the "c" is the entire MBR partition > methodology to "d" is the entire disk containing multiple MBR > partitions. Again, the value seems to be in backup and recovery. > > But I still simply do not understand why I would ever want partition "e" > to be blocks 100-199, partition "f" to be blocks 195-299, and partition > "g" to be blocks 295-399. What value is there in having partitions e, > f, and g overlap each other? > > I get dd if=/dev/<something>0c of=/dev/rmt. Or even /dev/<something>0d. > > I fail to understand why I'd ever want other partitions to overlap. > It's more like you can use two or three partitions with non overlapping sets that cover the whole disk. > It was also helpful, if you had the drives, to nightly dd your real > > root to the "a" partition on another, identical drive, so that you > > could boot the backup root in an emergency. > > Sure. But I don't see what that has to do with overlapping partitions. > > I'd naively think that I could do something like the following: > > dd if=/dev/<something>0a of=/dev/<something>1a > > And get the same effect. > > > I am guessing that the original conventions date back to V7 or 32V, > > but one would have to go looking at code to be sure. > > "a" for root makes some intuitive sense as the root file system is > required to do anything else. > > Then when you want swap, the next partition is "b". > > Wanting another partition that is the entire disk (as seen by BSD) makes > some logical sense to me too, so "c". > > Were subsequent partitions sort of used as needed and had less > consistency? Especially when "d" because the entire disk containing > multiple MBR partitions when "c" was restricted to the MBR partition the > label was in? > > Aside: > > Would that mean that the following "d" partitions would be the same > thing, as in the entire /dev/ad0 disk? > > /dev/ad0s0d > /dev/ad0s1d > > Wherein I'm borrowing the FreeBSD slice nomenclature -- as I understand > it -- to identify the first (zero) and second (one) MBR partition on > /dev/ad0 > Yes. But ancient Unix didn’t have nested partitioning schemes like FreeBSD supports... History and how we got to where we are today can be both very confusing > and even more enlightening once you understand it. What's more is that > once you understand it, things start making more intuitive sense when > you look at them. > Think more of a limited number of ways to mix and match for greater flexibility w/o editing the tables. A silly example: a is first 2/3 of the disk. B is 2nd 2/3, c d and e are 1/3 each. Warner -- > Grant. . . . > [-- Attachment #2: Type: text/html, Size: 6932 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2024-01-01 2:22 ` Warner Losh @ 2024-01-01 3:24 ` Grant Taylor via TUHS 0 siblings, 0 replies; 44+ messages in thread From: Grant Taylor via TUHS @ 2024-01-01 3:24 UTC (permalink / raw) To: tuhs [-- Attachment #1: Type: text/plain, Size: 1024 bytes --] On 12/31/23 20:22, Warner Losh wrote: > It means you either use one set of non overlapping partitions or > another set. They were setup in clever ways > > It's more like you can use two or three partitions with non overlapping > sets that cover the whole disk. ... > Think more of a limited number of ways to mix and match for greater > flexibility w/o editing the tables. > > A silly example: a is first 2/3 of the disk. B is 2nd 2/3, c d and > e are 1/3 each. Okay. This makes more sense. Pre-define overlapping partitions but only use non-overlapping. Sort of like tool boxes that come with a bunch of supports for dividers, but you can only put so many dividers in the pre-defined spots, but contents between the dividers doesn't overlap. Sort of like how many squares are in this bigger grid of dots that is itself a big square. > But ancient Unix didn’t have nested partitioning schemes like > FreeBSD supports... Understood and appreciated. -- Grant. . . . [-- Attachment #2: S/MIME Cryptographic Signature --] [-- Type: application/pkcs7-signature, Size: 4033 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2023-12-31 17:30 [TUHS] Question about BSD disklabel history Grant Taylor via TUHS 2023-12-31 17:38 ` [TUHS] " arnold @ 2023-12-31 21:31 ` Clem Cole 2023-12-31 22:07 ` Warner Losh 2023-12-31 22:46 ` G. Branden Robinson 1 sibling, 2 replies; 44+ messages in thread From: Clem Cole @ 2023-12-31 21:31 UTC (permalink / raw) To: Grant Taylor; +Cc: The Unix Heritage Society [-- Attachment #1: Type: text/plain, Size: 8156 bytes --] Below... apologies if I forgot something -- this is pretty much as I knew it/play it out. On Sun, Dec 31, 2023 at 12:31 PM Grant Taylor via TUHS <tuhs@tuhs.org> wrote: > Hi, > > I've found myself wondering about partitions inside of BSD disk labels. > > Specifically, when and where was the convention that "a" is root, "b" is > swap, etc? > > I also understand the "c" partition to be the entire disk, unless it > isn't, at which point it's the entire slice (BIOS / MBR partition) > containing the BSD disklabel and "d" is the entire disk. > > I also found something last night that indicated that OpenBSD uses disk > labels somewhat differently than FreeBSD. > > Aside: This is one of the dangers of wondering how something curious > came to be and why it came to be when working on 10-15 year old FreeBSD > systems. > > A long, strange trip as were. Step one was the introduction of DEC RP04 disks -- 19 Heads, 21 Sectors/Track, 411 cylinders or 163989 blocks which will overflow a 16-bit PDP-11 register. You find it in the V5 code, but I have Dennis's rp.c driver showing split the RP04 into from v6: struct { char *nblocks; int cyloff; } rp_sizes[] { 40600, 0, /* cyl 0 thru 202 */ 40600, 203, /* cyl 203 thru 405 */ 9200, 0, /* cyl 0 thru 45 */ 9200, 360, /* cyl 360 thru 405 */ -1, 0, /* cyl 0 thru 327 */ -1, 78, /* cyl 78 thru 405 */ 15600, 0, /* cyl 0 thru 77 */ 15600, 328, /* cyl 328 thru 405 */ }; (V5/V6 C does not have an unsigned or longs; char * is how the compiler created a type that was treated as an unsigned 16-bit integer). If you look at things like the rk05 in V6, we reserve a few blocks at the end of the disk for swap. Remember quotas and file single file management are not there yet, so the scheme with the RP04 starts to be the way of the world. Also, remember Ted has not introduced what would later be called fsck(1), so recovery from a crash was painful. The corruption was often on the root disk, a quick dd(1) of the root FS you were quickly back - or at least bookable and then you could use check, ncheck, *et al *on the other partitions. Also, with PWB - a new set of tools was created for disk management that Research did not pick up: volcopy, cpio, et al. The V6 version of PWB is where *disklabels are first added to the superblock* BTW. In fact, when we did the original fsck(8) at CMU, we had v5 then v6. No, PWB pollution into our IP stream. So the original fsck I recently recovered for V6 does not have the disklabel code in it. When Ted goes to USG later, he adds the label support (which is why if you compile the version in the V7 addenda, it will get errors -- that one is the version on UNIX/TS - which the kernel supported disklabels). I had to look at the code, I think labels are simple, just names not information about that specific FS. The labels are not per disk pack. Meanwhile back in Murray Hill .... With V7, we have long int, but the partition scheme is already in use and has some nice side effects, so it is now a feature, and with the RP06 -- so Dennis's new scheme: struct size { daddr_t nblocks; int cyloff; } hp_sizes[8] = { 9614, 0, /* cyl 0 thru 22 */ 8778, 23, /* cyl 23 thru 43 */ 0, 0, 0, 0, 161348, 44, /* cyl 44 thru 429 */ 160930, 430, /* cyl 430 thru 814 */ 153406, 44, /* cyl 44 thru 410 (rp04, rp05) */ 322278, 44, /* cyl 44 thru 814 (rp06) */ }; At this point, as Arnold said, people would change the size/shape of the tables locally by recompiling and setting up the disk they way you wanted. It could be awkward, you typically needed a second disk pack, *etc*. With DEC's release of V7m, 2.x (2.1, I think), Fred Cantor of DEC put the first versions of labels into what would later become Ultrix-11. That code was released to CRSG (it is also where 2.9 et al. get Fred's wonderful swapping support). Shannon (still at DEC) helped Sam add some of that into a 4.1a/b/c stream... This was where Sam added things like newfs(8) and /etc/disktab and started to try to make BSD more resilient to different types of disks. But my memory is that it was nascent in the 4.2 release, so you had /etc/disktab and some of the tools but not everything until later, 4.3, when what you know today came into being. Meanwhile, in Pittsburgh ... CMU is working on the Mach2.5 project. Intel had donated some of the original 386 PC support to Rasid's group (they were partial funding) that they had developed with ISC/AT&T *et al* to handle things particular to the PC architecture -- such as the PC's fdisk(1) since PC's have a completely different way of partitioning things. Bob Baron did the Mach version - as to what was new and what was from Intel I'm not 100% sure. But CMU did have the tools that Intel felt they owned and were not part of System III/V. The key is that a PC's fdisk(1) - creates a UNIX partition within the DOS partition table so DOS knows to leave it alone, but then Unix sub-partitions using its own scheme. My memory here is hazy, I think the sub-partition code Bob used is different from the ATT's 386 [which most people saw as ISC's 386/ix who did the original work under contract for AT&T] Meanwhile, across the pond ... Andy's team starts to want a V7 for the PC/XT and starts the Minux project. I know they had CMU's Mach 2.5 stuff and there fdisk, while it seems to be independent (i.e. clean room) is very similar to Bob Baron's scheme. Meanwhile, in Walnut Creek ... Jolitiz *et al.* started 386BSD, which would begat FreeBSD/NetBSD/OpenBSD. At some point, Baron's fdisk (1) and support tools make their way to that project. This was why all the original UNIX for a PC used the same PC fdisk "type" ID number (the ISC 386/ix type -- from the original 386 port). [Linus built his own tools, which I think started from the Minux tools, and thus he did his own thing --- so Linux disks were not marked with the same types). If you look at the sources to both Mach and early 386BSD, you'll see Intel markings as well as Bob's name in a couple of places [Truth in advertising here (different story) -- I wrote most of the original AT disk driver Bill used -- I was consulting for NCR and had access to the WD specs as well as the CMU code but I was not the conduit of the CMU code base to Bill]. At some point, 386BSD started to go one way, and what would become FreeBSD started to ramp up. Jordan Hubbard wants an "easy install" like Microsoft has for Win98, so very early FreeBSD has the first PC-based install that can run without much intervention. At some point, Jordon or someone else in the FreeBSD team introduced what they called slices, furthering the BSD scheme but designed to overlay the PC better. The NetBSD folks had already split off, so they did not pick it up. When OpenBSD split from NetBSD, they did something similar but different to FreeBSD's slices... Meanwhile, in Cupertino .... Next (ney Apple) started with the Mach code base from CMU. There was a push in the Valley in those days for something called OpenFirmware [Warner help here -- I think that it was forth based IIRC, and Sun may have had their hand in it also]. But the key is that it ran on 68K's. So Next's original partition scheme was influenced if not based on some of Bob's work, although they did not need the silly BIOS stuff, so they did make it all talk to OpenFW (BTW: I think that was still all Next) By the time the Intel Mac's BIOS had begun to be replaced in the WINTEL world by UEFI, Apple committed to using a flavor of it. My old lab partner, Guy Sotomayer (*a.k.a.* gss on many mailing lists) was mixed up during his time at Apple. He might be able to fill in some details if I ask him. I also know that Apple was not happy with how disks were handled with the original partition scheme, and by later release started over with their partitioning scheme(s) at least once. I've undoubtedly left stuff out, such as much of Linux - but this is all part of the UNIX war I was a player. [-- Attachment #2: Type: text/html, Size: 18366 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2023-12-31 21:31 ` Clem Cole @ 2023-12-31 22:07 ` Warner Losh 2024-01-01 16:00 ` Clem Cole 2023-12-31 22:46 ` G. Branden Robinson 1 sibling, 1 reply; 44+ messages in thread From: Warner Losh @ 2023-12-31 22:07 UTC (permalink / raw) To: Clem Cole; +Cc: Grant Taylor, The Unix Heritage Society [-- Attachment #1: Type: text/plain, Size: 13159 bytes --] On Sun, Dec 31, 2023 at 2:31 PM Clem Cole <clemc@ccc.com> wrote: > Below... apologies if I forgot something -- this is pretty much as I knew > it/play it out. > > On Sun, Dec 31, 2023 at 12:31 PM Grant Taylor via TUHS <tuhs@tuhs.org> > wrote: > >> Hi, >> >> I've found myself wondering about partitions inside of BSD disk labels. >> >> Specifically, when and where was the convention that "a" is root, "b" is >> swap, etc? >> >> I also understand the "c" partition to be the entire disk, unless it >> isn't, at which point it's the entire slice (BIOS / MBR partition) >> containing the BSD disklabel and "d" is the entire disk. >> >> I also found something last night that indicated that OpenBSD uses disk >> labels somewhat differently than FreeBSD. >> >> Aside: This is one of the dangers of wondering how something curious >> came to be and why it came to be when working on 10-15 year old FreeBSD >> systems. >> >> A long, strange trip as were. > > Step one was the introduction of DEC RP04 disks -- 19 Heads, 21 > Sectors/Track, 411 cylinders or 163989 blocks which will overflow a 16-bit > PDP-11 register. You find it in the V5 code, but I have Dennis's rp.c > driver showing split the RP04 into from v6: > > struct { > > char *nblocks; > > int cyloff; > > } rp_sizes[] { > > 40600, 0, /* cyl 0 thru 202 */ > 40600, 203, /* cyl 203 thru 405 */ > 9200, 0, /* cyl 0 thru 45 */ > 9200, 360, /* cyl 360 thru 405 */ > -1, 0, /* cyl 0 thru 327 */ > -1, 78, /* cyl 78 thru 405 */ > 15600, 0, /* cyl 0 thru 77 */ > 15600, 328, /* cyl 328 thru 405 */ > > }; > > (V5/V6 C does not have an unsigned or longs; char * is how the compiler > created a type that was treated as an unsigned 16-bit integer). > > If you look at things like the rk05 in V6, we reserve a few blocks at the > end of the disk for swap. Remember quotas and file single file management > are not there yet, so the scheme with the RP04 starts to be the way of the > world. Also, remember Ted has not introduced what would later be called > fsck(1), so recovery from a crash was painful. The corruption was often > on the root disk, a quick dd(1) of the root FS you were quickly back - or > at least bookable and then you could use check, ncheck, *et al *on the > other partitions. > > Also, with PWB - a new set of tools was created for disk management that > Research did not pick up: volcopy, cpio, et al. The V6 version of PWB is > where *disklabels are first added to the superblock* BTW. In fact, > when we did the original fsck(8) at CMU, we had v5 then v6. No, PWB > pollution into our IP stream. So the original fsck I recently recovered > for V6 does not have the disklabel code in it. When Ted goes to USG > later, he adds the label support (which is why if you compile the version > in the V7 addenda, it will get errors -- that one is the version on UNIX/TS > - which the kernel supported disklabels). I had to look at the code, I > think labels are simple, just names not information about that specific > FS. The labels are not per disk pack. > > Meanwhile back in Murray Hill .... > > With V7, we have long int, but the partition scheme is already in use and > has some nice side effects, so it is now a feature, and with the RP06 -- > so Dennis's new scheme: > > struct size > { > > daddr_t nblocks; > int cyloff; > > > } hp_sizes[8] = > > { > 9614, 0, /* cyl 0 thru 22 */ > 8778, 23, /* cyl 23 thru 43 */ > 0, 0, > 0, 0, > 161348, 44, /* cyl 44 thru 429 */ > 160930, 430, /* cyl 430 thru 814 */ > 153406, 44, /* cyl 44 thru 410 (rp04, rp05) */ > 322278, 44, /* cyl 44 thru 814 (rp06) */ > > }; > > At this point, as Arnold said, people would change the size/shape of the > tables locally by recompiling and setting up the disk they way you wanted. > It could be awkward, you typically needed a second disk pack, *etc*. > > With DEC's release of V7m, 2.x (2.1, I think), Fred Cantor of DEC put the > first versions of labels into what would later become Ultrix-11. That > code was released to CRSG (it is also where 2.9 et al. get Fred's wonderful > swapping support). Shannon (still at DEC) helped Sam add some of that > into a 4.1a/b/c stream... This was where Sam added things like newfs(8) > and /etc/disktab and started to try to make BSD more resilient to > different types of disks. But my memory is that it was nascent in the 4.2 > release, so you had /etc/disktab and some of the tools but not everything > until later, 4.3, when what you know today came into being. > Except I can't find any of this in our V7m or ultrix-11 that we have. This is 2.1. And there all the device drivers still have the fixed tables: grep sizes * | grep '\[[0-9]' hk.c:} hk_sizes[8] = hk.c_i:} hk_sizes[8] = hk.c_id:} hk_sizes[8] = hm.c:} hm_sizes[24] = hp.c:} hp_sizes[24] = and the ultrix-11 3.1 sources we have are similar, but move all of the above into /usr/sys/conf/dksizes.c. The early disktab stuff encoded the default tables into a text file to make it easier to cope with all the different types of disks, but didn't affect what we know as bsdlabels until after 4.3BSD. The naming convention was there, but the 'write bits to the disk to describe partitioning' I couldn't find at all in ultrix. I think you may be confusing that stuff (which was a very clever way to cope with all these disks that have different partitioning in a increasingly generic way) with disk labeling which I think. Sun was first to market with. Maybe you can help me find it? I did find the SunOS 0.x images (UniSoft V7 port) didn't appear have a disk partition, but 1.0 (post 4BSD pre 4.1BSD) and later did. That puts it at 1982. Although based off BSD, the disk label stuff wasn't in BSD for another 5 years when the BSD folks started the Tahoe port. It's entirely possible one of the early UniSoft ports had this feature, and that was adapted into SunOS and I've not seen the paper trail (studying the early UniSoft ports is rather problematic for a variety of logistical reasons)... Not least of which is the fragmentary nature of the surviving source, the difficulty in obtaining boot images, etc. This is a very under-documented time. It wouldn't surprise me if someone produced evidence of Sun's partitioning scheme coming from this mess... Or a very similar thing. And Pixar may have been involved (though that's a memory I have that I don't have in my notes documented). Maybe that's the Sam Leffler connection (though I thought his time at Pixar post-dated this period somewhat). Meanwhile, in Pittsburgh ... > > CMU is working on the Mach2.5 project. Intel had donated some of the > original 386 PC support to Rasid's group (they were partial funding) that > they had developed with ISC/AT&T *et al* to handle things particular to > the PC architecture -- such as the PC's fdisk(1) since PC's have a > completely different way of partitioning things. Bob Baron did the Mach > version - as to what was new and what was from Intel I'm not 100% sure. > But CMU did have the tools that Intel felt they owned and were not part of > System III/V. > > The key is that a PC's fdisk(1) - creates a UNIX partition within the DOS > partition table so DOS knows to leave it alone, but then Unix > sub-partitions using its own scheme. My memory here is hazy, I think the > sub-partition code Bob used is different from the ATT's 386 [which most > people saw as ISC's 386/ix who did the original work under contract for > AT&T] > > Meanwhile, across the pond ... > Andy's team starts to want a V7 for the PC/XT and starts the Minux > project. I know they had CMU's Mach 2.5 stuff and there fdisk, while it > seems to be independent (i.e. clean room) is very similar to Bob Baron's > scheme. > Yea, the fdisk stuff came from early versions of PC-DOS to partition hard drives in 2.0 / 1983. I do know the DEC Rainbow had yet another set of partitioning (which the PRO and DecMATE didn't share). It never adopted what today is the MBR partitioning scheme because it's hard disk support pre-dates this. It quickly became standard, and we see evidence of support for it in the Venix/86 sources that we have (both the XT version and the AT version, though I think that was disk controller version, and not MBR data). > Meanwhile, in Walnut Creek ... > > Jolitiz *et al.* started 386BSD, which would begat > FreeBSD/NetBSD/OpenBSD. At some point, Baron's fdisk (1) and support > tools make their way to that project. This was why all the original > UNIX for a PC used the same PC fdisk "type" ID number (the ISC 386/ix type > -- from the original 386 port). [Linus built his own tools, which I think > started from the Minux tools, and thus he did his own thing --- so Linux > disks were not marked with the same types). If you look at the sources to > both Mach and early 386BSD, you'll see Intel markings as well as Bob's name > in a couple of places [Truth in advertising here (different story) -- I > wrote most of the original AT disk driver Bill used -- I was consulting for > NCR and had access to the WD specs as well as the CMU code but I was not > the conduit of the CMU code base to Bill]. > At some point, 386BSD started to go one way, and what would become FreeBSD > started to ramp up. Jordan Hubbard wants an "easy install" like Microsoft > has for Win98, so very early FreeBSD has the first PC-based install that > can run without much intervention. At some point, Jordon or someone else > in the FreeBSD team introduced what they called slices, furthering the BSD > scheme but designed to overlay the PC better. The NetBSD folks had > already split off, so they did not pick it up. When OpenBSD split from > NetBSD, they did something similar but different to FreeBSD's slices... > Yea, this was 1992 or 1993. In FreeBSD 1.x you just had BSD Labels which would default in interesting ways to what the MBR of the disk was (I've not double checked the code, this is my memory). The 'c' partition was the entire BSD partition while the 'd' partition was the entire disk. And that duality lived on for a while. The slice code came in during the 3.x series (or after 2.0, which used the same scheme as 1.x). And it was mostly an attempt to make the BSD labels independent of each other. There was some weird code for bsd-label only disks but most of the visible changes from slices came in device naming (if you had only one disk label, and it was on the first partition it was compatible with old bsd labels by and large). FreeBSD wouldn't have true nested MBR and bsd labels until 5.0 and geom entered the tree... A lot of the 'differences' between the BSDs were in presentation of the data to the user, rather than physical differences on the disk (though such differences did exist, especially in endian). It was a real mess of NIH + incompatible UI + almost-but-not-quite compatible on-disk formats coupled with a bunch of bugs that differed among the BSDs. I do not miss these days. > Meanwhile, in Cupertino .... > > Next (ney Apple) started with the Mach code base from CMU. There was a > push in the Valley in those days for something called OpenFirmware [Warner > help here -- I think that it was forth based IIRC, and Sun may have had > their hand in it also]. But the key is that it ran on 68K's. So Next's > original partition scheme was influenced if not based on some of Bob's > work, although they did not need the silly BIOS stuff, so they did make it > all talk to OpenFW (BTW: I think that was still all Next) > I'm not sure about Next's partitioning scheme. I do know that Apple adopted the OpenFirmware partitioning scheme for PPC Mac leading to the APM scheme which FreeBSD supports to this day for PowerPC. This was different from Sun's sunlabels (which differed from BSD labels since they pre-dated them by half a decade. By the time the Intel Mac's BIOS had begun to be replaced in the WINTEL > world by UEFI, Apple committed to using a flavor of it. My old lab partner, > Guy Sotomayer (*a.k.a.* gss on many mailing lists) was mixed up during > his time at Apple. He might be able to fill in some details if I ask him. > I also know that Apple was not happy with how disks were handled with the > original partition scheme, and by later release started over with their > partitioning scheme(s) at least once. > Yea, GPT partitioning came along a lot later, kinda sort with EFI which Intel and Apple did (intel for ia64, Apple for 32-bit x86 macs). It's one of the few partitioning schemes I rather like. > I've undoubtedly left stuff out, such as much of Linux - but this is all > part of the UNIX war I was a player. > Yea, not to mention the HP, AIX, DG and other partitioning schemes that grew up in the late 80s and early 90s to have parity with MBR disks at the feature level, but rarely at the disk format level. A lot of it is documented, but in hard to find places and I've not done a systematic survey. Warner [-- Attachment #2: Type: text/html, Size: 22409 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2023-12-31 22:07 ` Warner Losh @ 2024-01-01 16:00 ` Clem Cole 2024-01-02 18:49 ` Warner Losh 0 siblings, 1 reply; 44+ messages in thread From: Clem Cole @ 2024-01-01 16:00 UTC (permalink / raw) To: Warner Losh; +Cc: Grant Taylor, The Unix Heritage Society [-- Attachment #1: Type: text/plain, Size: 4496 bytes --] Warner -- thanks... On Sun, Dec 31, 2023 at 5:07 PM Warner Losh <imp@bsdimp.com> wrote: > > > Except I can't find any of this in our V7m or ultrix-11 that we have. This > is 2.1. And there all the device drivers still have the fixed tables: > .... > > and the ultrix-11 3.1 sources we have are similar, but move all of the > above into /usr/sys/conf/dksizes.c. > I remember dksizes.c -- I'll take your word for it being 3.X, not 2.X; I'm pretty sure that was the start of the work in Merrimack by Fred Cantor to make things more independent. Shannon would have been aware of all of it, and before he went to Sun, he and Jean were working with the folks at CSRG (Sam, in particular), moving some of the DEC I/O support into BSD. Bill Munson convinced his management that it was worth it for DEC to at least make sure the DEC peripherals were being well handled and in mostly the same manner. IIRC Jean spent 6-9 months embedded in the CSRG working on much of that. > The early disktab stuff encoded the default tables into a text file to > make it easier to cope with all the different types of disks, but didn't > affect what we know as bsdlabels until after 4.3BSD. > Right. > The naming convention was there, but the 'write bits to the disk to > describe partitioning' I couldn't find at all in ultrix. I think you may be > confusing that stuff (which was a very clever way to cope with all these > disks that have different partitioning in a increasingly generic way) with > disk labeling which I think. > Quite possible. The key is that partitioning the disk to allow its use for different things and disk geometry support get all mixed together in the different schemes. As we discussed, it often happens in multiple places (since the ROMs, like the PC's BIOS, need at least some of the info at boot), and the loaded OS (particularly ones with multiple OSs on them) might want to do something completely different. This is why Grant's question is a little hard to answer directly - as I said, it is a long, slow trip. UNIX Partitioning, like what Dennis did, came first. If you look at PDP-11 and later Vaxen, the "disk support" for booting is pretty crude and built into the ROM -- *i.e*., boot RP0 or RK0 -- the "disk geometry" is hidden in the Boot ROM. But when so many disks start to show up using the same controllers, the ROM needs to be smarter. So, some way to encode geometry is needed. But Partitioning for the OS is still something that is handy and so often got put into the same support (such as the PC's BIOS tables - which were a good idea, poorly executed). > Sun was first to market with. > To be fair, Masscomp's "disk geometry" code that Paul Cantrell wrote pre-dated Sun by at least a year or more. I did not include it in my history, as it is private to their boot ROM. Nice scheme, actually - but proprietary, and I don't think any of the ideas went anywhere else - other than later Sun ROMs supported a similar functionality, and they would have at least seen them at customers and known about it, particularly since Sun picked up the Xylogic disk and tape controllers that they developed for Masscomp originally (Paul spent many hours at Xylogics helping with their Microcode). The point is, by that time, the proliferation of different disk manufacturers -- something to make the boot ROMs and OS's more independent was forced on the different systems providers if they were going to have any chance of being able to be flexible in the market/field. > > Maybe you can help me find it? > I'll see what I can dig up. > > I did find the SunOS 0.x images (UniSoft V7 port) didn't appear have a > disk partition, but 1.0 (post 4BSD pre 4.1BSD) and later did. That puts it > at 1982. Although based off BSD, the disk label stuff wasn't in BSD for > another 5 years when the BSD folks started the Tahoe port. > Right, this is all of the MK0, CSRG, and Sun connection via Fred, Bill, and Sam. At a minimum, they knew what the other was doing/had done, and some of it definitely migrated. WRT to OpenFW, my memory is that Larry points out that Sun was the primary driver. But a lot of the Motorola club got interested in using it, too, during the "JAWS" timeframe. I am pretty sure that there were versions for 68K, 88K, and PPC, as well as the SPARC version from Sun. Somewhere I have early source distribution. ᐧ ᐧ [-- Attachment #2: Type: text/html, Size: 8835 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2024-01-01 16:00 ` Clem Cole @ 2024-01-02 18:49 ` Warner Losh 2024-01-02 19:30 ` Chet Ramey 2024-01-02 19:50 ` Dan Cross 0 siblings, 2 replies; 44+ messages in thread From: Warner Losh @ 2024-01-02 18:49 UTC (permalink / raw) To: Clem Cole; +Cc: Grant Taylor, The Unix Heritage Society [-- Attachment #1: Type: text/plain, Size: 6872 bytes --] On Mon, Jan 1, 2024 at 9:00 AM Clem Cole <clemc@ccc.com> wrote: > > Warner -- thanks... > > On Sun, Dec 31, 2023 at 5:07 PM Warner Losh <imp@bsdimp.com> wrote: > >> >> >> Except I can't find any of this in our V7m or ultrix-11 that we have. >> This is 2.1. And there all the device drivers still have the fixed tables: >> .... >> >> and the ultrix-11 3.1 sources we have are similar, but move all of the >> above into /usr/sys/conf/dksizes.c. >> > I remember dksizes.c -- I'll take your word for it being 3.X, not 2.X; I'm > pretty sure that was the start of the work in Merrimack by Fred Cantor to > make things more independent. Shannon would have been aware of all of it, > and before he went to Sun, he and Jean were working with the folks at CSRG > (Sam, in particular), moving some of the DEC I/O support into BSD. Bill > Munson convinced his management that it was worth it for DEC to at least > make sure the DEC peripherals were being well handled and in mostly the > same manner. IIRC Jean spent 6-9 months embedded in the CSRG working on > much of that. > I don't have access to anything but Ultrix-11 3.1 sources. But there's no disk label stuff there. The dksizes.c file is just the individual partition sizes removed from the respective driver and added to dksize.c, presumably so it could be rebuilt easily w/o giving away any Unix IP. And the disktab stuff is way better than what came before. It enshrined into a text file what the right partitions should be for each of the devices. And even that took a lot of effort and would feed into the defaults for partitioning disks when that came along. > The early disktab stuff encoded the default tables into a text file to >> make it easier to cope with all the different types of disks, but didn't >> affect what we know as bsdlabels until after 4.3BSD. >> > Right. > > > >> The naming convention was there, but the 'write bits to the disk to >> describe partitioning' I couldn't find at all in ultrix. I think you may be >> confusing that stuff (which was a very clever way to cope with all these >> disks that have different partitioning in a increasingly generic way) with >> disk labeling which I think. >> > Quite possible. > > The key is that partitioning the disk to allow its use for different > things and disk geometry support get all mixed together in the different > schemes. As we discussed, it often happens in multiple places (since the > ROMs, like the PC's BIOS, need at least some of the info at boot), and the > loaded OS (particularly ones with multiple OSs on them) might want to do > something completely different. This is why Grant's question is a > little hard to answer directly - as I said, it is a long, slow trip. > Yes. There's all kinds of compounding factors, as we saw play out when old-school BSD disklabels started existing in a MBR world. > UNIX Partitioning, like what Dennis did, came first. If you look at > PDP-11 and later Vaxen, the "disk support" for booting is pretty crude and > built into the ROM -- *i.e*., boot RP0 or RK0 -- the "disk geometry" is > hidden in the Boot ROM. But when so many disks start to show up using the > same controllers, the ROM needs to be smarter. So, some way to encode > geometry is needed. But Partitioning for the OS is still something that is > handy and so often got put into the same support (such as the PC's BIOS > tables - which were a good idea, poorly executed). > Yes. Dennis did it in like 75, and it wasn't until the mid to late 80s that most disks had their partitioning encoded on the drive itself. Some of the encoding schemes included the geometry of the device (some implicitly, some explicitly), while others didn't. It was a real mess. > Sun was first to market with. >> > To be fair, Masscomp's "disk geometry" code that Paul Cantrell wrote > pre-dated Sun by at least a year or more. I did not include it in my > history, as it is private to their boot ROM. Nice scheme, actually - but > proprietary, and I don't think any of the ideas went anywhere else - other > than later Sun ROMs supported a similar functionality, and they would have > at least seen them at customers and known about it, particularly since Sun > picked up the Xylogic disk and tape controllers that they developed for > Masscomp originally (Paul spent many hours at Xylogics helping with their > Microcode). The point is, by that time, the proliferation of different > disk manufacturers -- something to make the boot ROMs and OS's more > independent was forced on the different systems providers if they were > going to have any chance of being able to be flexible in the market/field. > I have no data on Masscomp's / Xylogic's ROMs, alas. And I know that getting the geometry from the disk lived in the driver for a long time because of issues like this, and why bootstrapping these machines were always a complex affair. I don't recall there being any Xylogic specific labels on the drives we had at Solbourne, though. And did Masscomp's stuff write data to the media that was then used to figure out the geometry / partitioning? Or was it still a largish table of drive model X has geometry x/y/z (which PC BIOS had initially after the AT mainstreamed winchester disks there). > > >> >> Maybe you can help me find it? >> > I'll see what I can dig up. > Yea. I'd be quite keen on the details before Sun's scheme, which was the first I could find in the artifacts we have today. But there's so many artifacts they are all hard to go through. Especially scanned manuals that have no OCR encodings... I did find the SunOS 0.x images (UniSoft V7 port) didn't appear have a disk >> partition, but 1.0 (post 4BSD pre 4.1BSD) and later did. That puts it at >> 1982. Although based off BSD, the disk label stuff wasn't in BSD for >> another 5 years when the BSD folks started the Tahoe port. >> > Right, this is all of the MK0, CSRG, and Sun connection via Fred, Bill, > and Sam. At a minimum, they knew what the other was doing/had done, and > some of it definitely migrated. > What's MK0? Is that the group at DEC that did Unix? > WRT to OpenFW, my memory is that Larry points out that Sun was the primary > driver. But a lot of the Motorola club got interested in using it, too, > during the "JAWS" timeframe. I am pretty sure that there were versions for > 68K, 88K, and PPC, as well as the SPARC version from Sun. Somewhere I > have early source distribution. > > Yea, I have a memory of the Sun 3 (68k) machines having a different ROM than the Sun 4 (sparc) machines in the late 80s/early 90s with similar interfaces, but the Sun 3's being simpler. Maybe I just worked with older Sun 3's that didn't have a newer OpenFirmware. Warner [-- Attachment #2: Type: text/html, Size: 11316 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2024-01-02 18:49 ` Warner Losh @ 2024-01-02 19:30 ` Chet Ramey 2024-01-02 20:07 ` Clem Cole 2024-01-02 19:50 ` Dan Cross 1 sibling, 1 reply; 44+ messages in thread From: Chet Ramey @ 2024-01-02 19:30 UTC (permalink / raw) To: tuhs On 1/2/24 1:49 PM, Warner Losh wrote: > Right, this is all of the MK0, CSRG, and Sun connection via Fred, Bill, > and Sam. At a minimum, they knew what the other was doing/had done, > and some of it definitely migrated. > > > What's MK0? Is that the group at DEC that did Unix? I don't know what MK0 is either, but Bill Shannon and Sam Leffler had been friends since they were both grad students here at CWRU, so I would assume they kept in touch about what they were doing. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRU chet@case.edu http://tiswww.cwru.edu/~chet/ ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2024-01-02 19:30 ` Chet Ramey @ 2024-01-02 20:07 ` Clem Cole 0 siblings, 0 replies; 44+ messages in thread From: Clem Cole @ 2024-01-02 20:07 UTC (permalink / raw) To: chet.ramey; +Cc: tuhs [-- Attachment #1: Type: text/plain, Size: 1691 bytes --] On Tue, Jan 2, 2024 at 2:31 PM Chet Ramey <chet.ramey@case.edu> wrote: > On 1/2/24 1:49 PM, Warner Losh wrote: > > > Right, this is all of the MK0, CSRG, and Sun connection via Fred, > Bill, > > and Sam. At a minimum, they knew what the other was doing/had done, > > and some of it definitely migrated. > > > > > > What's MK0? Is that the group at DEC that did Unix? > DEC Merrimack NH Site - where Telephone Industries Group (TIG) and Custom Special Systems (CSS) were located. It was originally the site dedicated to supporting DEC's largest customer - AT & T. Bill Munson originally ran TIG. Armando, Jean, Shannon, Cantor, *et al.* worked in TIG. Those folks eventually moved to ZK03 - Spit Brook Road, in Nashua, with the other OS teams when Ultrix was created. Sorry, old habits are hard to change. Except for the Mill itself, many, if not most, of us from the old days still refer to the different DEC sites (particularly the original New Englands) by their site code. For instance, my old offices were in MR02 (Marlboro, MA) and ZK03. Years later, the compiler team (in ZK02) was sold twice, and they never moved their desk. Hence, when I later joined Intel, my desk was placed in ZK02 with the compiler folks like Paul W. In fact, when Intel finally moved us out, we realized that folks like Steve Hobbs had not moved his desk in 30 years. I don't know what MK0 is either, but Bill Shannon and Sam Leffler had been > friends since they were both grad students here at CWRU, so I would assume > they kept in touch about what they were doing. > Indeed, and having been friends with them both, I can verify that they did. ᐧ [-- Attachment #2: Type: text/html, Size: 3650 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2024-01-02 18:49 ` Warner Losh 2024-01-02 19:30 ` Chet Ramey @ 2024-01-02 19:50 ` Dan Cross 2024-01-02 19:55 ` Jim Capp 2024-01-02 20:30 ` Dan Cross 1 sibling, 2 replies; 44+ messages in thread From: Dan Cross @ 2024-01-02 19:50 UTC (permalink / raw) To: Warner Losh; +Cc: Grant Taylor, The Unix Heritage Society On Tue, Jan 2, 2024 at 2:00 PM Warner Losh <imp@bsdimp.com> wrote: > [snip] > Yea, I have a memory of the Sun 3 (68k) machines having a different ROM than the Sun 4 (sparc) machines in the late 80s/early 90s with similar interfaces, but the Sun 3's being simpler. Maybe I just worked with older Sun 3's that didn't have a newer OpenFirmware. Sun 3/50 and Sun 3/60 definitely had a different PROM arrangement than SPARCstations. However, I have a vague memory that the Sun 3/80 used OpenFirmware. The 3/80 (and similar Sun-3x machines introduced at the same time) would have been among the last, if not the last, MC68k-based Sun3 machine. I may be wrong, though; it's been a while. Regardless, I vaguely remember the "ok" prompt on a 3/80. - Dan C. ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2024-01-02 19:50 ` Dan Cross @ 2024-01-02 19:55 ` Jim Capp 2024-01-02 20:11 ` Dan Cross 2024-01-02 20:30 ` Dan Cross 1 sibling, 1 reply; 44+ messages in thread From: Jim Capp @ 2024-01-02 19:55 UTC (permalink / raw) To: Dan Cross; +Cc: Grant Taylor, The Unix Heritage Society [-- Attachment #1: Type: text/plain, Size: 1180 bytes --] The "ok" prompt is from "OpenBoot", a boot manager, written in FORTH: https://docs.oracle.com/cd/E19253-01/816-1177-10/816-1177-10.pdf Jim From: "Dan Cross" <crossd@gmail.com> To: "Warner Losh" <imp@bsdimp.com> Cc: "Grant Taylor" <gtaylor@tnetconsulting.net>, "The Unix Heritage Society" <tuhs@tuhs.org> Sent: Tuesday, January 2, 2024 2:50:52 PM Subject: [TUHS] Re: Question about BSD disklabel history On Tue, Jan 2, 2024 at 2:00 PM Warner Losh <imp@bsdimp.com> wrote: > [snip] > Yea, I have a memory of the Sun 3 (68k) machines having a different ROM than the Sun 4 (sparc) machines in the late 80s/early 90s with similar interfaces, but the Sun 3's being simpler. Maybe I just worked with older Sun 3's that didn't have a newer OpenFirmware. Sun 3/50 and Sun 3/60 definitely had a different PROM arrangement than SPARCstations. However, I have a vague memory that the Sun 3/80 used OpenFirmware. The 3/80 (and similar Sun-3x machines introduced at the same time) would have been among the last, if not the last, MC68k-based Sun3 machine. I may be wrong, though; it's been a while. Regardless, I vaguely remember the "ok" prompt on a 3/80. - Dan C. [-- Attachment #2: Type: text/html, Size: 2882 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2024-01-02 19:55 ` Jim Capp @ 2024-01-02 20:11 ` Dan Cross 0 siblings, 0 replies; 44+ messages in thread From: Dan Cross @ 2024-01-02 20:11 UTC (permalink / raw) To: Jim Capp; +Cc: Grant Taylor, The Unix Heritage Society On Tue, Jan 2, 2024 at 2:55 PM Jim Capp <jcapp@anteil.com> wrote: > The "ok" prompt is from "OpenBoot", Yes. See Phil Bunde's message earlier in this thread. - Dan C. > a boot manager, written in FORTH: > > https://docs.oracle.com/cd/E19253-01/816-1177-10/816-1177-10.pdf ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2024-01-02 19:50 ` Dan Cross 2024-01-02 19:55 ` Jim Capp @ 2024-01-02 20:30 ` Dan Cross 2024-01-02 20:50 ` Clem Cole 1 sibling, 1 reply; 44+ messages in thread From: Dan Cross @ 2024-01-02 20:30 UTC (permalink / raw) To: Warner Losh; +Cc: Grant Taylor, The Unix Heritage Society On Tue, Jan 2, 2024 at 2:50 PM Dan Cross <crossd@gmail.com> wrote: > On Tue, Jan 2, 2024 at 2:00 PM Warner Losh <imp@bsdimp.com> wrote: > > [snip] > > Yea, I have a memory of the Sun 3 (68k) machines having a different ROM than the Sun 4 (sparc) machines in the late 80s/early 90s with similar interfaces, but the Sun 3's being simpler. Maybe I just worked with older Sun 3's that didn't have a newer OpenFirmware. > > Sun 3/50 and Sun 3/60 definitely had a different PROM arrangement than > SPARCstations. However, I have a vague memory that the Sun 3/80 used > OpenFirmware. The 3/80 (and similar Sun-3x machines introduced at the > same time) would have been among the last, if not the last, > MC68k-based Sun3 machine. I may be wrong, though; it's been a while. > Regardless, I vaguely remember the "ok" prompt on a 3/80. Hmm: correcting myself here. I can find no evidence that the 3/80 used OBP, but I can find documentation to the contrary (mostly in the form of documentation on how to rewrite the HostID when the NVRAM battery fails...Gah, the things one not only forgets but actively blots out of one's memory). It seems likely that Sun never shipped OF on 68k. - Dan C. ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2024-01-02 20:30 ` Dan Cross @ 2024-01-02 20:50 ` Clem Cole 2024-01-02 21:04 ` Dan Cross 0 siblings, 1 reply; 44+ messages in thread From: Clem Cole @ 2024-01-02 20:50 UTC (permalink / raw) To: Dan Cross; +Cc: Grant Taylor, The Unix Heritage Society [-- Attachment #1: Type: text/plain, Size: 1624 bytes --] I never said Sun or anyone else actually shipped it -- only that it was around, and there was a lot of discussion about using it/trying to create a boot ROM standard one of some type. IIRC I remember that NCR was messing with it for their 88K boxes (which I had a hand in killing). But NCR switched to all x86-based (their "7 levels "strategy) for everything from Cash Registers/ATMs to their largest Database machines. They had always been in the "better and cheaper than IBM" business, particularly in their computer products. [1] So after they switch, their boots start to be all BIOS and EBIOS flavored. But other Motorola folks I was working with at the time were exploring OF. I had it on a 68K and another 88K prototype machine at one point (I remember the "ok" prompt), and I remember I somehow had a source distribution tape. I was an independent consultant then and wrote a couple of drivers under contract for some different folks. I can not think of the firm's name now -- there was another 68K workstation firm here in Lowell, MA. They actually wrote and made a MacOS emulator work with their UNIX under covers. I've forgotten most of the details now, but they may have been one of the folks who was playing with OF. My memory is that I first encountered it at one of my consulting gigs. Clem 1] When they threw the x86 everywhere die, they also started using Microchannel (the only large licensee I know of for it -- everyone else stayed with ISA/EISA and then later PCI from DEC/AMD) - tossing out all the Moto IP like VME that so many UNIX systems had used). ᐧ [-- Attachment #2: Type: text/html, Size: 3198 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2024-01-02 20:50 ` Clem Cole @ 2024-01-02 21:04 ` Dan Cross 0 siblings, 0 replies; 44+ messages in thread From: Dan Cross @ 2024-01-02 21:04 UTC (permalink / raw) To: Clem Cole; +Cc: Grant Taylor, The Unix Heritage Society On Tue, Jan 2, 2024 at 3:50 PM Clem Cole <clemc@ccc.com> wrote: > I never said Sun or anyone else actually shipped it -- only that it was around, and there was a lot of discussion about using it/trying to create a boot ROM standard one of some type. Sorry! I didn't mean to imply that you had; rather, I thought that they had because I thought I remembered seeing it on Sun-3x, but I think I'm just misremembering. > IIRC I remember that NCR was messing with it for their 88K boxes (which I had a hand in killing). But NCR switched to all x86-based (their "7 levels "strategy) for everything from Cash Registers/ATMs to their largest Database machines. They had always been in the "better and cheaper than IBM" business, particularly in their computer products. [1] So after they switch, their boots start to be all BIOS and EBIOS flavored. Poor suckers. :-( > But other Motorola folks I was working with at the time were exploring OF. I had it on a 68K and another 88K prototype machine at one point (I remember the "ok" prompt), and I remember I somehow had a source distribution tape. I was an independent consultant then and wrote a couple of drivers under contract for some different folks. I can not think of the firm's name now -- there was another 68K workstation firm here in Lowell, MA. They actually wrote and made a MacOS emulator work with their UNIX under covers. I've forgotten most of the details now, but they may have been one of the folks who was playing with OF. > > My memory is that I first encountered it at one of my consulting gigs. That's cool. It's sad that, around the time of the 386, we didn't have more x86 vendors doing something like OF. Maybe we could have avoided a bunch of big messes. Maybe not. > 1] When they threw the x86 everywhere die, they also started using Microchannel (the only large licensee I know of for it -- everyone else stayed with ISA/EISA and then later PCI from DEC/AMD) - tossing out all the Moto IP like VME that so many UNIX systems had used). I knew a bunch of folks in the early/mid 90s who were pretty tight in the IBM camp; RS/6000's were shipping on MCA and they said they saw better IO performance than with PCI for a couple of generations. - Dan C. ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2023-12-31 21:31 ` Clem Cole 2023-12-31 22:07 ` Warner Losh @ 2023-12-31 22:46 ` G. Branden Robinson 2023-12-31 23:06 ` Larry McVoy ` (2 more replies) 1 sibling, 3 replies; 44+ messages in thread From: G. Branden Robinson @ 2023-12-31 22:46 UTC (permalink / raw) To: The Unix Heritage Society [-- Attachment #1: Type: text/plain, Size: 3439 bytes --] At 2023-12-31T16:31:00-0500, Clem Cole wrote: > Next (ney Apple) started with the Mach code base from CMU. There was > a push in the Valley in those days for something called OpenFirmware > [Warner help here -- I think that it was forth based IIRC, and Sun may > have had their hand in it also]. I'm not Warner but I owned and operated a few OpenFirmware based machines. > But the key is that it ran on 68K's. I don't think that's the case. OpenFirmware (OF) ran on SPARC and PowerPC hardware, at least. And since it was indeed Forth-based, in principle it could have been ported practically anywhere (assuming memory requirements for OF itself were met). The m68k "Old World" PowerMacs used a different firmware entirely; I assume boot ROM code descended from the original Macintosh (or even Lisa, maybe). The PowerPC "New World" PowerMacs, which immediately departed from the beige color scheme, did come in with Apple's acquisition of NeXT. This may have been the last good thing that Steve Jobs had a hand in. On Sun SPARC machines you could get to an OF prompt at any time by typing "Stop+A" (a.k.a. "L1+A"), using one of the funny left-hand side function keys on a generation or two (Type 4??) Sun keyboards. This was a lot like the "programmer's switch" on some m68k Macs, which was wired directly to an NMI that MacOS ("Classic") had a fixed vector for. Open Firmware was an excellent idea; true peripheral portability was achieved by having "option ROMs" on devices that needed them implemented in Forth like OF itself. A lot of flexibility here. I don't know how much of a performance price was paid--people did and do enjoy getting into bootup-time races--but even if it were competitive, the PC side of the industry would have stridently claimed it wasn't. A lot of non-x86 Cisco kit in the 2000s (some PowerPC, and _maybe_ some MIPS stuff too(?)) used some kind of cut-down descendant of OF called OpenHackWare. I'm not sure what its dimensions were; it may have mostly been just the conventions and format that we now recognize as "device tree" (DTS/DTB), which has perhaps been OpenFirmware's proudest legacy. It beat the hell out of the PC BIOS alternative for device enumeration, which always appeared to me to be pure binary chaos with no standard apart from whatever Microsoft wanted for Windows. By contrast, OpenFirmware was standardized in IEEE 1275. > By the time the Intel Mac's BIOS had begun to be replaced in the > WINTEL world by UEFI, Apple committed to using a flavor of it. Originally EFI, without the "U". I first saw these on first-generation Itanium machines--huge, hot deskside HP boxes whose innards appeared to use composite foam like that from aircraft wings for heat piping. The company I worked for was contracted to help achieve the Linux ia64 port. I was immediately horrified by EFI's huge step backwards in concept and implementation. All this technical progress just to return to unportable device driver ROMs and a C:\ prompt. The hatred of the Wintel duopoly toward elegance or cleanliness in any aspect of computing cannot be overstated. I'm weakly hopeful that the RISC-V community will rediscover OpenFirmware. It has to date had the good sense to avoid UEFI. A wise choice, as if they they hand Intel that much control over their ecosystem, they will surrender all independence, and possibly the architecture itself, at least in real silicon. Regards, Branden [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2023-12-31 22:46 ` G. Branden Robinson @ 2023-12-31 23:06 ` Larry McVoy 2023-12-31 23:37 ` Al Kossow ` (2 more replies) 2023-12-31 23:08 ` Phil Budne 2023-12-31 23:50 ` G. Branden Robinson 2 siblings, 3 replies; 44+ messages in thread From: Larry McVoy @ 2023-12-31 23:06 UTC (permalink / raw) To: G. Branden Robinson; +Cc: The Unix Heritage Society On Sun, Dec 31, 2023 at 04:46:49PM -0600, G. Branden Robinson wrote: > At 2023-12-31T16:31:00-0500, Clem Cole wrote: > > Next (ney Apple) started with the Mach code base from CMU. There was > > a push in the Valley in those days for something called OpenFirmware > > [Warner help here -- I think that it was forth based IIRC, and Sun may > > have had their hand in it also]. > > I'm not Warner but I owned and operated a few OpenFirmware based > machines. > > > But the key is that it ran on 68K's. > > I don't think that's the case. OpenFirmware (OF) ran on SPARC and > PowerPC hardware, at least. And since it was indeed Forth-based, in > principle it could have been ported practically anywhere (assuming > memory requirements for OF itself were met). OpenFirmware is Mitch Bradley's baby. I believe it ran on 68k Suns, there was some sort of boot prom there. I mostly used it on SPARC. It was pretty powerful but my personal feeling is the choice of Forth didn't help. Yeah, I get it, Forth is like some weird lisp and the lisp people love lisp. What the lisp people don't get is there are a lot more people who don't love lisp than do love lisp. And trying to get everyone to love lisp isn't gonna happen. That said, what else could Mitch have used at the time? Tcl? Please, another weird lisp. Perl? Not really something that wants to talk to the bare metal. It's a serious question, is there anything that Mitch could have used that would have had wider appeal? And I agree whole heartedly with the EFI crap being a giant step backwards. ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2023-12-31 23:06 ` Larry McVoy @ 2023-12-31 23:37 ` Al Kossow 2023-12-31 23:41 ` Alec Muffett 2024-01-02 20:48 ` Dan Cross 2 siblings, 0 replies; 44+ messages in thread From: Al Kossow @ 2023-12-31 23:37 UTC (permalink / raw) To: tuhs On 12/31/23 3:06 PM, Larry McVoy wrote: > OpenFirmware is Mitch Bradley's baby. I believe it ran on 68k Suns, > there was some sort of boot prom there. I mostly used it on SPARC. It was developed for SPARC and never ran on Sun-3's. The point of it was to be processor independent. I was involved in bringing it up on Power Macs. It was championed by Ron Hochsprung there, who is also a FORTH fan. In theory, you should have been able to use PCI expansion cards at boot time across vendors by using it, but in practice that didn't happen because very few vendors ever bothered to support anything in their BIOS ROMs other than the PC. And when Apple went with x86 they used the PC's way of the world, like everyone else. ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2023-12-31 23:06 ` Larry McVoy 2023-12-31 23:37 ` Al Kossow @ 2023-12-31 23:41 ` Alec Muffett 2024-01-02 20:48 ` Dan Cross 2 siblings, 0 replies; 44+ messages in thread From: Alec Muffett @ 2023-12-31 23:41 UTC (permalink / raw) To: Larry McVoy; +Cc: The Unix Heritage Society [-- Attachment #1: Type: text/plain, Size: 548 bytes --] On Sun, 31 Dec 2023, 23:25 Larry McVoy, <lm@mcvoy.com> wrote: > Yeah, I get it, Forth is like some weird lisp > and the lisp people love lisp. > HERESY! Forth is a RPN stack language. It's closer to Java bytecode in a whole bunch of ways, but its lineage comes squarely out of embedded systems like radio telescope control. It's a solid choice. Funny I was only posting about it earlier today in respective of the recent iPhone privilege escalation / exploit: https://alecmuffett.com/article/108789 ...which links to some Forth examples. -a [-- Attachment #2: Type: text/html, Size: 1393 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2023-12-31 23:06 ` Larry McVoy 2023-12-31 23:37 ` Al Kossow 2023-12-31 23:41 ` Alec Muffett @ 2024-01-02 20:48 ` Dan Cross 2024-01-02 21:17 ` John Cowan 2024-01-03 3:33 ` Theodore Ts'o 2 siblings, 2 replies; 44+ messages in thread From: Dan Cross @ 2024-01-02 20:48 UTC (permalink / raw) To: Larry McVoy; +Cc: The Unix Heritage Society On Sun, Dec 31, 2023 at 6:25 PM Larry McVoy <lm@mcvoy.com> wrote: >[snip] > OpenFirmware is Mitch Bradley's baby. I believe it ran on 68k Suns, > there was some sort of boot prom there. I mostly used it on SPARC. > It was pretty powerful but my personal feeling is the choice of > Forth didn't help. Yeah, I get it, Forth is like some weird lisp > and the lisp people love lisp. What the lisp people don't get is > there are a lot more people who don't love lisp than do love lisp. > And trying to get everyone to love lisp isn't gonna happen. > > That said, what else could Mitch have used at the time? Tcl? > Please, another weird lisp. Perl? Not really something that > wants to talk to the bare metal. > > It's a serious question, is there anything that Mitch could have > used that would have had wider appeal? The thing about FORTH isn't that it's Lisp-like (as Alec mentioned), though its supporters do often exhibit a fervor reminiscent of Lispers. Rather, I think FORTH shows up in places like this because it's possible to write _incredibly_ lean threaded-code interpreters for it that can run in really primitive environments, so you can shove a really small interpreter in a ROM and keep your big CPU in reset while you run it out of a tiny SRAM on an 8-bit microcontroller or something until you've got enough of an environment going to train DRAM and transfer over to the real thing. E.g., something like: https://pygmy.utoh.org/3ins4th.html What could you have done differently? Meh; I don't really know, but see below. > And I agree whole heartedly with the EFI crap being a giant step > backwards. Ironically, the UEFI people have done something _similar_ to OF in the form of AML (ACPI Machine Language), which is a byte-code serialization ASL (ACPI Source Language); presumably that's system independent. The idea of a p-code representation is about where the similarity ends, though: AML exposes a mechanism to talk to the UEFI OS for a whole slew of stuff, which is rather unlike what OF did (though I again have a vague memory that on SPARCstations some devices went through the PROM monitor; the text console, for example, and maybe the keyboard? It's been too long now to properly remember). Anyway, an alternative to FORTH might have been a well-defined p-code and a little VM in ROM to drive it. Then one could compile to that using whatever language one liked (and was willing to write a compiler for!). Perhaps the feeling is that that is what FORTH was; for that I guess I don't see any reason one couldn't transpile to FORTH from some other language. - Dan C. ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2024-01-02 20:48 ` Dan Cross @ 2024-01-02 21:17 ` John Cowan 2024-01-03 3:33 ` Theodore Ts'o 1 sibling, 0 replies; 44+ messages in thread From: John Cowan @ 2024-01-02 21:17 UTC (permalink / raw) To: Dan Cross; +Cc: The Unix Heritage Society [-- Attachment #1: Type: text/plain, Size: 733 bytes --] On Tue, Jan 2, 2024 at 3:49 PM Dan Cross <crossd@gmail.com> wrote: Perhaps the feeling is that that is what FORTH was; for that I > guess I don't see any reason one couldn't transpile to FORTH from some > other language. Henry Baker wrote a beautiful little paper (see https://www.plover.com/~mjd/misc/hbaker-archive/ForthStack.html>) showing the mapping between Linear Lisp (a Lisp in which all variables are referenced once and only once) and a Forth-style "frameless stack". There is also a discussion of Forth as a set of linear combinators: Manfred von Thun's Joy <https://en.wikipedia.org/wiki/Joy_(programming_language)> is a Forth-like using nested lists in which the stack is the top-level list. > > [-- Attachment #2: Type: text/html, Size: 1708 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2024-01-02 20:48 ` Dan Cross 2024-01-02 21:17 ` John Cowan @ 2024-01-03 3:33 ` Theodore Ts'o 2024-01-03 3:57 ` Warner Losh 2024-01-03 14:39 ` Dan Cross 1 sibling, 2 replies; 44+ messages in thread From: Theodore Ts'o @ 2024-01-03 3:33 UTC (permalink / raw) To: Dan Cross; +Cc: The Unix Heritage Society On Tue, Jan 02, 2024 at 03:48:28PM -0500, Dan Cross wrote: > On Sun, Dec 31, 2023 at 6:25 PM Larry McVoy <lm@mcvoy.com> wrote: > > Ironically, the UEFI people have done something _similar_ to OF in the > form of AML (ACPI Machine Language), which is a byte-code > serialization ASL (ACPI Source Language); presumably that's system > independent. The idea of a p-code representation is about where the > similarity ends, though: AML exposes a mechanism to talk to the UEFI > OS for a whole slew of stuff, which is rather unlike what OF did ACPI (Advanced Configuration and Power Interface) predates UEFI. ACPI was released in 1996, and was originally intended to be a replacement for APM (Advanced Power Management). With APM, the OS would crowbar itself into x86 Real mode in order to call into the APM BIOS in order to suspend the laptop, fetch power management events from the APM, etc. Later on, APM grew 32-bit protected mode interfaces, but effectively, the OS would completely lose control of the thread of execution while running APM BIOS code, which made life "exciting" for OS engineers. So ACPI was originally intended to solve a problem, where the OS could incorporate a byte code interpreter to do those things that would be very hardware specific (for example, how to diddle the various random I/O ports on a HP laptop to bring the perpipherals into a low power state, which of course would be completely different on a Dell or IBM laptop. Previously, the APM bios was the abstraction layer; ACPI was the new abstraction layer. UEFI came later. UEFI was the second system disease replacement for EFI (extensible firmware interface), which Intel had developed. EFI was an implementation that attempted to retain full backwards compatibility with the de factor standard originally established by the IBM BIOS. UEFI was an attempt to completely rework the interface between the OS boot code (which before would talk to the BIOS interface) with the all new singing, all dancing UEFI interface. Being a second system, of course it was made horribly complicated, so as to meet all of the requirements that might be dreamed up by the industry committee that put it together. Normally, once the OS has gotten to a certain point in its OS initialization, the OS can "terminate UEFI services". At that point, the OS can no longer call into the UEFI functions --- but it also doesn't have to worry about the UEFI code servicing an interrupt take taking control of the CPU away from the OS. However, the OS can still execute various ACPI functions, assuming the OS has its own AML interpreter. (Of course, the UEFI BIOS has its own AML interpreter --- and scheduler --- and everything else that a simple OS might have.) Ron Minnich has a number of really good presentations about the "joys" of working with UEFI. See the YouTube video, "Replace your Expoit-ridden Firmware with Linux"[1] and prepare to be horrified about what horrors lurks in the heart of UEFI.... [1] https://www.youtube.com/watch?v=iffTJ1vPCSo So you can very much be *not* using UEFI, and still be using ACPI --- either if you are using a pre-2004 computer, or if you are using a more modern platform which uses LinuxBoot. - Ted ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2024-01-03 3:33 ` Theodore Ts'o @ 2024-01-03 3:57 ` Warner Losh 2024-01-03 4:03 ` Warner Losh 2024-01-03 4:30 ` Theodore Ts'o 2024-01-03 14:39 ` Dan Cross 1 sibling, 2 replies; 44+ messages in thread From: Warner Losh @ 2024-01-03 3:57 UTC (permalink / raw) To: Theodore Ts'o; +Cc: The Unix Heritage Society [-- Attachment #1: Type: text/plain, Size: 6609 bytes --] On Tue, Jan 2, 2024 at 8:33 PM Theodore Ts'o <tytso@mit.edu> wrote: > On Tue, Jan 02, 2024 at 03:48:28PM -0500, Dan Cross wrote: > > On Sun, Dec 31, 2023 at 6:25 PM Larry McVoy <lm@mcvoy.com> wrote: > > > > Ironically, the UEFI people have done something _similar_ to OF in the > > form of AML (ACPI Machine Language), which is a byte-code > > serialization ASL (ACPI Source Language); presumably that's system > > independent. The idea of a p-code representation is about where the > > similarity ends, though: AML exposes a mechanism to talk to the UEFI > > OS for a whole slew of stuff, which is rather unlike what OF did > > ACPI (Advanced Configuration and Power Interface) predates UEFI. ACPI > was released in 1996, and was originally intended to be a replacement > for APM (Advanced Power Management). With APM, the OS would crowbar > itself into x86 Real mode in order to call into the APM BIOS in order > to suspend the laptop, fetch power management events from the APM, > etc. Later on, APM grew 32-bit protected mode interfaces, but > effectively, the OS would completely lose control of the thread of > execution while running APM BIOS code, which made life "exciting" for > OS engineers. > > So ACPI was originally intended to solve a problem, where the OS could > incorporate a byte code interpreter to do those things that would be > very hardware specific (for example, how to diddle the various random > I/O ports on a HP laptop to bring the perpipherals into a low power > state, which of course would be completely different on a Dell or IBM > laptop. Previously, the APM bios was the abstraction layer; ACPI was > the new abstraction layer. > > UEFI came later. UEFI was the second system disease replacement for > EFI (extensible firmware interface), which Intel had developed. EFI > was an implementation that attempted to retain full backwards > compatibility with the de factor standard originally established by > the IBM BIOS. UEFI was an attempt to completely rework the interface > between the OS boot code (which before would talk to the BIOS > interface) with the all new singing, all dancing UEFI interface. > Being a second system, of course it was made horribly complicated, so > as to meet all of the requirements that might be dreamed up by the > industry committee that put it together. > Not quite. EFI was the second system. It didn't try to retain any backwards compatibility with the original IBM BIOS. It was completely different. Early Intel macs used it to boot. Intel and Apple worked on it together. Intel wrote it for the Itanium IA64 fiasco because it couldn't use the IBM BIOS interfaces and such to boot its new machine because IA64 lacked many of the low-level tricks that you used to interface. to the BIOS. It was an attempt to redo things from scratch. Intel did then turn it over to something more public to manage, but it was well on its way to being an insane mess before it morphed into UEFI. But even in EFI times it was clearly on the way to being the trainwreck it became. > Normally, once the OS has gotten to a certain point in its OS > initialization, the OS can "terminate UEFI services". At that point, > the OS can no longer call into the UEFI functions --- but it also > doesn't have to worry about the UEFI code servicing an interrupt take > taking control of the CPU away from the OS. > There's a 'however' here. The OS may call UEFI Runtime Services from time to time (to interact with non-volatile variables), and it has to carefully arrange things to do this (it has to swap in a special memory map, etc). So UEFI isn't completely gone away.. You just don't have to worry about it fielding interrupts after you exit boot services. > However, the OS can still execute various ACPI functions, assuming the > OS has its own AML interpreter. (Of course, the UEFI BIOS has its own > AML interpreter --- and scheduler --- and everything else that a > simple OS might have.) Ron Minnich has a number of really good > presentations about the "joys" of working with UEFI. See the YouTube > video, "Replace your Expoit-ridden Firmware with Linux"[1] and prepare to > be horrified about what horrors lurks in the heart of UEFI.... > > [1] https://www.youtube.com/watch?v=iffTJ1vPCSo > > So you can very much be *not* using UEFI, and still be using ACPI --- > either if you are using a pre-2004 computer, or if you are using a > more modern platform which uses LinuxBoot. > Indeed. I got to deal with all of that, and more. I have finished writing LinuxBoot support for FreeBSD. The normal kexec-tools, u-root, etc aren't sufficient for FreeBSD because FreeBSD's kernel expects the boot loader to setup a number of meta-data items that go with the kernel that include all the information about the system that the kernel simply can't get once you've entered long mode... Even with LinuxBoot, you are booting with UEFI, albeit with a much small much smaller UEFI. The PEI is still there (the pre-efi initialization phase: that runs the CPU vendor's binary blobs, and does other baisc bringup of the machine). In addition, UEFI's runtime services remain. Linux's EFI capsule replaces almost all fo the DEX drivers, though, which is a huge win It's a stub that acts as the first EFI program that runs before UEFI gets too far along (MDS on x86, Shell,efi on all the others). It completely takes over, modulo the callbacks to Runtime Services. Why am I on about Runtime Services so much? That was one of the pain points of my port. ExitBootServices had been called, and SetVirtualMapping had been as well by the time my boot loader got to run. So I had to pass all that into FreeBSD's kernel (along with an interesting amount of metadata to work around chip errata) and the FreeBSD kernel expected 1:1 PA:VA mapping, but the Linux kernel doesn't do that and if I want to run there I have to cope. Thankfully, the kernel only needed one bug fix and two asserts removed to work... So even with LinuxBoot you still have a tiny bit of UEFI left behind... Though the huge ugly mess of it finding what to boot, executing another boot loader, and that loading the kernel is thankfully gone. My BSDcan talk, my EuroBSDcon talk and a forthcoming article on booting FreeBSD in a LinuxBoot environment are all on this. https://2023.eurobsdcon.org/slides/eurobsdcon2023-warner_losh-kboot.pdf are the slides, but the videos aren't up for some reason.... It was very eye-opening doing this work... Warner [-- Attachment #2: Type: text/html, Size: 8240 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2024-01-03 3:57 ` Warner Losh @ 2024-01-03 4:03 ` Warner Losh 2024-01-03 4:30 ` Theodore Ts'o 1 sibling, 0 replies; 44+ messages in thread From: Warner Losh @ 2024-01-03 4:03 UTC (permalink / raw) To: Theodore Ts'o; +Cc: The Unix Heritage Society [-- Attachment #1: Type: text/plain, Size: 287 bytes --] On Tue, Jan 2, 2024 at 8:57 PM Warner Losh <imp@bsdimp.com> wrote: > . It didn't try to retain any backwards > compatibility with the original IBM BIOS. > I should have added... Later, UEFI also added CSM booting mode to boot legacy mode that was a bit compatible. Warner [-- Attachment #2: Type: text/html, Size: 713 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2024-01-03 3:57 ` Warner Losh 2024-01-03 4:03 ` Warner Losh @ 2024-01-03 4:30 ` Theodore Ts'o 2024-01-03 5:10 ` Warner Losh 1 sibling, 1 reply; 44+ messages in thread From: Theodore Ts'o @ 2024-01-03 4:30 UTC (permalink / raw) To: Warner Losh; +Cc: The Unix Heritage Society On Tue, Jan 02, 2024 at 08:57:34PM -0700, Warner Losh wrote: > > Indeed. I got to deal with all of that, and more. I have finished writing > LinuxBoot support for FreeBSD. The normal kexec-tools, u-root, etc aren't > sufficient for FreeBSD because FreeBSD's kernel expects the boot loader > to setup a number of meta-data items that go with the kernel that include > all the information about the system that the kernel simply can't get once > you've entered long mode... > > Even with LinuxBoot, you are booting with UEFI, albeit with a much small > much smaller UEFI. Yeah, one of the older names of LinuxBoot was NERF (Non-Extensible Reduced Firmware). I was confusing LinuxBoot with coreboot, which is used on all ChromeOS devices after 2012, and which completely doesn't use any magic binary blobs supplied by the mainbord vendor. The tradeoff is that coreboot only supports a very restricted set of hardware, since it has to do all of the things that are "normally" done by the vendor's binary blobs to initialize the hardware devices, etc. This only works if you have very tight control over hardware, and you have enough influence that you can lean on the mainboard vendors to allow the low-level programming details of their devices to be released in open source code which that can be independently verified and digitally signed by the OS vendor (such as Google in the case of ChromeOS). Many hyper-scale cloud companies will tend to use coreboot or related software instead of UEFI. A public/published example of this is Facebook's Open Compute Project. It *is* nice not to have to deal with UEFI at all, if you're lucky enough to be able to use hardware where it's not necessary.... Of course you won't be able to run Windows on those systems, but some would consider that a feature. :-) - Ted ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2024-01-03 4:30 ` Theodore Ts'o @ 2024-01-03 5:10 ` Warner Losh 2024-01-03 15:56 ` Dan Cross 0 siblings, 1 reply; 44+ messages in thread From: Warner Losh @ 2024-01-03 5:10 UTC (permalink / raw) To: Theodore Ts'o; +Cc: The Unix Heritage Society [-- Attachment #1: Type: text/plain, Size: 3245 bytes --] On Tue, Jan 2, 2024 at 9:30 PM Theodore Ts'o <tytso@mit.edu> wrote: > On Tue, Jan 02, 2024 at 08:57:34PM -0700, Warner Losh wrote: > > > > Indeed. I got to deal with all of that, and more. I have finished writing > > LinuxBoot support for FreeBSD. The normal kexec-tools, u-root, etc aren't > > sufficient for FreeBSD because FreeBSD's kernel expects the boot loader > > to setup a number of meta-data items that go with the kernel that include > > all the information about the system that the kernel simply can't get > once > > you've entered long mode... > > > > Even with LinuxBoot, you are booting with UEFI, albeit with a much small > > much smaller UEFI. > > Yeah, one of the older names of LinuxBoot was NERF (Non-Extensible > Reduced Firmware). I love the secondary meaning for nerf from gaming too: "to reconfigure (an existing character or weapon), making it less powerful." > I was confusing LinuxBoot with coreboot, which is > used on all ChromeOS devices after 2012, and which completely doesn't > use any magic binary blobs supplied by the mainbord vendor. The > tradeoff is that coreboot only supports a very restricted set of > hardware, since it has to do all of the things that are "normally" > done by the vendor's binary blobs to initialize the hardware devices, > etc. This only works if you have very tight control over hardware, > and you have enough influence that you can lean on the mainboard > vendors to allow the low-level programming details of their devices to > be released in open source code which that can be independently > verified and digitally signed by the OS vendor (such as Google in the > case of ChromeOS). > I think that some of the open laptops might have a binary blob or two buried deep in their coreboot implementation. But it is far more open system than EDK2 is... Most of the open parts of EDK2 are what are nerfed by LinuxBoot. The private hidden ones are all that remain, alas. So if you are large enough, you can get all the sources. If not, then you have to put up with the binary blob, or not run on some computers. :(. There's also a rewrite of coreboot in Rust called 'oreboot' which is coreboot without the 'C', since they want it to be even safer. Many hyper-scale cloud companies will tend to use coreboot or related > software instead of UEFI. A public/published example of this is > Facebook's Open Compute Project. > Yes. It was a concerted effort of all these stakeholders that got the Linux kernel so it could tolerate such an early handoff. > It *is* nice not to have to deal with UEFI at all, if you're lucky > enough to be able to use hardware where it's not necessary.... Of > course you won't be able to run Windows on those systems, but some > would consider that a feature. :-) > Yea... There's several experimental attempts at booting windows on at least the UEFI-remnant machines. And it's (a) too F'ing weird and (b) to far afield to get into here... https://www.osfc.io/2019/talks/booting-windows-on-linuxboot/ has all the horrors of one such attempt. I think this is the 'create an exec handler in Linux that can run .efi programs and emulate UEFI' version... Warner [-- Attachment #2: Type: text/html, Size: 4550 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2024-01-03 5:10 ` Warner Losh @ 2024-01-03 15:56 ` Dan Cross 2024-01-03 16:37 ` Theodore Ts'o 0 siblings, 1 reply; 44+ messages in thread From: Dan Cross @ 2024-01-03 15:56 UTC (permalink / raw) To: Warner Losh; +Cc: The Unix Heritage Society On Wed, Jan 3, 2024 at 12:10 AM Warner Losh <imp@bsdimp.com> wrote: > On Tue, Jan 2, 2024 at 9:30 PM Theodore Ts'o <tytso@mit.edu> wrote: >> [snip] >> I was confusing LinuxBoot with coreboot, which is >> used on all ChromeOS devices after 2012, and which completely doesn't >> use any magic binary blobs supplied by the mainbord vendor. The >> tradeoff is that coreboot only supports a very restricted set of >> hardware, since it has to do all of the things that are "normally" >> done by the vendor's binary blobs to initialize the hardware devices, >> etc. This only works if you have very tight control over hardware, >> and you have enough influence that you can lean on the mainboard >> vendors to allow the low-level programming details of their devices to >> be released in open source code which that can be independently >> verified and digitally signed by the OS vendor (such as Google in the >> case of ChromeOS). > > I think that some of the open laptops might have a binary blob or two > buried deep in their coreboot implementation. But it is far more open > system than EDK2 is... Most of the open parts of EDK2 are what are nerfed > by LinuxBoot. The private hidden ones are all that remain, alas. Sadly the situation is even more complex than this. Consider AMD's EPYC processors: before the x86 cores start up, the PSP (Platform Security Processor) starts up and does a lot of pre-pre-initialization: it does DRAM timing training, for instance. It's also responsible for loading the x86 payload out of the local flash and setting up the x86 environment so that when those cores come out of reset, they're running whatever was loaded (for instance, they can load %cs on the BSC so that it starts somewhere other than the architecturally-defined segment right below 4GiB). While cool in some ways ("I don't have to train DRAM? Score!") the PSP is embedded in the SoC and the firmware is a signed blob you get from AMD. I know there's an ARM Cortex-A5 in there, but don't know much more about it and even if I did, I have no way to generate signed images for it. :-/ The point is, even if you've got a completely open stack running on x86 from the reset vector, there's almost certainly something else somewhere that's not open (yet). > So if you are large enough, you can get all the sources. If not, then you > have to put up with the binary blob, or not run on some computers. :(. Yup. :-( - Dan C. ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2024-01-03 15:56 ` Dan Cross @ 2024-01-03 16:37 ` Theodore Ts'o 2024-01-03 16:41 ` Dan Cross 2024-01-04 8:42 ` arnold 0 siblings, 2 replies; 44+ messages in thread From: Theodore Ts'o @ 2024-01-03 16:37 UTC (permalink / raw) To: Dan Cross; +Cc: The Unix Heritage Society On Wed, Jan 03, 2024 at 10:56:14AM -0500, Dan Cross wrote: > Sadly the situation is even more complex than this. > > Consider AMD's EPYC processors: before the x86 cores start up, the PSP > (Platform Security Processor) starts up and does a lot of > pre-pre-initialization: it does DRAM timing training, for instance. > It's also responsible for loading the x86 payload out of the local > flash and setting up the x86 environment so that when those cores come > out of reset, they're running whatever was loaded (for instance, they > can load %cs on the BSC so that it starts somewhere other than the > architecturally-defined segment right below 4GiB). While cool in some > ways ("I don't have to train DRAM? Score!") the PSP is embedded in the > SoC and the firmware is a signed blob you get from AMD. I know there's > an ARM Cortex-A5 in there, but don't know much more about it and even > if I did, I have no way to generate signed images for it. :-/ > > The point is, even if you've got a completely open stack running on > x86 from the reset vector, there's almost certainly something else > somewhere that's not open (yet). Or there's something running on a completely different x86 core with unpatched securiy bugs in the Minix and Apache cores that you can't even disable (unless you are the National Security Agency).... Sadly, Intel refuses to make it available the magic bits to disable the Intel ME to anyone else. :-( - Ted ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2024-01-03 16:37 ` Theodore Ts'o @ 2024-01-03 16:41 ` Dan Cross 2024-01-04 8:42 ` arnold 1 sibling, 0 replies; 44+ messages in thread From: Dan Cross @ 2024-01-03 16:41 UTC (permalink / raw) To: Theodore Ts'o; +Cc: The Unix Heritage Society On Wed, Jan 3, 2024 at 11:37 AM Theodore Ts'o <tytso@mit.edu> wrote: > On Wed, Jan 03, 2024 at 10:56:14AM -0500, Dan Cross wrote: > > Sadly the situation is even more complex than this. > > > > Consider AMD's EPYC processors: before the x86 cores start up, the PSP > > (Platform Security Processor) starts up and does a lot of > > pre-pre-initialization: it does DRAM timing training, for instance. > > It's also responsible for loading the x86 payload out of the local > > flash and setting up the x86 environment so that when those cores come > > out of reset, they're running whatever was loaded (for instance, they > > can load %cs on the BSC so that it starts somewhere other than the > > architecturally-defined segment right below 4GiB). While cool in some > > ways ("I don't have to train DRAM? Score!") the PSP is embedded in the > > SoC and the firmware is a signed blob you get from AMD. I know there's > > an ARM Cortex-A5 in there, but don't know much more about it and even > > if I did, I have no way to generate signed images for it. :-/ > > > > The point is, even if you've got a completely open stack running on > > x86 from the reset vector, there's almost certainly something else > > somewhere that's not open (yet). > > Or there's something running on a completely different x86 core with > unpatched securiy bugs in the Minix and Apache cores that you can't > even disable (unless you are the National Security Agency).... Sadly, > Intel refuses to make it available the magic bits to disable the Intel > ME to anyone else. :-( ...and let's not even get started on SMM. :-( - Dan C. ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2024-01-03 16:37 ` Theodore Ts'o 2024-01-03 16:41 ` Dan Cross @ 2024-01-04 8:42 ` arnold 2024-01-04 18:26 ` Kevin Bowling 1 sibling, 1 reply; 44+ messages in thread From: arnold @ 2024-01-04 8:42 UTC (permalink / raw) To: tytso, crossd; +Cc: tuhs Things have wandered a little far afield... :-) "Theodore Ts'o" <tytso@mit.edu> wrote: > Or there's something running on a completely different x86 core with > unpatched securiy bugs in the Minix and Apache cores that you can't > even disable (unless you are the National Security Agency).... Sadly, > Intel refuses to make it available the magic bits to disable the Intel > ME to anyone else. :-( I worked for a number of years in the design center where the firmware and software for the ME were develped. Although it's possible that the firmware developers were sworn to secrecy, I never heard anything about back doors for the NSA or anyone else. Intel took security and code quality in the ME very seriously, and during my tenure the quality of the ME firmware improved a lot. ISTR that the BIOS had settings for disabling the ME. Is that no longer true? I know there are lots of people who despise the ME, which I never understood. It was designed to solve the very real problem of remote PC management, and for that it works. My own feeling is, if you don't want the ME, buy a processor without it; there are plenty from Intel and AMD. Quite seriously, and with no animosity, I'd be happy to learn what I'm missing here. Thanks, Arnold ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2024-01-04 8:42 ` arnold @ 2024-01-04 18:26 ` Kevin Bowling 0 siblings, 0 replies; 44+ messages in thread From: Kevin Bowling @ 2024-01-04 18:26 UTC (permalink / raw) To: arnold; +Cc: tuhs On Thu, Jan 4, 2024 at 1:42 AM <arnold@skeeve.com> wrote: > > Things have wandered a little far afield... :-) > > "Theodore Ts'o" <tytso@mit.edu> wrote: > > > Or there's something running on a completely different x86 core with > > unpatched securiy bugs in the Minix and Apache cores that you can't > > even disable (unless you are the National Security Agency).... Sadly, > > Intel refuses to make it available the magic bits to disable the Intel > > ME to anyone else. :-( > > I worked for a number of years in the design center where the firmware > and software for the ME were develped. Although it's possible that > the firmware developers were sworn to secrecy, I never heard anything > about back doors for the NSA or anyone else. > > Intel took security and code quality in the ME very seriously, > and during my tenure the quality of the ME firmware improved a lot. > > ISTR that the BIOS had settings for disabling the ME. Is that > no longer true? > > I know there are lots of people who despise the ME, which I never > understood. It was designed to solve the very real problem of remote > PC management, and for that it works. My own feeling is, if you don't > want the ME, buy a processor without it; there are plenty from Intel > and AMD. I have tried out the AMT stuff for OS development and it is a mess. I am skeptical anyone seriously uses it. Laptops already have microcontrollers for various functionality so it is hard to see why the already standardized NC-SI and IPMI couldn't be applied to the problem space in some way that is secure, standardized, and doesn't significantly change the BOM cost. For whatever reason, intel makes it difficult to impossible to remove the ME in later generations. It seems more than accidental incompetence since people have figured out how to force it into a brain dead state (coreboot with me_cleaner). It is doubly suspicious that the US government has a killswitch for it that the commercial and general public do not. Which are the intel CPUs without the ME? Just because a CPU doesn't have vPro licensed doesn't mean the ME isn't there. > Quite seriously, and with no animosity, I'd be happy to learn what > I'm missing here. https://en.wikipedia.org/wiki/Intel_Management_Engine has a good enough survey and links to other soruces. It's a complete mess on the NIC too, the firmware on e1000 NICs has all sorts of issues and much of it is related to the insane errata and complexity of transitioning to and fro Management mode and different interpretations of who is responsible for power management. > > Thanks, > > Arnold ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2024-01-03 3:33 ` Theodore Ts'o 2024-01-03 3:57 ` Warner Losh @ 2024-01-03 14:39 ` Dan Cross 1 sibling, 0 replies; 44+ messages in thread From: Dan Cross @ 2024-01-03 14:39 UTC (permalink / raw) To: Theodore Ts'o; +Cc: The Unix Heritage Society On Tue, Jan 2, 2024 at 10:33 PM Theodore Ts'o <tytso@mit.edu> wrote: > On Tue, Jan 02, 2024 at 03:48:28PM -0500, Dan Cross wrote: > > On Sun, Dec 31, 2023 at 6:25 PM Larry McVoy <lm@mcvoy.com> wrote: > > > > Ironically, the UEFI people have done something _similar_ to OF in the > > form of AML (ACPI Machine Language), which is a byte-code > > serialization ASL (ACPI Source Language); presumably that's system > > independent. The idea of a p-code representation is about where the > > similarity ends, though: AML exposes a mechanism to talk to the UEFI > > OS for a whole slew of stuff, which is rather unlike what OF did > > [Ted's excellent history of ACPI/UEFI elided for brevity.] > > Normally, once the OS has gotten to a certain point in its OS > initialization, the OS can "terminate UEFI services". At that point, > the OS can no longer call into the UEFI functions --- but it also > doesn't have to worry about the UEFI code servicing an interrupt take > taking control of the CPU away from the OS. Warner corrected this: UEFI remains resident and callable. > However, the OS can still execute various ACPI functions, assuming the > OS has its own AML interpreter. (Of course, the UEFI BIOS has its own > AML interpreter --- and scheduler --- and everything else that a > simple OS might have.) Ron Minnich has a number of really good > presentations about the "joys" of working with UEFI. See the YouTube > video, "Replace your Expoit-ridden Firmware with Linux"[1] and prepare to > be horrified about what horrors lurks in the heart of UEFI.... > > [1] https://www.youtube.com/watch?v=iffTJ1vPCSo I'm well aware. :-) But I'll see your LinuxBoot and raise you this: https://www.osfc.io/2022/talks/i-have-come-to-bury-the-bios-not-to-open-it-the-need-for-holistic-systems/ Of course, we at Oxide have the luxury of controlling the hardware in addition to the software that runs [directly] on it, so we can get away with doing our own thing. It's not so easy in the larger world, where integrating components from disparate vendors demands _some_ kind of interface. It's just unfortunate that UEFI is the interface that we got; see also Mothy Roscoe's OSDI'21 keynote about rediscovering hardware. > So you can very much be *not* using UEFI, and still be using ACPI --- > either if you are using a pre-2004 computer, or if you are using a > more modern platform which uses LinuxBoot. Indeed; caveat the details that have already been mentioned about coreboot/oreboot/holistic boot vs LinuxBoot (which replaces the UEFI DXE layer). However, note that the ACPI spec has fallen under the auspices of the UEFI forum for the last ten years or so, which is what I meant (e.g., that one finds the ACPI spec on uefi.org). It's true, though, that I was being unacceptably imprecise by not mentioning the history at all and that ACPI came before UEFI. - Dan C. ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2023-12-31 22:46 ` G. Branden Robinson 2023-12-31 23:06 ` Larry McVoy @ 2023-12-31 23:08 ` Phil Budne 2023-12-31 23:37 ` G. Branden Robinson 2023-12-31 23:50 ` G. Branden Robinson 2 siblings, 1 reply; 44+ messages in thread From: Phil Budne @ 2023-12-31 23:08 UTC (permalink / raw) To: tuhs, g.branden.robinson G. Branden Robinson wrote: > OpenFirmware (OF) ran on SPARC and PowerPC hardware, at least. OF had its origin in Sun OpenBoot PROM (OBP), which, ISTR, first appeared in the SPARCstation 1 (sun4c kernel arch) along with SBus. Earlier (VME) based SPARC (sun4 kernel arch) systems had only a Sun-3 like boot PROM (single letter commands, no Forth). ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2023-12-31 23:08 ` Phil Budne @ 2023-12-31 23:37 ` G. Branden Robinson 2023-12-31 23:59 ` Warner Losh 0 siblings, 1 reply; 44+ messages in thread From: G. Branden Robinson @ 2023-12-31 23:37 UTC (permalink / raw) To: tuhs [-- Attachment #1: Type: text/plain, Size: 865 bytes --] At 2023-12-31T15:06:15-0800, Larry McVoy wrote: > On Sun, Dec 31, 2023 at 04:46:49PM -0600, G. Branden Robinson wrote: > > At 2023-12-31T16:31:00-0500, Clem Cole wrote: > > > But the key is that it ran on 68K's. > > > > I don't think that's the case. > > OpenFirmware is Mitch Bradley's baby. I believe it ran on 68k Suns, > there was some sort of boot prom there. At 2023-12-31T18:08:36-0500, Phil Budne wrote: > OF had its origin in Sun OpenBoot PROM (OBP), which, ISTR, first > appeared in the SPARCstation 1 (sun4c kernel arch) along with SBus. > Earlier (VME) based SPARC (sun4 kernel arch) systems had only a Sun-3 > like boot PROM (single letter commands, no Forth). Thanks for the clarifications, gentlemen. My first Sun experience was a SPARC IPC--a bunch of them, actually, in a "freshman engineering lab"... Regards, Branden [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2023-12-31 23:37 ` G. Branden Robinson @ 2023-12-31 23:59 ` Warner Losh 0 siblings, 0 replies; 44+ messages in thread From: Warner Losh @ 2023-12-31 23:59 UTC (permalink / raw) To: G. Branden Robinson; +Cc: The Eunuchs Hysterical Society [-- Attachment #1: Type: text/plain, Size: 1106 bytes --] On Sun, Dec 31, 2023, 4:38 PM G. Branden Robinson < g.branden.robinson@gmail.com> wrote: > At 2023-12-31T15:06:15-0800, Larry McVoy wrote: > > On Sun, Dec 31, 2023 at 04:46:49PM -0600, G. Branden Robinson wrote: > > > At 2023-12-31T16:31:00-0500, Clem Cole wrote: > > > > But the key is that it ran on 68K's. > > > > > > I don't think that's the case. > > > > OpenFirmware is Mitch Bradley's baby. I believe it ran on 68k Suns, > > there was some sort of boot prom there. > > At 2023-12-31T18:08:36-0500, Phil Budne wrote: > > OF had its origin in Sun OpenBoot PROM (OBP), which, ISTR, first > > appeared in the SPARCstation 1 (sun4c kernel arch) along with SBus. > > Earlier (VME) based SPARC (sun4 kernel arch) systems had only a Sun-3 > > like boot PROM (single letter commands, no Forth). > > Thanks for the clarifications, gentlemen. My first Sun experience was a > SPARC IPC--a bunch of them, actually, in a "freshman engineering lab"... > The Sun 3 systems had a similar interface (L1/a etc) but no 4th and a different set of ROMs. Warner Regards, > Branden > [-- Attachment #2: Type: text/html, Size: 1788 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2023-12-31 22:46 ` G. Branden Robinson 2023-12-31 23:06 ` Larry McVoy 2023-12-31 23:08 ` Phil Budne @ 2023-12-31 23:50 ` G. Branden Robinson 2024-01-01 0:09 ` Al Kossow 2 siblings, 1 reply; 44+ messages in thread From: G. Branden Robinson @ 2023-12-31 23:50 UTC (permalink / raw) To: The Unix Heritage Society [-- Attachment #1: Type: text/plain, Size: 782 bytes --] [self-follow-up] At 2023-12-31T16:46:51-0600, G. Branden Robinson wrote: > The m68k "Old World" PowerMacs used a different firmware entirely; I Sorry, I was missing the word "and" after "m68k" here. I certainly would not deliberately claim that Old World PowerMacs were m68k boxes. > assume boot ROM code descended from the original Macintosh (or even > Lisa, maybe). The PowerPC "New World" PowerMacs, which immediately > departed from the beige color scheme, did come in with Apple's > acquisition of NeXT. ...and used OpenFirmware for their boot ROMs universally. It's my understanding that the OLPC ("One Laptop Per Child") laptop used OF as well, and would have been one of the more unusual devices to do so, but I never got to play with one of those. Regards, Branden [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history 2023-12-31 23:50 ` G. Branden Robinson @ 2024-01-01 0:09 ` Al Kossow 0 siblings, 0 replies; 44+ messages in thread From: Al Kossow @ 2024-01-01 0:09 UTC (permalink / raw) To: tuhs >> assume boot ROM code descended from the original Macintosh (or even >> Lisa, maybe). The PowerPC "New World" PowerMacs, which immediately >> departed from the beige color scheme, did come in with Apple's >> acquisition of NeXT. Sorry, I don't see the original of this misstatement The first generation Nubus PowerMacs use essentially an unmodified 68K ROM for booting, emulating a 68LC040 to do so. Also, Mac and Lisa firmware are completely different from each other. The second generation machines with PCI had Open Firmware. "New World" refers to ROM in RAM systems which have pretty much just some hardware init and OF in them and started around the time of the iMac ^ permalink raw reply [flat|nested] 44+ messages in thread
* [TUHS] Re: Question about BSD disklabel history @ 2023-12-31 21:55 Norman Wilson 0 siblings, 0 replies; 44+ messages in thread From: Norman Wilson @ 2023-12-31 21:55 UTC (permalink / raw) To: tuhs Disk sections (I don't think anyone in Research called them partitions--certainly the Research manuals didn't) were originally defined in the device driver, not by data on the disk. In those days, system management included recompiling stuff, including the OS kernel, and it was not unusual for sites to edit hp.c or whatnot to adjust things to local preference. There was nothing magic about the mapping between device names and minor device numbers either; the system came with certain conventions on the original tape, but it was not at all uncommon to change them. By the time I arrived at the first Unix site I ever helped run, in a physics group at Caltech, we already used a different naming convention: a BSD-like ddNs, where dd was a driver name, N the physical drive unit number, s a section letter. I don't know whether that was borrowed from BSD (it must have started during the 3BSD era, since I started there in mid-1980 and 4BSD appears to have been released late in that year). Looking at my archival copy of that much-locally-hacked source tree, I see that we later moved the definitions of all the disk-section tables to a single file compiled at system-configuration time (we used a USG-like scheme that compiled most of the system into libraries, rather than compiling every file separately for each target system a la V7 and BSD). That simplified handling our somewhat- complicated disk topology: all but system disks were connected through System Industries 9400 disk controllers, which were a neat design (each controller could interface to as many as four hosts and four disks) but in practice were not always reliable. On one hand, we arranged for one disk to be used in parts by our main time-sharing VAX and a subsidiary PDP-11/45, making the 11/45 cheaper to keep around; on the other, the main VAX had two paths to each of its disks, through different SI controllers, so when an SI controller conked out we could run without it until the service guys fixed it. (Each disk was dual-ported, as was common in the SMD world, hence connected to two controllers.) Reliability took rather more work in those days. A different data point: by the time I moved from California to New Jersey and joined 1127, Research was also using a different naming scheme for disk sections. By then the internal naming convention was e.g. ra17 for physical unit 1, section 7; by further convention 7 (the highest-numbered section). At some point a little later we added an ioctl to set the starting block and size of a particular section on a particular drive, but we never went to having the OS itself try to find a label and trust its contents (something that still makes the 1980s part of me feel a little creepy, though 21st century me has come to terms with it). Norman Wilson Certified old fart Toronto ON ^ permalink raw reply [flat|nested] 44+ messages in thread
end of thread, other threads:[~2024-01-04 18:27 UTC | newest] Thread overview: 44+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2023-12-31 17:30 [TUHS] Question about BSD disklabel history Grant Taylor via TUHS 2023-12-31 17:38 ` [TUHS] " arnold 2023-12-31 20:07 ` Warner Losh 2024-01-01 0:13 ` Bakul Shah 2023-12-31 20:27 ` Phil Budne 2023-12-31 21:02 ` Warner Losh 2024-01-01 0:26 ` Grant Taylor via TUHS 2024-01-01 2:22 ` Warner Losh 2024-01-01 3:24 ` Grant Taylor via TUHS 2023-12-31 21:31 ` Clem Cole 2023-12-31 22:07 ` Warner Losh 2024-01-01 16:00 ` Clem Cole 2024-01-02 18:49 ` Warner Losh 2024-01-02 19:30 ` Chet Ramey 2024-01-02 20:07 ` Clem Cole 2024-01-02 19:50 ` Dan Cross 2024-01-02 19:55 ` Jim Capp 2024-01-02 20:11 ` Dan Cross 2024-01-02 20:30 ` Dan Cross 2024-01-02 20:50 ` Clem Cole 2024-01-02 21:04 ` Dan Cross 2023-12-31 22:46 ` G. Branden Robinson 2023-12-31 23:06 ` Larry McVoy 2023-12-31 23:37 ` Al Kossow 2023-12-31 23:41 ` Alec Muffett 2024-01-02 20:48 ` Dan Cross 2024-01-02 21:17 ` John Cowan 2024-01-03 3:33 ` Theodore Ts'o 2024-01-03 3:57 ` Warner Losh 2024-01-03 4:03 ` Warner Losh 2024-01-03 4:30 ` Theodore Ts'o 2024-01-03 5:10 ` Warner Losh 2024-01-03 15:56 ` Dan Cross 2024-01-03 16:37 ` Theodore Ts'o 2024-01-03 16:41 ` Dan Cross 2024-01-04 8:42 ` arnold 2024-01-04 18:26 ` Kevin Bowling 2024-01-03 14:39 ` Dan Cross 2023-12-31 23:08 ` Phil Budne 2023-12-31 23:37 ` G. Branden Robinson 2023-12-31 23:59 ` Warner Losh 2023-12-31 23:50 ` G. Branden Robinson 2024-01-01 0:09 ` Al Kossow 2023-12-31 21:55 Norman Wilson
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).