[TUHS] Question about BSD disklabel history

[TUHS] Question about BSD disklabel history

[Grant Taylor via TUHS]

[-- Attachment #1: Type: text/plain, Size: 649 bytes --]

Hi,

I've found myself wondering about partitions inside of BSD disk labels.

Specifically, when and where was the convention that "a" is root, "b" is 
swap, etc?

I also understand the "c" partition to be the entire disk, unless it 
isn't, at which point it's the entire slice (BIOS / MBR partition) 
containing the BSD disklabel and "d" is the entire disk.

I also found something last night that indicated that OpenBSD uses disk 
labels somewhat differently than FreeBSD.

Aside:  This is one of the dangers of wondering how something curious 
came to be and why it came to be when working on 10-15 year old FreeBSD 
systems.



-- 
Grant. . . .

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4033 bytes --]

[TUHS] Re: Question about BSD disklabel history

[arnold]

The different overlapping partitions predates disk labels.
Up to and including 4.3 BSD, to change the size of partitions
on a particular disk, you had to recompile the kernel.

They were that way so that if you had multiple disks, you
could use one for root + swap + some thing small and use another
whole disk for a single filesystem.

It was also helpful, if you had the drives, to nightly dd
your real root to the "a" partition on another, identical
drive, so that you could boot the backup root in an emergency.

I don't remember for sure, but I think that Ultrix may have
been the first BSD-style system to have disk labels, followed
by some version of SunOS. All of that is way in the distant
past though: mid- to late 80's.

I am guessing that the original conventions date back to
V7 or 32V, but one would have to go looking at code to be sure.

HTH,

Arnold

Grant Taylor via TUHS <tuhs@tuhs.org> wrote:

> Hi,
>
> I've found myself wondering about partitions inside of BSD disk labels.
>
> Specifically, when and where was the convention that "a" is root, "b" is 
> swap, etc?
>
> I also understand the "c" partition to be the entire disk, unless it 
> isn't, at which point it's the entire slice (BIOS / MBR partition) 
> containing the BSD disklabel and "d" is the entire disk.
>
> I also found something last night that indicated that OpenBSD uses disk 
> labels somewhat differently than FreeBSD.
>
> Aside:  This is one of the dangers of wondering how something curious 
> came to be and why it came to be when working on 10-15 year old FreeBSD 
> systems.
>
>
>
> -- 
> Grant. . . .

[TUHS] Re: Question about BSD disklabel history

[Warner Losh]

[-- Attachment #1: Type: text/plain, Size: 3924 bytes --]

On Sun, Dec 31, 2023 at 10:39 AM <arnold@skeeve.com> wrote:

> The different overlapping partitions predates disk labels.
> Up to and including 4.3 BSD, to change the size of partitions
> on a particular disk, you had to recompile the kernel.
>

Yes. Disk partitions were implemented in the disk driver dating back
at least to the 5th edition. Lack of sources makes it harder to know
for sure. The man pages for each of the disk drivers included the
partition layout (IIRC, I didn't cross verify).


> They were that way so that if you had multiple disks, you
> could use one for root + swap + some thing small and use another
> whole disk for a single filesystem.
>

Yes. Unlike today, the partitions covered the disk in different, overlapping
ways. And allowed for some parts of the disk to be uncovered by a
partition. You could then patch the offset and length into the kernel with
adb and use that area of the disk for swap space.


> It was also helpful, if you had the drives, to nightly dd
> your real root to the "a" partition on another, identical
> drive, so that you could boot the backup root in an emergency.
>
> I don't remember for sure, but I think that Ultrix may have
> been the first BSD-style system to have disk labels, followed
> by some version of SunOS. All of that is way in the distant
> past though: mid- to late 80's.
>

When I looked into it years ago, I convinced myself that SunOS
was the first to have it (since the very first version of SunOS 1.0
had disk labels) and that all the other vendors followed suit within
a couple of years. Ultrix-11 had the fixed labels through its EOL.
I didn't see any disklable stuff in the Ultrix-32M that we have, but
it was admittedly a quick look.


> I am guessing that the original conventions date back to
> V7 or 32V, but one would have to go looking at code to be sure.
>

Yes. The conventions are as old as Unix. And did change at least
between V5 and V6 and I think maybe again for some drivers between
V6 and V7 (though I'm less sure of this).

BSD4.3 Tahoe, the first release after BSD4.3, fixed the disklabel format.
Sun's vtoc format (aka sunlabel) was different from that, but quite
similar. This is why it's called 4.3BSD format in many places, even though
it wasn't in the original 4.3BSD release.

OpenBSD and FreeBSD's disklabels are the same format. However, OpenBSD's
have twice as many entries than FreeBSD's. And there's a number of other
fussy
details about what different labels mean based on the shared history and
divergent
evolution of the two systems. OpenBSD also supports both endians, while
FreeBSD
only supports native endian. So these changes and different decisions lead
to
the situation today where it's hard to read them on each-other's systems.
Plus with
the new, larger disks FreeBSD has effectively abandoned them in lue of GPT
partitioning.
There is a disklabel64 format too, but I'm unsure how widely that's
deployed.

All the commercial Unixes also had their own format. AIX, HPUX, DG, etc.
That's
a topic for another time...

Warner


> Grant Taylor via TUHS <tuhs@tuhs.org> wrote:
>
> > Hi,
> >
> > I've found myself wondering about partitions inside of BSD disk labels.
> >
> > Specifically, when and where was the convention that "a" is root, "b" is
> > swap, etc?
> >
> > I also understand the "c" partition to be the entire disk, unless it
> > isn't, at which point it's the entire slice (BIOS / MBR partition)
> > containing the BSD disklabel and "d" is the entire disk.
> >
> > I also found something last night that indicated that OpenBSD uses disk
> > labels somewhat differently than FreeBSD.
> >
> > Aside:  This is one of the dangers of wondering how something curious
> > came to be and why it came to be when working on 10-15 year old FreeBSD
> > systems.
> >
> >
> >
> > --
> > Grant. . . .
>

[-- Attachment #2: Type: text/html, Size: 5514 bytes --]

[TUHS] Re: Question about BSD disklabel history

[Phil Budne]

arnold@skeeve.com wrote:
> I am guessing that the original conventions date back to
> V7 or 32V, but one would have to go looking at code to be sure.

Looking at the hp (RP-04/5/6) man page in V7, 32/v, and 3bsd the
convention was rpN, where 0 <= N <= 7 were partitions on the first
drive, and rp8 was the first partition on the second drive.  rp0 was
root, rp1 was swap, rp2 and rp3 (in the v7 man page) are unassigned.

The rpNx convention, where N is the drive number, with "c" as the
whole disk is present in the 4.1BSD hp(4) man page.

[TUHS] Re: Question about BSD disklabel history

[Warner Losh]

[-- Attachment #1: Type: text/plain, Size: 961 bytes --]

On Sun, Dec 31, 2023 at 1:27 PM Phil Budne <phil@ultimate.com> wrote:

> arnold@skeeve.com wrote:
> > I am guessing that the original conventions date back to
> > V7 or 32V, but one would have to go looking at code to be sure.
>
> Looking at the hp (RP-04/5/6) man page in V7, 32/v, and 3bsd the
> convention was rpN, where 0 <= N <= 7 were partitions on the first
> drive, and rp8 was the first partition on the second drive.  rp0 was
> root, rp1 was swap, rp2 and rp3 (in the v7 man page) are unassigned.
>
> The rpNx convention, where N is the drive number, with "c" as the
> whole disk is present in the 4.1BSD hp(4) man page.
>

This was just a change to MAKEDEV. The kernel used the same minor
number encoding. 4BSD still used the V7 make script in /dev, but at some
point before 4.1cBSD (I don't have the 4.1BSD exploded locally) it changed,
both in name from MAKE to MAKEDEV and to start the [a-h] suffix for
partitions.

Warner

[-- Attachment #2: Type: text/html, Size: 1444 bytes --]

[TUHS] Re: Question about BSD disklabel history

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 8156 bytes --]

Below... apologies if I forgot something -- this is pretty much as I knew
it/play it out.

On Sun, Dec 31, 2023 at 12:31 PM Grant Taylor via TUHS <tuhs@tuhs.org>
wrote:

> Hi,
>
> I've found myself wondering about partitions inside of BSD disk labels.
>
> Specifically, when and where was the convention that "a" is root, "b" is
> swap, etc?
>
> I also understand the "c" partition to be the entire disk, unless it
> isn't, at which point it's the entire slice (BIOS / MBR partition)
> containing the BSD disklabel and "d" is the entire disk.
>
> I also found something last night that indicated that OpenBSD uses disk
> labels somewhat differently than FreeBSD.
>
> Aside:  This is one of the dangers of wondering how something curious
> came to be and why it came to be when working on 10-15 year old FreeBSD
> systems.
>
> A long, strange trip as were.

Step one was the introduction of DEC RP04 disks -- 19 Heads, 21
Sectors/Track, 411 cylinders or 163989 blocks which will overflow a 16-bit
PDP-11 register. You find it in the V5 code, but I have Dennis's rp.c
driver showing split the RP04 into from v6:

struct {

char *nblocks;

int cyloff;

} rp_sizes[] {

40600, 0, /* cyl 0 thru 202 */
40600, 203, /* cyl 203 thru 405 */
9200, 0, /* cyl 0 thru 45 */
9200, 360, /* cyl 360 thru 405 */
-1, 0, /* cyl 0 thru 327 */
-1, 78, /* cyl 78 thru 405 */
15600, 0, /* cyl 0 thru 77 */
15600, 328, /* cyl 328 thru 405 */

};

(V5/V6 C does not have an unsigned or longs; char * is how the compiler
created a type that was treated as an unsigned 16-bit integer).

If you look at things like the rk05 in V6, we reserve a few blocks at the
end of the disk for swap.  Remember quotas and file single file management
are not there yet, so the scheme with the RP04 starts to be the way of the
world.  Also, remember Ted has not introduced what would later be called
fsck(1), so recovery from a crash was painful.   The corruption was often
on the root disk, a quick dd(1) of the root FS you were quickly back - or
at least bookable and then you could use  check, ncheck, *et al *on the
other partitions.

Also, with PWB - a new set of tools was created for disk management that
Research did not pick up: volcopy, cpio, et al.  The V6 version of PWB is
where *disklabels are first added to the superblock* BTW.     In fact, when
we did the original fsck(8) at CMU, we had v5 then v6.  No, PWB pollution
into our IP stream.  So the original fsck I recently recovered for V6 does
not have the disklabel code in it.   When Ted goes to USG later, he adds
the label support (which is why if you compile the version in the V7
addenda, it will get errors -- that one is the version on UNIX/TS - which
the kernel supported disklabels). I had to look at the code, I think labels
are simple, just names not information about that specific FS.  The labels
are not per disk pack.

Meanwhile back in Murray Hill ....

With V7, we have long int, but the partition scheme is already in use and
has some nice side  effects, so it is now a feature, and with the RP06 --
so Dennis's new  scheme:

struct size
{

daddr_t nblocks;
int cyloff;


} hp_sizes[8] =

{
9614, 0, /* cyl 0 thru 22 */
8778, 23, /* cyl 23 thru 43 */
0, 0,
0, 0,
161348, 44, /* cyl 44 thru 429 */
160930, 430, /* cyl 430 thru 814 */
153406, 44, /* cyl 44 thru 410 (rp04, rp05) */
322278, 44, /* cyl 44 thru 814 (rp06) */

};

At this point, as Arnold said, people would change the size/shape of the
tables locally by recompiling and setting up the disk they way you wanted.
It could be awkward, you typically needed a second disk pack, *etc*.

With DEC's release of V7m, 2.x (2.1, I think), Fred Cantor of DEC put the
first versions of labels into what would later become Ultrix-11.   That
code was released to CRSG (it is also where 2.9 et al. get Fred's wonderful
swapping support).   Shannon (still at DEC) helped Sam add some of that
into a 4.1a/b/c stream...  This was where Sam added things like newfs(8) and
/etc/disktab and started to try to make BSD more resilient to different
types of disks.   But my memory is that it was nascent in the 4.2 release,
so you had /etc/disktab and some of the tools but not everything until
later, 4.3, when what you know today came into being.

Meanwhile, in Pittsburgh ...

CMU  is working on the Mach2.5 project.   Intel had donated some of the
original 386 PC support to Rasid's group (they were partial funding) that
they had developed with ISC/AT&T *et al* to handle things particular to the
PC architecture -- such as the PC's fdisk(1) since PC's have a completely
different way of partitioning things. Bob Baron did the Mach version - as
to what was new and what was from Intel I'm not 100% sure.  But CMU did
have the tools that Intel felt they owned and were not part of System
III/V.

The key is that a PC's fdisk(1) - creates a UNIX partition within the DOS
partition table so DOS knows to leave it alone, but then Unix
sub-partitions using its own scheme.   My memory here is hazy, I think the
sub-partition code Bob used is different from the ATT's 386 [which most
people saw as ISC's 386/ix who did the original work under contract for
AT&T]

Meanwhile, across the pond ...
Andy's team starts to want a V7 for the PC/XT and starts the Minux
project.   I know they had CMU's Mach 2.5 stuff and there fdisk, while it
seems to be independent (i.e. clean room) is very similar to Bob Baron's
scheme.


Meanwhile, in Walnut Creek ...

Jolitiz *et al.* started 386BSD, which would begat FreeBSD/NetBSD/OpenBSD.
  At some point, Baron's fdisk (1) and support tools make their way to that
project.    This was why all the original UNIX for a PC used the same PC
fdisk "type" ID number (the ISC 386/ix type -- from the original 386 port).
[Linus built his own tools, which I think started from the Minux tools, and
thus he did his own thing --- so Linux disks were not marked with the same
types). If you look at the sources to both Mach and early 386BSD, you'll
see Intel markings as well as Bob's name in a couple of places [Truth in
advertising here (different story) -- I wrote most of the original AT disk
driver Bill used -- I was consulting for NCR and had access to the WD specs
as well as the CMU code but I was not the conduit of the CMU code base to
Bill].

At some point, 386BSD started to go one way, and what would become FreeBSD
started to ramp up.  Jordan Hubbard wants an "easy install" like Microsoft
has for Win98, so very early FreeBSD has the first PC-based install that
can run without much intervention.    At some point, Jordon or someone else
in the  FreeBSD team introduced what they called slices, furthering the BSD
scheme but designed to overlay the PC better.   The NetBSD folks had
already split off, so they did not pick it up.  When OpenBSD split from
NetBSD, they did something similar but different to FreeBSD's slices...


Meanwhile, in Cupertino ....

Next (ney Apple) started with the Mach code base from CMU.  There was a
push in the Valley in those days for something called OpenFirmware [Warner
help here -- I think that it was forth based IIRC, and Sun may have had
their hand in it also].  But the key is that it ran on 68K's.   So Next's
original partition scheme was influenced if not based on some of Bob's
work, although they did not need the silly BIOS stuff, so they did make it
all talk to OpenFW (BTW: I think that was still all Next)

By the time the Intel Mac's BIOS had begun to be replaced in the WINTEL
world by UEFI, Apple committed to using a flavor of it. My old lab partner,
Guy Sotomayer (*a.k.a.* gss on many mailing lists) was mixed up during his
time at Apple. He might be able to fill in some details if I ask him.  I
also know that Apple was not happy with how disks were handled with the
original partition scheme, and by later release started over with their
partitioning scheme(s) at least once.


I've undoubtedly left stuff out, such as much of Linux - but this is all
part of the UNIX war I was a player.

[-- Attachment #2: Type: text/html, Size: 18366 bytes --]

[TUHS] Re: Question about BSD disklabel history

[Warner Losh]

[-- Attachment #1: Type: text/plain, Size: 13159 bytes --]

On Sun, Dec 31, 2023 at 2:31 PM Clem Cole <clemc@ccc.com> wrote:

> Below... apologies if I forgot something -- this is pretty much as I knew
> it/play it out.
>
> On Sun, Dec 31, 2023 at 12:31 PM Grant Taylor via TUHS <tuhs@tuhs.org>
> wrote:
>
>> Hi,
>>
>> I've found myself wondering about partitions inside of BSD disk labels.
>>
>> Specifically, when and where was the convention that "a" is root, "b" is
>> swap, etc?
>>
>> I also understand the "c" partition to be the entire disk, unless it
>> isn't, at which point it's the entire slice (BIOS / MBR partition)
>> containing the BSD disklabel and "d" is the entire disk.
>>
>> I also found something last night that indicated that OpenBSD uses disk
>> labels somewhat differently than FreeBSD.
>>
>> Aside:  This is one of the dangers of wondering how something curious
>> came to be and why it came to be when working on 10-15 year old FreeBSD
>> systems.
>>
>> A long, strange trip as were.
>
> Step one was the introduction of DEC RP04 disks -- 19 Heads, 21
> Sectors/Track, 411 cylinders or 163989 blocks which will overflow a 16-bit
> PDP-11 register. You find it in the V5 code, but I have Dennis's rp.c
> driver showing split the RP04 into from v6:
>
> struct {
>
> char *nblocks;
>
> int cyloff;
>
> } rp_sizes[] {
>
> 40600, 0, /* cyl 0 thru 202 */
> 40600, 203, /* cyl 203 thru 405 */
> 9200, 0, /* cyl 0 thru 45 */
> 9200, 360, /* cyl 360 thru 405 */
> -1, 0, /* cyl 0 thru 327 */
> -1, 78, /* cyl 78 thru 405 */
> 15600, 0, /* cyl 0 thru 77 */
> 15600, 328, /* cyl 328 thru 405 */
>
> };
>
> (V5/V6 C does not have an unsigned or longs; char * is how the compiler
> created a type that was treated as an unsigned 16-bit integer).
>
> If you look at things like the rk05 in V6, we reserve a few blocks at the
> end of the disk for swap.  Remember quotas and file single file management
> are not there yet, so the scheme with the RP04 starts to be the way of the
> world.  Also, remember Ted has not introduced what would later be called
> fsck(1), so recovery from a crash was painful.   The corruption was often
> on the root disk, a quick dd(1) of the root FS you were quickly back - or
> at least bookable and then you could use  check, ncheck, *et al *on the
> other partitions.
>
> Also, with PWB - a new set of tools was created for disk management that
> Research did not pick up: volcopy, cpio, et al.  The V6 version of PWB is
> where *disklabels are first added to the superblock* BTW.     In fact,
> when we did the original fsck(8) at CMU, we had v5 then v6.  No, PWB
> pollution into our IP stream.  So the original fsck I recently recovered
> for V6 does not have the disklabel code in it.   When Ted goes to USG
> later, he adds the label support (which is why if you compile the version
> in the V7 addenda, it will get errors -- that one is the version on UNIX/TS
> - which the kernel supported disklabels). I had to look at the code, I
> think labels are simple, just names not information about that specific
> FS.  The labels are not per disk pack.
>
> Meanwhile back in Murray Hill ....
>
> With V7, we have long int, but the partition scheme is already in use and
> has some nice side  effects, so it is now a feature, and with the RP06 --
> so Dennis's new  scheme:
>
> struct size
> {
>
> daddr_t nblocks;
> int cyloff;
>
>
> } hp_sizes[8] =
>
> {
> 9614, 0, /* cyl 0 thru 22 */
> 8778, 23, /* cyl 23 thru 43 */
> 0, 0,
> 0, 0,
> 161348, 44, /* cyl 44 thru 429 */
> 160930, 430, /* cyl 430 thru 814 */
> 153406, 44, /* cyl 44 thru 410 (rp04, rp05) */
> 322278, 44, /* cyl 44 thru 814 (rp06) */
>
> };
>
> At this point, as Arnold said, people would change the size/shape of the
> tables locally by recompiling and setting up the disk they way you wanted.
> It could be awkward, you typically needed a second disk pack, *etc*.
>
> With DEC's release of V7m, 2.x (2.1, I think), Fred Cantor of DEC put the
> first versions of labels into what would later become Ultrix-11.   That
> code was released to CRSG (it is also where 2.9 et al. get Fred's wonderful
> swapping support).   Shannon (still at DEC) helped Sam add some of that
> into a 4.1a/b/c stream...  This was where Sam added things like newfs(8)
> and /etc/disktab and started to try to make BSD more resilient to
> different types of disks.   But my memory is that it was nascent in the 4.2
> release, so you had /etc/disktab and some of the tools but not everything
> until later, 4.3, when what you know today came into being.
>

Except I can't find any of this in our V7m or ultrix-11 that we have. This
is 2.1. And there all the device drivers still have the fixed tables:
grep sizes * | grep '\[[0-9]'
hk.c:} hk_sizes[8] =
hk.c_i:} hk_sizes[8] =
hk.c_id:} hk_sizes[8] =
hm.c:} hm_sizes[24] =
hp.c:} hp_sizes[24] =

and the ultrix-11 3.1 sources we have are similar, but move all of the
above into /usr/sys/conf/dksizes.c.

The early disktab stuff encoded the default tables into a text file to make
it easier to cope with all the different types of disks, but didn't affect
what we know as bsdlabels until after 4.3BSD. The naming convention was
there, but the 'write bits to the disk to describe partitioning' I couldn't
find at all in ultrix. I think you may be confusing that stuff (which was a
very clever way to cope with all these disks that have different
partitioning in a increasingly generic way) with disk labeling which I
think. Sun was first to market with.

Maybe you can help me find it?

I did find the SunOS 0.x images (UniSoft V7 port) didn't appear have a disk
partition, but 1.0 (post 4BSD pre 4.1BSD) and later did. That puts it at
1982. Although based off BSD, the disk label stuff wasn't in BSD for
another 5 years when the BSD folks started the Tahoe port.

It's entirely possible one of the early UniSoft ports had this feature, and
that was adapted into SunOS and I've not seen the paper trail (studying the
early UniSoft ports is rather problematic for a variety of logistical
reasons)... Not least of which is the fragmentary nature of the surviving
source, the difficulty in obtaining boot images, etc. This is a very
under-documented time. It wouldn't surprise me if someone produced evidence
of Sun's partitioning scheme coming from this mess...  Or a very similar
thing. And Pixar may have been involved (though that's a memory I have that
I don't have in my notes documented). Maybe that's the Sam Leffler
connection (though I thought his time at Pixar post-dated this period
somewhat).

Meanwhile, in Pittsburgh ...
>
> CMU  is working on the Mach2.5 project.   Intel had donated some of the
> original 386 PC support to Rasid's group (they were partial funding) that
> they had developed with ISC/AT&T *et al* to handle things particular to
> the PC architecture -- such as the PC's fdisk(1) since PC's have a
> completely different way of partitioning things. Bob Baron did the Mach
> version - as to what was new and what was from Intel I'm not 100% sure.
> But CMU did have the tools that Intel felt they owned and were not part of
> System III/V.
>
> The key is that a PC's fdisk(1) - creates a UNIX partition within the DOS
> partition table so DOS knows to leave it alone, but then Unix
> sub-partitions using its own scheme.   My memory here is hazy, I think the
> sub-partition code Bob used is different from the ATT's 386 [which most
> people saw as ISC's 386/ix who did the original work under contract for
> AT&T]
>
> Meanwhile, across the pond ...
> Andy's team starts to want a V7 for the PC/XT and starts the Minux
> project.   I know they had CMU's Mach 2.5 stuff and there fdisk, while it
> seems to be independent (i.e. clean room) is very similar to Bob Baron's
> scheme.
>

Yea, the fdisk stuff came from early versions of PC-DOS to partition hard
drives in 2.0 / 1983. I do know the DEC Rainbow had yet another set of
partitioning (which the PRO and DecMATE didn't share). It never adopted
what today is the MBR partitioning scheme because it's hard disk support
pre-dates this. It quickly became standard, and we see evidence of support
for it in the Venix/86 sources that we have (both the XT version and the AT
version, though I think that was disk controller version, and not MBR data).


> Meanwhile, in Walnut Creek ...
>
> Jolitiz *et al.* started 386BSD, which would begat
> FreeBSD/NetBSD/OpenBSD.    At some point, Baron's fdisk (1) and support
> tools make their way to that project.    This was why all the original
> UNIX for a PC used the same PC fdisk "type" ID number (the ISC 386/ix type
> -- from the original 386 port). [Linus built his own tools, which I think
> started from the Minux tools, and thus he did his own thing --- so Linux
> disks were not marked with the same types). If you look at the sources to
> both Mach and early 386BSD, you'll see Intel markings as well as Bob's name
> in a couple of places [Truth in advertising here (different story) -- I
> wrote most of the original AT disk driver Bill used -- I was consulting for
> NCR and had access to the WD specs as well as the CMU code but I was not
> the conduit of the CMU code base to  Bill].
>
At some point, 386BSD started to go one way, and what would become FreeBSD
> started to ramp up.  Jordan Hubbard wants an "easy install" like Microsoft
> has for Win98, so very early FreeBSD has the first PC-based install that
> can run without much intervention.    At some point, Jordon or someone else
> in the  FreeBSD team introduced what they called slices, furthering the BSD
> scheme but designed to overlay the PC better.   The NetBSD folks had
> already split off, so they did not pick it up.  When OpenBSD split from
> NetBSD, they did something similar but different to FreeBSD's slices...
>

Yea, this was 1992 or 1993. In FreeBSD 1.x you just had BSD Labels which
would default in interesting ways to what the MBR of the disk was (I've not
double checked the code, this is my memory). The 'c' partition was the
entire BSD partition while the 'd' partition was the entire disk. And that
duality lived on for a while. The slice code came in during the 3.x series
(or after 2.0, which used the same scheme as 1.x). And it was mostly an
attempt to make the BSD labels independent of each other. There was some
weird code for bsd-label only disks but most of the visible changes from
slices came in device naming (if you had only one disk label, and it was on
the first partition it was compatible with old bsd labels by and large).
FreeBSD wouldn't have true nested MBR and bsd labels until 5.0 and geom
entered the tree... A lot of the 'differences' between the BSDs were in
presentation of the data to the user, rather than physical differences on
the disk (though such differences did exist, especially in endian). It was
a real mess of NIH + incompatible UI + almost-but-not-quite compatible
on-disk formats coupled with a bunch of bugs that differed among the BSDs.
I do not miss these days.


> Meanwhile, in Cupertino ....
>
> Next (ney Apple) started with the Mach code base from CMU.  There was a
> push in the Valley in those days for something called OpenFirmware [Warner
> help here -- I think that it was forth based IIRC, and Sun may have had
> their hand in it also].  But the key is that it ran on 68K's.   So Next's
> original partition scheme was influenced if not based on some of Bob's
> work, although they did not need the silly BIOS stuff, so they did make it
> all talk to OpenFW (BTW: I think that was still all Next)
>

I'm not sure about Next's partitioning scheme. I do know that Apple adopted
the OpenFirmware partitioning scheme for PPC Mac leading to the APM scheme
which  FreeBSD supports to this day for PowerPC. This was different from
Sun's sunlabels (which differed from BSD labels since they pre-dated them
by half a decade.

By the time the Intel Mac's BIOS had begun to be replaced in the WINTEL
> world by UEFI, Apple committed to using a flavor of it. My old lab partner,
> Guy Sotomayer (*a.k.a.* gss on many mailing lists) was mixed up during
> his time at Apple. He might be able to fill in some details if I ask him.
> I also know that Apple was not happy with how disks were handled with the
> original partition scheme, and by later release started over with their
> partitioning scheme(s) at least once.
>

Yea, GPT partitioning came along a lot later, kinda sort with EFI which
Intel and Apple did (intel for ia64, Apple for 32-bit x86 macs). It's one
of the few partitioning schemes I rather like.


> I've undoubtedly left stuff out, such as much of Linux - but this is all
> part of the UNIX war I was a player.
>

Yea, not to mention the HP, AIX, DG and other partitioning schemes that
grew up in the late 80s and early 90s to have parity with MBR disks at the
feature level, but rarely at the disk format level. A lot of it is
documented, but in hard to find places and I've not done a systematic
survey.

Warner

[-- Attachment #2: Type: text/html, Size: 22409 bytes --]

[TUHS] Re: Question about BSD disklabel history

[G. Branden Robinson]

[-- Attachment #1: Type: text/plain, Size: 3439 bytes --]

At 2023-12-31T16:31:00-0500, Clem Cole wrote:
> Next (ney Apple) started with the Mach code base from CMU.  There was
> a push in the Valley in those days for something called OpenFirmware
> [Warner help here -- I think that it was forth based IIRC, and Sun may
> have had their hand in it also].

I'm not Warner but I owned and operated a few OpenFirmware based
machines.

> But the key is that it ran on 68K's.

I don't think that's the case.  OpenFirmware (OF) ran on SPARC and
PowerPC hardware, at least.  And since it was indeed Forth-based, in
principle it could have been ported practically anywhere (assuming
memory requirements for OF itself were met).

The m68k "Old World" PowerMacs used a different firmware entirely; I
assume boot ROM code descended from the original Macintosh (or even
Lisa, maybe).  The PowerPC "New World" PowerMacs, which immediately
departed from the beige color scheme, did come in with Apple's
acquisition of NeXT.  This may have been the last good thing that Steve
Jobs had a hand in.

On Sun SPARC machines you could get to an OF prompt at any time by
typing "Stop+A" (a.k.a. "L1+A"), using one of the funny left-hand side
function keys on a generation or two (Type 4??) Sun keyboards.  This was
a lot like the "programmer's switch" on some m68k Macs, which was wired
directly to an NMI that MacOS ("Classic") had a fixed vector for.

Open Firmware was an excellent idea; true peripheral portability was
achieved by having "option ROMs" on devices that needed them implemented
in Forth like OF itself.  A lot of flexibility here.  I don't know how
much of a performance price was paid--people did and do enjoy getting
into bootup-time races--but even if it were competitive, the PC side of
the industry would have stridently claimed it wasn't.

A lot of non-x86 Cisco kit in the 2000s (some PowerPC, and _maybe_ some
MIPS stuff too(?)) used some kind of cut-down descendant of OF called
OpenHackWare.  I'm not sure what its dimensions were; it may have mostly
been just the conventions and format that we now recognize as "device
tree" (DTS/DTB), which has perhaps been OpenFirmware's proudest legacy.
It beat the hell out of the PC BIOS alternative for device enumeration,
which always appeared to me to be pure binary chaos with no standard
apart from whatever Microsoft wanted for Windows.

By contrast, OpenFirmware was standardized in IEEE 1275.

> By the time the Intel Mac's BIOS had begun to be replaced in the
> WINTEL world by UEFI, Apple committed to using a flavor of it.

Originally EFI, without the "U".  I first saw these on first-generation
Itanium machines--huge, hot deskside HP boxes whose innards appeared to
use composite foam like that from aircraft wings for heat piping.  The
company I worked for was contracted to help achieve the Linux ia64 port.

I was immediately horrified by EFI's huge step backwards in concept and
implementation.  All this technical progress just to return to
unportable device driver ROMs and a C:\ prompt.  The hatred of the
Wintel duopoly toward elegance or cleanliness in any aspect of computing
cannot be overstated.

I'm weakly hopeful that the RISC-V community will rediscover
OpenFirmware.  It has to date had the good sense to avoid UEFI.  A wise
choice, as if they they hand Intel that much control over their
ecosystem, they will surrender all independence, and possibly the
architecture itself, at least in real silicon.

Regards,
Branden

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[TUHS] Re: Question about BSD disklabel history

[Larry McVoy]

On Sun, Dec 31, 2023 at 04:46:49PM -0600, G. Branden Robinson wrote:
> At 2023-12-31T16:31:00-0500, Clem Cole wrote:
> > Next (ney Apple) started with the Mach code base from CMU.  There was
> > a push in the Valley in those days for something called OpenFirmware
> > [Warner help here -- I think that it was forth based IIRC, and Sun may
> > have had their hand in it also].
> 
> I'm not Warner but I owned and operated a few OpenFirmware based
> machines.
> 
> > But the key is that it ran on 68K's.
> 
> I don't think that's the case.  OpenFirmware (OF) ran on SPARC and
> PowerPC hardware, at least.  And since it was indeed Forth-based, in
> principle it could have been ported practically anywhere (assuming
> memory requirements for OF itself were met).

OpenFirmware is Mitch Bradley's baby.  I believe it ran on 68k Suns, 
there was some sort of boot prom there.  I mostly used it on SPARC.
It was pretty powerful but my personal feeling is the choice of 
Forth didn't help.  Yeah, I get it, Forth is like some weird lisp
and the lisp people love lisp.  What the lisp people don't get is
there are a lot more people who don't love lisp than do love lisp.
And trying to get everyone to love lisp isn't gonna happen.

That said, what else could Mitch have used at the time?  Tcl?
Please, another weird lisp.  Perl?  Not really something that
wants to talk to the bare metal.

It's a serious question, is there anything that Mitch could have
used that would have had wider appeal?

And I agree whole heartedly with the EFI crap being a giant step
backwards.

[TUHS] Re: Question about BSD disklabel history

[Phil Budne]

G. Branden Robinson wrote:
> OpenFirmware (OF) ran on SPARC and PowerPC hardware, at least.

OF had its origin in Sun OpenBoot PROM (OBP), which, ISTR, first
appeared in the SPARCstation 1 (sun4c kernel arch) along with SBus.
Earlier (VME) based SPARC (sun4 kernel arch) systems had only a Sun-3
like boot PROM (single letter commands, no Forth).

[TUHS] Re: Question about BSD disklabel history

[Al Kossow]

On 12/31/23 3:06 PM, Larry McVoy wrote:

> OpenFirmware is Mitch Bradley's baby.  I believe it ran on 68k Suns,
> there was some sort of boot prom there.  I mostly used it on SPARC.

It was developed for SPARC and never ran on Sun-3's.

The point of it was to be processor independent.
I was involved in bringing it up on Power Macs. It was
championed by Ron Hochsprung there, who is also a
FORTH fan.
In theory, you should have been able to use PCI expansion
cards at boot time across vendors by using it, but in practice that didn't
happen because very few vendors ever bothered to support
anything in their BIOS ROMs other than the PC.

And when Apple went with x86 they used the PC's way of the
world, like everyone else.

[TUHS] Re: Question about BSD disklabel history

[G. Branden Robinson]

[-- Attachment #1: Type: text/plain, Size: 865 bytes --]

At 2023-12-31T15:06:15-0800, Larry McVoy wrote:
> On Sun, Dec 31, 2023 at 04:46:49PM -0600, G. Branden Robinson wrote:
> > At 2023-12-31T16:31:00-0500, Clem Cole wrote:
> > > But the key is that it ran on 68K's.
> > 
> > I don't think that's the case.
> 
> OpenFirmware is Mitch Bradley's baby.  I believe it ran on 68k Suns, 
> there was some sort of boot prom there.

At 2023-12-31T18:08:36-0500, Phil Budne wrote:
> OF had its origin in Sun OpenBoot PROM (OBP), which, ISTR, first
> appeared in the SPARCstation 1 (sun4c kernel arch) along with SBus.
> Earlier (VME) based SPARC (sun4 kernel arch) systems had only a Sun-3
> like boot PROM (single letter commands, no Forth).

Thanks for the clarifications, gentlemen.  My first Sun experience was a
SPARC IPC--a bunch of them, actually, in a "freshman engineering lab"...

Regards,
Branden

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[TUHS] Re: Question about BSD disklabel history

[Alec Muffett]

[-- Attachment #1: Type: text/plain, Size: 548 bytes --]

On Sun, 31 Dec 2023, 23:25 Larry McVoy, <lm@mcvoy.com> wrote:

> Yeah, I get it, Forth is like some weird lisp
> and the lisp people love lisp.
>

HERESY! Forth is a RPN stack language.

It's closer to Java bytecode in a whole bunch of ways, but its lineage
comes squarely out of embedded systems like radio telescope control. It's a
solid choice.

Funny I was only posting about it earlier today in respective of the recent
iPhone privilege escalation / exploit:

https://alecmuffett.com/article/108789

...which links to some Forth examples.

-a

[-- Attachment #2: Type: text/html, Size: 1393 bytes --]

[TUHS] Re: Question about BSD disklabel history

[G. Branden Robinson]

[-- Attachment #1: Type: text/plain, Size: 782 bytes --]

[self-follow-up]

At 2023-12-31T16:46:51-0600, G. Branden Robinson wrote:
> The m68k "Old World" PowerMacs used a different firmware entirely; I

Sorry, I was missing the word "and" after "m68k" here.  I certainly
would not deliberately claim that Old World PowerMacs were m68k boxes.

> assume boot ROM code descended from the original Macintosh (or even
> Lisa, maybe).  The PowerPC "New World" PowerMacs, which immediately
> departed from the beige color scheme, did come in with Apple's
> acquisition of NeXT.

...and used OpenFirmware for their boot ROMs universally.

It's my understanding that the OLPC ("One Laptop Per Child") laptop used
OF as well, and would have been one of the more unusual devices to do
so, but I never got to play with one of those.

Regards,
Branden

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[TUHS] Re: Question about BSD disklabel history

[Warner Losh]

[-- Attachment #1: Type: text/plain, Size: 1106 bytes --]

On Sun, Dec 31, 2023, 4:38 PM G. Branden Robinson <
g.branden.robinson@gmail.com> wrote:

> At 2023-12-31T15:06:15-0800, Larry McVoy wrote:
> > On Sun, Dec 31, 2023 at 04:46:49PM -0600, G. Branden Robinson wrote:
> > > At 2023-12-31T16:31:00-0500, Clem Cole wrote:
> > > > But the key is that it ran on 68K's.
> > >
> > > I don't think that's the case.
> >
> > OpenFirmware is Mitch Bradley's baby.  I believe it ran on 68k Suns,
> > there was some sort of boot prom there.
>
> At 2023-12-31T18:08:36-0500, Phil Budne wrote:
> > OF had its origin in Sun OpenBoot PROM (OBP), which, ISTR, first
> > appeared in the SPARCstation 1 (sun4c kernel arch) along with SBus.
> > Earlier (VME) based SPARC (sun4 kernel arch) systems had only a Sun-3
> > like boot PROM (single letter commands, no Forth).
>
> Thanks for the clarifications, gentlemen.  My first Sun experience was a
> SPARC IPC--a bunch of them, actually, in a "freshman engineering lab"...
>

The Sun 3 systems had a similar interface (L1/a etc) but no 4th and a
different set of ROMs.

Warner

Regards,
> Branden
>

[-- Attachment #2: Type: text/html, Size: 1788 bytes --]

[TUHS] Re: Question about BSD disklabel history

[Al Kossow]

>> assume boot ROM code descended from the original Macintosh (or even
>> Lisa, maybe).  The PowerPC "New World" PowerMacs, which immediately
>> departed from the beige color scheme, did come in with Apple's
>> acquisition of NeXT.

Sorry, I don't see the original of this misstatement

The first generation Nubus PowerMacs use essentially an unmodified 68K
ROM for booting, emulating a 68LC040 to do so. Also, Mac and Lisa firmware
are completely different from each other.

The second generation machines with PCI had Open Firmware.

"New World" refers to ROM in RAM systems which have pretty much just
some hardware init and OF in them and started around the time of the iMac

[TUHS] Re: Question about BSD disklabel history

[Bakul Shah]

[-- Attachment #1: Type: text/plain, Size: 1990 bytes --]

On Dec 31, 2023, at 12:07 PM, Warner Losh <imp@bsdimp.com> wrote:
> 
> Yes. Unlike today, the partitions covered the disk in different, overlapping
> ways. And allowed for some parts of the disk to be uncovered by a
> partition. You could then patch the offset and length into the kernel with
> adb and use that area of the disk for swap space.
>  
>> It was also helpful, if you had the drives, to nightly dd
>> your real root to the "a" partition on another, identical
>> drive, so that you could boot the backup root in an emergency.
>> 
>> I don't remember for sure, but I think that Ultrix may have
>> been the first BSD-style system to have disk labels, followed
>> by some version of SunOS. All of that is way in the distant
>> past though: mid- to late 80's.
> 
> When I looked into it years ago, I convinced myself that SunOS
> was the first to have it (since the very first version of SunOS 1.0
> had disk labels) and that all the other vendors followed suit within
> a couple of years. Ultrix-11 had the fixed labels through its EOL.
> I didn't see any disklable stuff in the Ultrix-32M that we have, but
> it was admittedly a quick look.

I wrote the first 2-3 HD drivers for Fortune Systems. I had
the first one up and running[1] by late 1981. IIRC we used an
ioctl to read/write sector 0 of a disk, which is where we
stored partition info. I think by 1983 we were using some
4.1a bits (or at least influenced by it) so likely disklabel
was used by then. The first disk drive was 5MB and cost
$1700. But 5 1/4" disk capacities were growing fast so there
was no question of hardwiring a disktab in source code. I
even had a program that would try to "step" through cylinders
until it ran into errors, to find at usable capacity!

[1] Well, more like walking! Initially DMA didn't work on the
first wirewrap boards so had to use PIO (programmed IO),
at 25KB/s. A quick hack doubled that performance, while
an ST506 disk could do 5Mbits/sec.

[-- Attachment #2: Type: text/html, Size: 7052 bytes --]

[TUHS] Re: Question about BSD disklabel history

[Grant Taylor via TUHS]

[-- Attachment #1: Type: text/plain, Size: 3545 bytes --]

On 12/31/23 11:38, arnold@skeeve.com wrote:
> The different overlapping partitions predates disk labels.

Okay.  That in and of itself doesn't surprise me much that a convention 
of overlapping partitions was carried forward from the driver based 
partitioning into label based partitioning.

> Up to and including 4.3 BSD, to change the size of partitions on a 
> particular disk, you had to recompile the kernel.

So I've learned over the last couple of years as I read more about Unix 
history.

> They were that way so that if you had multiple disks, you could use 
> one for root + swap + some thing small and use another whole disk 
> for a single filesystem.

I'm not understanding how /overlapping/ partitions helps make use of 
portions of disks.

Maybe I should back up and ask for clarification.  What /overlapping/ 
partitions means in this context?

My naive assumption was that partition -- I use that term loosely -- "c" 
overlaps / contains / all other partitions on the disk; "a", "b", and 
maybe "d".

I'm eliding the "c" MBR partition vs "d" entire drive" distinction for 
the moment.

I see some value in the "c" partition being the entire disk as used by 
BSD so that it's possible to point backup / restore / copy utilities at 
the entire disk.

But I don't understand value in having partitions overlap / contain each 
other's blocks, save for backup via "c".

I do see some value in extending the "c" is the entire MBR partition 
methodology to "d" is the entire disk containing multiple MBR 
partitions.  Again, the value seems to be in backup and recovery.

But I still simply do not understand why I would ever want partition "e" 
to be blocks 100-199, partition "f" to be blocks 195-299, and partition 
"g" to be blocks 295-399.  What value is there in having partitions e, 
f, and g overlap each other?

I get dd if=/dev/<something>0c of=/dev/rmt.  Or even /dev/<something>0d.

I fail to understand why I'd ever want other partitions to overlap.

> It was also helpful, if you had the drives, to nightly dd your real 
> root to the "a" partition on another, identical drive, so that you 
> could boot the backup root in an emergency.

Sure.  But I don't see what that has to do with overlapping partitions.

I'd naively think that I could do something like the following:

    dd if=/dev/<something>0a of=/dev/<something>1a

And get the same effect.

> I am guessing that the original conventions date back to V7 or 32V, 
> but one would have to go looking at code to be sure.

"a" for root makes some intuitive sense as the root file system is 
required to do anything else.

Then when you want swap, the next partition is "b".

Wanting another partition that is the entire disk (as seen by BSD) makes 
some logical sense to me too, so "c".

Were subsequent partitions sort of used as needed and had less 
consistency?  Especially when "d" because the entire disk containing 
multiple MBR partitions when "c" was restricted to the MBR partition the 
label was in?

Aside:

Would that mean that the following "d" partitions would be the same 
thing, as in the entire /dev/ad0 disk?

/dev/ad0s0d
/dev/ad0s1d

Wherein I'm borrowing the FreeBSD slice nomenclature -- as I understand 
it -- to identify the first (zero) and second (one) MBR partition on 
/dev/ad0

History and how we got to where we are today can be both very confusing 
and even more enlightening once you understand it.  What's more is that 
once you understand it, things start making more intuitive sense when 
you look at them.



-- 
Grant. . . .

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4033 bytes --]

[TUHS] Re: Question about BSD disklabel history

[Warner Losh]

[-- Attachment #1: Type: text/plain, Size: 4425 bytes --]

8

On Sun, Dec 31, 2023, 5:26 PM Grant Taylor via TUHS <tuhs@tuhs.org> wrote:

> On 12/31/23 11:38, arnold@skeeve.com wrote:
> > The different overlapping partitions predates disk labels.
>
> Okay.  That in and of itself doesn't surprise me much that a convention
> of overlapping partitions was carried forward from the driver based
> partitioning into label based partitioning.
>


Many editors dont let you do this...

> Up to and including 4.3 BSD, to change the size of partitions on a
> > particular disk, you had to recompile the kernel.
>
> So I've learned over the last couple of years as I read more about Unix
> history.
>
> > They were that way so that if you had multiple disks, you could use
> > one for root + swap + some thing small and use another whole disk
> > for a single filesystem.
>
> I'm not understanding how /overlapping/ partitions helps make use of
> portions of disks.
>


Maybe I should back up and ask for clarification.  What /overlapping/
> partitions means in this context?
>

It means you either use one set of non overlapping partitions or another
set. They were setup in clever ways

My naive assumption was that partition -- I use that term loosely -- "c"
> overlaps / contains / all other partitions on the disk; "a", "b", and
> maybe "d".
>
> I'm eliding the "c" MBR partition vs "d" entire drive" distinction for
> the moment.
>
> I see some value in the "c" partition being the entire disk as used by
> BSD so that it's possible to point backup / restore / copy utilities at
> the entire disk.
>
> But I don't understand value in having partitions overlap / contain each
> other's blocks, save for backup via "c".
>
> I do see some value in extending the "c" is the entire MBR partition
> methodology to "d" is the entire disk containing multiple MBR
> partitions.  Again, the value seems to be in backup and recovery.
>
> But I still simply do not understand why I would ever want partition "e"
> to be blocks 100-199, partition "f" to be blocks 195-299, and partition
> "g" to be blocks 295-399.  What value is there in having partitions e,
> f, and g overlap each other?
>
> I get dd if=/dev/<something>0c of=/dev/rmt.  Or even /dev/<something>0d.
>
> I fail to understand why I'd ever want other partitions to overlap.
>

It's more like you can use two or three partitions with non overlapping
sets that cover the whole disk.

> It was also helpful, if you had the drives, to nightly dd your real
> > root to the "a" partition on another, identical drive, so that you
> > could boot the backup root in an emergency.
>
> Sure.  But I don't see what that has to do with overlapping partitions.
>
> I'd naively think that I could do something like the following:
>
>     dd if=/dev/<something>0a of=/dev/<something>1a
>
> And get the same effect.
>
> > I am guessing that the original conventions date back to V7 or 32V,
> > but one would have to go looking at code to be sure.
>
> "a" for root makes some intuitive sense as the root file system is
> required to do anything else.
>
> Then when you want swap, the next partition is "b".
>
> Wanting another partition that is the entire disk (as seen by BSD) makes
> some logical sense to me too, so "c".
>
> Were subsequent partitions sort of used as needed and had less
> consistency?  Especially when "d" because the entire disk containing
> multiple MBR partitions when "c" was restricted to the MBR partition the
> label was in?
>
> Aside:
>
> Would that mean that the following "d" partitions would be the same
> thing, as in the entire /dev/ad0 disk?
>
> /dev/ad0s0d
> /dev/ad0s1d
>
> Wherein I'm borrowing the FreeBSD slice nomenclature -- as I understand
> it -- to identify the first (zero) and second (one) MBR partition on
> /dev/ad0
>

Yes.

But ancient Unix didn’t have nested partitioning schemes like FreeBSD
supports...

History and how we got to where we are today can be both very confusing
> and even more enlightening once you understand it.  What's more is that
> once you understand it, things start making more intuitive sense when
> you look at them.
>

Think more of a limited number of ways to mix and match for greater
flexibility w/o editing the tables.

A silly example: a is first 2/3 of the disk. B is 2nd 2/3, c d and e are
1/3 each.

Warner

-- 
> Grant. . . .
>

[-- Attachment #2: Type: text/html, Size: 6932 bytes --]

[TUHS] Re: Question about BSD disklabel history

[Grant Taylor via TUHS]

[-- Attachment #1: Type: text/plain, Size: 1024 bytes --]

On 12/31/23 20:22, Warner Losh wrote:
> It means you either use one set of non overlapping partitions or 
> another set. They were setup in clever ways
> 
> It's more like you can use two or three partitions with non overlapping 
> sets that cover the whole disk.

...

> Think more of a limited number of ways to mix and match for greater 
> flexibility w/o editing the tables.
> 
> A silly example: a is first 2/3 of the disk. B is 2nd 2/3, c d and 
> e are 1/3 each.

Okay.

This makes more sense.  Pre-define overlapping partitions but only use 
non-overlapping.  Sort of like tool boxes that come with a bunch of 
supports for dividers, but you can only put so many dividers in the 
pre-defined spots, but contents between the dividers doesn't overlap.

Sort of like how many squares are in this bigger grid of dots that is 
itself a big square.

> But ancient Unix didn’t have nested partitioning schemes like 
> FreeBSD supports...

Understood and appreciated.



-- 
Grant. . . .

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4033 bytes --]

[TUHS] Re: Question about BSD disklabel history

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 4496 bytes --]

Warner -- thanks...

On Sun, Dec 31, 2023 at 5:07 PM Warner Losh <imp@bsdimp.com> wrote:

>
>
> Except I can't find any of this in our V7m or ultrix-11 that we have. This
> is 2.1. And there all the device drivers still have the fixed tables:
> ....
>
> and the ultrix-11 3.1 sources we have are similar, but move all of the
> above into /usr/sys/conf/dksizes.c.
>
I remember dksizes.c -- I'll take your word for it being 3.X, not 2.X; I'm
pretty sure that was the start of the work in Merrimack by Fred Cantor to
make things more independent. Shannon would have been aware of all of it,
and before he went to Sun, he and Jean were working with the folks at CSRG
(Sam, in particular), moving some of the DEC I/O support into BSD.   Bill
Munson convinced his management that it was worth it for DEC to at least
make sure the DEC peripherals were being well handled and in mostly the
same manner.  IIRC Jean spent 6-9 months embedded in the CSRG working on
much of that.



> The early disktab stuff encoded the default tables into a text file to
> make it easier to cope with all the different types of disks, but didn't
> affect what we know as bsdlabels until after 4.3BSD.
>
Right.



> The naming convention was there, but the 'write bits to the disk to
> describe partitioning' I couldn't find at all in ultrix. I think you may be
> confusing that stuff (which was a very clever way to cope with all these
> disks that have different partitioning in a increasingly generic way) with
> disk labeling which I think.
>
Quite possible.

The key is that partitioning the disk to allow its use for different things
and disk geometry support get all mixed together in the different schemes.
 As we discussed, it often happens in multiple places (since the ROMs, like
the PC's BIOS, need at least some of the info at boot), and the loaded OS
(particularly ones with multiple OSs on them) might want to do something
completely different.     This is why Grant's question is a little hard to
answer directly - as I said, it is a long, slow trip.

UNIX Partitioning, like what Dennis did, came first.  If you look at PDP-11
and later Vaxen, the "disk support" for booting is pretty crude and built
into the ROM -- *i.e*., boot RP0 or RK0 -- the "disk geometry" is hidden in
the Boot ROM.   But when so many disks start to show up using the same
controllers, the ROM needs to be smarter.  So, some way to encode geometry
is needed.  But Partitioning for the OS is still something that is handy
and so often got put into the same support (such as the PC's BIOS tables -
which were a good idea, poorly executed).






> Sun was first to market with.
>
To be fair, Masscomp's "disk geometry" code that Paul Cantrell wrote
pre-dated Sun by at least a year or more.  I did not include it in my
history, as it is private to their boot ROM.  Nice scheme, actually - but
proprietary, and I don't think any of the ideas went anywhere else - other
than later Sun ROMs supported a similar functionality, and they would have
at least seen them at customers and known about it, particularly since Sun
picked up the Xylogic disk and tape controllers that they developed for
Masscomp originally (Paul spent many hours at Xylogics helping with their
Microcode).    The point is, by that time, the proliferation of different
disk manufacturers -- something to make the boot ROMs and OS's more
independent was forced on the different systems providers if they were
going to have any chance of being able to be flexible in the market/field.



>
> Maybe you can help me find it?
>
I'll see what I can dig up.


>
> I did find the SunOS 0.x images (UniSoft V7 port) didn't appear have a
> disk partition, but 1.0 (post 4BSD pre 4.1BSD) and later did. That puts it
> at 1982. Although based off BSD, the disk label stuff wasn't in BSD for
> another 5 years when the BSD folks started the Tahoe port.
>
Right, this is all of the MK0, CSRG, and Sun connection via Fred, Bill, and
Sam.  At a minimum, they knew what the other was doing/had done, and some
of it definitely migrated.

WRT to OpenFW, my memory is that Larry points out that Sun was the primary
driver.   But a lot of the Motorola club got interested in using it, too,
during the "JAWS" timeframe.  I am pretty sure that there were versions for
68K, 88K, and PPC, as well as the SPARC version from Sun.   Somewhere I
have early source distribution.

ᐧ
ᐧ

[-- Attachment #2: Type: text/html, Size: 8835 bytes --]

[TUHS] Re: Question about BSD disklabel history

[Warner Losh]

[-- Attachment #1: Type: text/plain, Size: 6872 bytes --]

On Mon, Jan 1, 2024 at 9:00 AM Clem Cole <clemc@ccc.com> wrote:

>
> Warner -- thanks...
>
> On Sun, Dec 31, 2023 at 5:07 PM Warner Losh <imp@bsdimp.com> wrote:
>
>>
>>
>> Except I can't find any of this in our V7m or ultrix-11 that we have.
>> This is 2.1. And there all the device drivers still have the fixed tables:
>> ....
>>
>> and the ultrix-11 3.1 sources we have are similar, but move all of the
>> above into /usr/sys/conf/dksizes.c.
>>
> I remember dksizes.c -- I'll take your word for it being 3.X, not 2.X; I'm
> pretty sure that was the start of the work in Merrimack by Fred Cantor to
> make things more independent. Shannon would have been aware of all of it,
> and before he went to Sun, he and Jean were working with the folks at CSRG
> (Sam, in particular), moving some of the DEC I/O support into BSD.   Bill
> Munson convinced his management that it was worth it for DEC to at least
> make sure the DEC peripherals were being well handled and in mostly the
> same manner.  IIRC Jean spent 6-9 months embedded in the CSRG working on
> much of that.
>

I don't have access to anything but Ultrix-11 3.1 sources.

But there's no disk label stuff there. The dksizes.c file is just the
individual partition sizes removed from the respective driver and added to
dksize.c, presumably so it could be rebuilt easily w/o giving away any Unix
IP.

And the disktab stuff is way better than what came before. It enshrined
into a text file what the right partitions should be for each of the
devices. And even that took a lot of effort and would feed into the
defaults for partitioning disks when that came along.


> The early disktab stuff encoded the default tables into a text file to
>> make it easier to cope with all the different types of disks, but didn't
>> affect what we know as bsdlabels until after 4.3BSD.
>>
> Right.
>
>
>
>> The naming convention was there, but the 'write bits to the disk to
>> describe partitioning' I couldn't find at all in ultrix. I think you may be
>> confusing that stuff (which was a very clever way to cope with all these
>> disks that have different partitioning in a increasingly generic way) with
>> disk labeling which I think.
>>
> Quite possible.
>
> The key is that partitioning the disk to allow its use for different
> things and disk geometry support get all mixed together in the different
> schemes.   As we discussed, it often happens in multiple places (since the
> ROMs, like the PC's BIOS, need at least some of the info at boot), and the
> loaded OS (particularly ones with multiple OSs on them) might want to do
> something completely different.     This is why Grant's question is a
> little hard to answer directly - as I said, it is a long, slow trip.
>

Yes. There's all kinds of compounding factors, as we saw play out when
old-school BSD disklabels started existing in a MBR world.


> UNIX Partitioning, like what Dennis did, came first.  If you look at
> PDP-11 and later Vaxen, the "disk support" for booting is pretty crude and
> built into the ROM -- *i.e*., boot RP0 or RK0 -- the "disk geometry" is
> hidden in the Boot ROM.   But when so many disks start to show up using the
> same controllers, the ROM needs to be smarter.  So, some way to encode
> geometry is needed.  But Partitioning for the OS is still something that is
> handy and so often got put into the same support (such as the PC's BIOS
> tables - which were a good idea, poorly executed).
>

Yes. Dennis did it in like 75, and it wasn't until the mid to late 80s that
most disks had their partitioning encoded on the drive itself. Some of the
encoding schemes included the geometry of the device (some implicitly, some
explicitly), while others didn't. It was a real mess.


> Sun was first to market with.
>>
> To be fair, Masscomp's "disk geometry" code that Paul Cantrell wrote
> pre-dated Sun by at least a year or more.  I did not include it in my
> history, as it is private to their boot ROM.  Nice scheme, actually - but
> proprietary, and I don't think any of the ideas went anywhere else - other
> than later Sun ROMs supported a similar functionality, and they would have
> at least seen them at customers and known about it, particularly since Sun
> picked up the Xylogic disk and tape controllers that they developed for
> Masscomp originally (Paul spent many hours at Xylogics helping with their
> Microcode).    The point is, by that time, the proliferation of different
> disk manufacturers -- something to make the boot ROMs and OS's more
> independent was forced on the different systems providers if they were
> going to have any chance of being able to be flexible in the market/field.
>

I have no data on Masscomp's / Xylogic's ROMs, alas. And I know that
getting the geometry from the disk lived in the driver for a long time
because of issues like this, and why bootstrapping these machines were
always a complex affair. I don't recall there being any Xylogic specific
labels on the drives we had at Solbourne, though. And did Masscomp's stuff
write data to the media that was then used to figure out the geometry /
partitioning? Or was it still a largish table of drive model X has geometry
x/y/z (which PC BIOS had initially after the AT mainstreamed winchester
disks there).


>
>
>>
>> Maybe you can help me find it?
>>
> I'll see what I can dig up.
>

Yea. I'd be quite keen on the details before Sun's scheme, which was the
first I could find in the artifacts we have today. But there's so many
artifacts they are all hard to go through. Especially scanned manuals that
have no OCR encodings...

I did find the SunOS 0.x images (UniSoft V7 port) didn't appear have a disk
>> partition, but 1.0 (post 4BSD pre 4.1BSD) and later did. That puts it at
>> 1982. Although based off BSD, the disk label stuff wasn't in BSD for
>> another 5 years when the BSD folks started the Tahoe port.
>>
> Right, this is all of the MK0, CSRG, and Sun connection via Fred, Bill,
> and Sam.  At a minimum, they knew what the other was doing/had done, and
> some of it definitely migrated.
>

What's MK0? Is that the group at DEC that did Unix?


> WRT to OpenFW, my memory is that Larry points out that Sun was the primary
> driver.   But a lot of the Motorola club got interested in using it, too,
> during the "JAWS" timeframe.  I am pretty sure that there were versions for
> 68K, 88K, and PPC, as well as the SPARC version from Sun.   Somewhere I
> have early source distribution.
>
>
Yea, I have a memory of the Sun 3 (68k) machines having a different ROM
than the Sun 4 (sparc) machines in the late 80s/early 90s with similar
interfaces, but the Sun 3's being simpler. Maybe I just worked with older
Sun 3's that didn't have a newer OpenFirmware.

Warner

[-- Attachment #2: Type: text/html, Size: 11316 bytes --]

[TUHS] Re: Question about BSD disklabel history

[Chet Ramey]

On 1/2/24 1:49 PM, Warner Losh wrote:

>     Right, this is all of the MK0, CSRG, and Sun connection via Fred, Bill,
>     and Sam.  At a minimum, they knew what the other was doing/had done,
>     and some of it definitely migrated.
> 
> 
> What's MK0? Is that the group at DEC that did Unix?

I don't know what MK0 is either, but Bill Shannon and Sam Leffler had been
friends since they were both grad students here at CWRU, so I would assume
they kept in touch about what they were doing.

Chet
-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
		 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet@case.edu    http://tiswww.cwru.edu/~chet/

[TUHS] Re: Question about BSD disklabel history

[Dan Cross]

On Tue, Jan 2, 2024 at 2:00 PM Warner Losh <imp@bsdimp.com> wrote:
> [snip]
> Yea, I have a memory of the Sun 3 (68k) machines having a different ROM than the Sun 4 (sparc) machines in the late 80s/early 90s with similar interfaces, but the Sun 3's being simpler. Maybe I just worked with older Sun 3's that didn't have a newer OpenFirmware.

Sun 3/50 and Sun 3/60 definitely had a different PROM arrangement than
SPARCstations. However, I have a vague memory that the Sun 3/80 used
OpenFirmware. The 3/80 (and similar Sun-3x machines introduced at the
same time) would have been among the last, if not the last,
MC68k-based Sun3 machine. I may be wrong, though; it's been a while.
Regardless, I vaguely remember the "ok" prompt on a 3/80.

        - Dan C.

[TUHS] Re: Question about BSD disklabel history

[Jim Capp]

[-- Attachment #1: Type: text/plain, Size: 1180 bytes --]

The "ok" prompt is from "OpenBoot", a boot manager, written in FORTH: 


https://docs.oracle.com/cd/E19253-01/816-1177-10/816-1177-10.pdf 


Jim 




From: "Dan Cross" <crossd@gmail.com> 
To: "Warner Losh" <imp@bsdimp.com> 
Cc: "Grant Taylor" <gtaylor@tnetconsulting.net>, "The Unix Heritage Society" <tuhs@tuhs.org> 
Sent: Tuesday, January 2, 2024 2:50:52 PM 
Subject: [TUHS] Re: Question about BSD disklabel history 

On Tue, Jan 2, 2024 at 2:00 PM Warner Losh <imp@bsdimp.com> wrote: 
> [snip] 
> Yea, I have a memory of the Sun 3 (68k) machines having a different ROM than the Sun 4 (sparc) machines in the late 80s/early 90s with similar interfaces, but the Sun 3's being simpler. Maybe I just worked with older Sun 3's that didn't have a newer OpenFirmware. 

Sun 3/50 and Sun 3/60 definitely had a different PROM arrangement than 
SPARCstations. However, I have a vague memory that the Sun 3/80 used 
OpenFirmware. The 3/80 (and similar Sun-3x machines introduced at the 
same time) would have been among the last, if not the last, 
MC68k-based Sun3 machine. I may be wrong, though; it's been a while. 
Regardless, I vaguely remember the "ok" prompt on a 3/80. 

- Dan C. 

[-- Attachment #2: Type: text/html, Size: 2882 bytes --]

[TUHS] Re: Question about BSD disklabel history

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 1691 bytes --]

On Tue, Jan 2, 2024 at 2:31 PM Chet Ramey <chet.ramey@case.edu> wrote:

> On 1/2/24 1:49 PM, Warner Losh wrote:
>
> >     Right, this is all of the MK0, CSRG, and Sun connection via Fred,
> Bill,
> >     and Sam.  At a minimum, they knew what the other was doing/had done,
> >     and some of it definitely migrated.
> >
> >
> > What's MK0? Is that the group at DEC that did Unix?
>
DEC Merrimack NH Site - where Telephone Industries Group (TIG) and Custom
Special Systems (CSS) were located. It was originally the site dedicated to
supporting DEC's largest customer - AT & T.  Bill Munson originally ran TIG.
Armando, Jean, Shannon, Cantor, *et al.* worked in TIG.  Those folks
eventually moved to ZK03 - Spit Brook Road, in Nashua, with the other OS
teams when Ultrix was created.

Sorry, old habits are hard to change.  Except for the Mill itself, many, if
not most, of us from the old days still refer to the different DEC sites
(particularly the original New Englands) by their site code.  For instance,
my old offices were in MR02 (Marlboro, MA) and ZK03.

Years later, the compiler team (in ZK02) was sold twice, and they never
moved their desk. Hence, when I later joined Intel, my desk was placed in
ZK02 with the compiler folks like Paul W. In fact, when Intel finally moved
us out, we realized that folks like Steve Hobbs had not moved his desk in
30 years.

I don't know what MK0 is either, but Bill Shannon and Sam Leffler had been
> friends since they were both grad students here at CWRU, so I would assume
> they kept in touch about what they were doing.
>
Indeed, and having been friends with them both, I can verify that they did.
ᐧ

[-- Attachment #2: Type: text/html, Size: 3650 bytes --]

[TUHS] Re: Question about BSD disklabel history

[Dan Cross]

On Tue, Jan 2, 2024 at 2:55 PM Jim Capp <jcapp@anteil.com> wrote:
> The "ok" prompt is from "OpenBoot",

Yes.  See Phil Bunde's message earlier in this thread.

        - Dan C.

> a boot manager, written in FORTH:
>
> https://docs.oracle.com/cd/E19253-01/816-1177-10/816-1177-10.pdf

[TUHS] Re: Question about BSD disklabel history

[Dan Cross]

On Tue, Jan 2, 2024 at 2:50 PM Dan Cross <crossd@gmail.com> wrote:
> On Tue, Jan 2, 2024 at 2:00 PM Warner Losh <imp@bsdimp.com> wrote:
> > [snip]
> > Yea, I have a memory of the Sun 3 (68k) machines having a different ROM than the Sun 4 (sparc) machines in the late 80s/early 90s with similar interfaces, but the Sun 3's being simpler. Maybe I just worked with older Sun 3's that didn't have a newer OpenFirmware.
>
> Sun 3/50 and Sun 3/60 definitely had a different PROM arrangement than
> SPARCstations. However, I have a vague memory that the Sun 3/80 used
> OpenFirmware. The 3/80 (and similar Sun-3x machines introduced at the
> same time) would have been among the last, if not the last,
> MC68k-based Sun3 machine. I may be wrong, though; it's been a while.
> Regardless, I vaguely remember the "ok" prompt on a 3/80.

Hmm: correcting myself here. I can find no evidence that the 3/80 used
OBP, but I can find documentation to the contrary (mostly in the form
of documentation on how to rewrite the HostID when the NVRAM battery
fails...Gah, the things one not only forgets but actively blots out of
one's memory). It seems likely that Sun never shipped OF on 68k.

        - Dan C.

[TUHS] Re: Question about BSD disklabel history

[Dan Cross]

On Sun, Dec 31, 2023 at 6:25 PM Larry McVoy <lm@mcvoy.com> wrote:
>[snip]
> OpenFirmware is Mitch Bradley's baby.  I believe it ran on 68k Suns,
> there was some sort of boot prom there.  I mostly used it on SPARC.
> It was pretty powerful but my personal feeling is the choice of
> Forth didn't help.  Yeah, I get it, Forth is like some weird lisp
> and the lisp people love lisp.  What the lisp people don't get is
> there are a lot more people who don't love lisp than do love lisp.
> And trying to get everyone to love lisp isn't gonna happen.
>
> That said, what else could Mitch have used at the time?  Tcl?
> Please, another weird lisp.  Perl?  Not really something that
> wants to talk to the bare metal.
>
> It's a serious question, is there anything that Mitch could have
> used that would have had wider appeal?

The thing about FORTH isn't that it's Lisp-like (as Alec mentioned),
though its supporters do often exhibit a fervor reminiscent of
Lispers.

Rather, I think FORTH shows up in places like this because it's
possible to write _incredibly_ lean threaded-code interpreters for it
that can run in really primitive environments, so you can shove a
really small interpreter in a ROM and keep your big CPU in reset while
you run it out of a tiny SRAM on an 8-bit microcontroller or something
until you've got enough of an environment going to train DRAM and
transfer over to the real thing. E.g., something like:
https://pygmy.utoh.org/3ins4th.html

What could you have done differently? Meh; I don't really know, but see below.

> And I agree whole heartedly with the EFI crap being a giant step
> backwards.

Ironically, the UEFI people have done something _similar_ to OF in the
form of AML (ACPI Machine Language), which is a byte-code
serialization ASL (ACPI Source Language); presumably that's system
independent. The idea of a p-code representation is about where the
similarity ends, though: AML exposes a mechanism to talk to the UEFI
OS for a whole slew of stuff, which is rather unlike what OF did
(though I again have a vague memory that on SPARCstations some devices
went through the PROM monitor; the text console, for example, and
maybe the keyboard? It's been too long now to properly remember).

Anyway, an alternative to FORTH might have been a well-defined p-code
and a little VM in ROM to drive it.  Then one could compile to that
using whatever language one liked (and was willing to write a compiler
for!). Perhaps the feeling is that that is what FORTH was; for that I
guess I don't see any reason one couldn't transpile to FORTH from some
other language.

        - Dan C.

[TUHS] Re: Question about BSD disklabel history

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 1624 bytes --]

I never said Sun or anyone else actually shipped it -- only that it was
around, and there was a lot of discussion about using it/trying to create a
boot ROM standard one of some type.   IIRC I remember that NCR was
messing with it for their 88K boxes (which I had a hand in killing). But
NCR switched to all x86-based (their "7 levels "strategy) for everything
from Cash Registers/ATMs to their largest Database machines.  They had
always been in the "better and cheaper than IBM" business, particularly in
their computer products. [1]  So after they switch, their boots start to be
all BIOS and EBIOS flavored.

But other Motorola folks I was working with at the time were exploring OF.
I had it on a 68K and another 88K prototype machine at one point (I
remember the "ok" prompt), and I remember I somehow had a source
distribution tape.  I was an independent consultant then and wrote a couple
of drivers under contract for some different folks. I can not think of the
firm's name now -- there was another 68K workstation firm here in Lowell,
MA.  They actually wrote and made a MacOS emulator work with their UNIX
under covers.   I've forgotten most of the details now, but they may have
been one of the folks who was playing with OF.

My memory is that I first encountered it at one of my consulting gigs.

Clem



1] When they threw the x86 everywhere die, they also started using
Microchannel (the only large licensee I know of for it -- everyone else
stayed with ISA/EISA and then later PCI from DEC/AMD) - tossing out all the
Moto IP like VME that so many UNIX systems had used).
ᐧ

[-- Attachment #2: Type: text/html, Size: 3198 bytes --]

[TUHS] Re: Question about BSD disklabel history

[Dan Cross]

On Tue, Jan 2, 2024 at 3:50 PM Clem Cole <clemc@ccc.com> wrote:
> I never said Sun or anyone else actually shipped it -- only that it was around, and there was a lot of discussion about using it/trying to create a boot ROM standard one of some type.

Sorry! I didn't mean to imply that you had; rather, I thought that
they had because I thought I remembered seeing it on Sun-3x, but I
think I'm just misremembering.

> IIRC I remember that NCR was messing with it for their 88K boxes (which I had a hand in killing). But NCR switched to all x86-based (their "7 levels "strategy) for everything from Cash Registers/ATMs to their largest Database machines.  They had always been in the "better and cheaper than IBM" business, particularly in their computer products. [1]  So after they switch, their boots start to be all BIOS and EBIOS flavored.

Poor suckers. :-(

> But other Motorola folks I was working with at the time were exploring OF.  I had it on a 68K and another 88K prototype machine at one point (I remember the "ok" prompt), and I remember I somehow had a source distribution tape.  I was an independent consultant then and wrote a couple of drivers under contract for some different folks. I can not think of the firm's name now -- there was another 68K workstation firm here in Lowell, MA.  They actually wrote and made a MacOS emulator work with their UNIX under covers.   I've forgotten most of the details now, but they may have been one of the folks who was playing with OF.
>
> My memory is that I first encountered it at one of my consulting gigs.

That's cool. It's sad that, around the time of the 386, we didn't have
more x86 vendors doing something like OF. Maybe we could have avoided
a bunch of big messes. Maybe not.

> 1] When they threw the x86 everywhere die, they also started using Microchannel (the only large licensee I know of for it -- everyone else stayed with ISA/EISA and then later PCI from DEC/AMD) - tossing out all the Moto IP like VME that so many UNIX systems had used).

I knew a bunch of folks in the early/mid 90s who were pretty tight in
the IBM camp; RS/6000's were shipping on MCA and they said they saw
better IO performance than with PCI for a couple of generations.

        - Dan C.

[TUHS] Re: Question about BSD disklabel history

[John Cowan]

[-- Attachment #1: Type: text/plain, Size: 733 bytes --]

On Tue, Jan 2, 2024 at 3:49 PM Dan Cross <crossd@gmail.com> wrote:

Perhaps the feeling is that that is what FORTH was; for that I
> guess I don't see any reason one couldn't transpile to FORTH from some
> other language.


Henry Baker wrote a beautiful little paper (see
https://www.plover.com/~mjd/misc/hbaker-archive/ForthStack.html>) showing
the mapping between Linear Lisp (a Lisp in which all variables are
referenced once and only once) and a Forth-style "frameless stack".  There
is also a discussion of Forth as a set of linear combinators: Manfred von
Thun's Joy <https://en.wikipedia.org/wiki/Joy_(programming_language)> is a
Forth-like using nested lists in which the stack is the top-level list.

>
>

[-- Attachment #2: Type: text/html, Size: 1708 bytes --]

[TUHS] Re: Question about BSD disklabel history

[Theodore Ts'o]

On Tue, Jan 02, 2024 at 03:48:28PM -0500, Dan Cross wrote:
> On Sun, Dec 31, 2023 at 6:25 PM Larry McVoy <lm@mcvoy.com> wrote:
> 
> Ironically, the UEFI people have done something _similar_ to OF in the
> form of AML (ACPI Machine Language), which is a byte-code
> serialization ASL (ACPI Source Language); presumably that's system
> independent. The idea of a p-code representation is about where the
> similarity ends, though: AML exposes a mechanism to talk to the UEFI
> OS for a whole slew of stuff, which is rather unlike what OF did

ACPI (Advanced Configuration and Power Interface) predates UEFI.  ACPI
was released in 1996, and was originally intended to be a replacement
for APM (Advanced Power Management).  With APM, the OS would crowbar
itself into x86 Real mode in order to call into the APM BIOS in order
to suspend the laptop, fetch power management events from the APM,
etc.  Later on, APM grew 32-bit protected mode interfaces, but
effectively, the OS would completely lose control of the thread of
execution while running APM BIOS code, which made life "exciting" for
OS engineers.

So ACPI was originally intended to solve a problem, where the OS could
incorporate a byte code interpreter to do those things that would be
very hardware specific (for example, how to diddle the various random
I/O ports on a HP laptop to bring the perpipherals into a low power
state, which of course would be completely different on a Dell or IBM
laptop.  Previously, the APM bios was the abstraction layer; ACPI was
the new abstraction layer.

UEFI came later.  UEFI was the second system disease replacement for
EFI (extensible firmware interface), which Intel had developed.  EFI
was an implementation that attempted to retain full backwards
compatibility with the de factor standard originally established by
the IBM BIOS.  UEFI was an attempt to completely rework the interface
between the OS boot code (which before would talk to the BIOS
interface) with the all new singing, all dancing UEFI interface.
Being a second system, of course it was made horribly complicated, so
as to meet all of the requirements that might be dreamed up by the
industry committee that put it together.

Normally, once the OS has gotten to a certain point in its OS
initialization, the OS can "terminate UEFI services".  At that point,
the OS can no longer call into the UEFI functions --- but it also
doesn't have to worry about the UEFI code servicing an interrupt take
taking control of the CPU away from the OS.

However, the OS can still execute various ACPI functions, assuming the
OS has its own AML interpreter.  (Of course, the UEFI BIOS has its own
AML interpreter --- and scheduler --- and everything else that a
simple OS might have.)  Ron Minnich has a number of really good
presentations about the "joys" of working with UEFI.  See the YouTube
video, "Replace your Expoit-ridden Firmware with Linux"[1] and prepare to
be horrified about what horrors lurks in the heart of UEFI....

[1] https://www.youtube.com/watch?v=iffTJ1vPCSo

So you can very much be *not* using UEFI, and still be using ACPI ---
either if you are using a pre-2004 computer, or if you are using a
more modern platform which uses LinuxBoot.

						- Ted

[TUHS] Re: Question about BSD disklabel history

[Warner Losh]

[-- Attachment #1: Type: text/plain, Size: 6609 bytes --]

On Tue, Jan 2, 2024 at 8:33 PM Theodore Ts'o <tytso@mit.edu> wrote:

> On Tue, Jan 02, 2024 at 03:48:28PM -0500, Dan Cross wrote:
> > On Sun, Dec 31, 2023 at 6:25 PM Larry McVoy <lm@mcvoy.com> wrote:
> >
> > Ironically, the UEFI people have done something _similar_ to OF in the
> > form of AML (ACPI Machine Language), which is a byte-code
> > serialization ASL (ACPI Source Language); presumably that's system
> > independent. The idea of a p-code representation is about where the
> > similarity ends, though: AML exposes a mechanism to talk to the UEFI
> > OS for a whole slew of stuff, which is rather unlike what OF did
>
> ACPI (Advanced Configuration and Power Interface) predates UEFI.  ACPI
> was released in 1996, and was originally intended to be a replacement
> for APM (Advanced Power Management).  With APM, the OS would crowbar
> itself into x86 Real mode in order to call into the APM BIOS in order
> to suspend the laptop, fetch power management events from the APM,
> etc.  Later on, APM grew 32-bit protected mode interfaces, but
> effectively, the OS would completely lose control of the thread of
> execution while running APM BIOS code, which made life "exciting" for
> OS engineers.
>
> So ACPI was originally intended to solve a problem, where the OS could
> incorporate a byte code interpreter to do those things that would be
> very hardware specific (for example, how to diddle the various random
> I/O ports on a HP laptop to bring the perpipherals into a low power
> state, which of course would be completely different on a Dell or IBM
> laptop.  Previously, the APM bios was the abstraction layer; ACPI was
> the new abstraction layer.
>
> UEFI came later.  UEFI was the second system disease replacement for
> EFI (extensible firmware interface), which Intel had developed.  EFI
> was an implementation that attempted to retain full backwards
> compatibility with the de factor standard originally established by
> the IBM BIOS.  UEFI was an attempt to completely rework the interface
> between the OS boot code (which before would talk to the BIOS
> interface) with the all new singing, all dancing UEFI interface.
> Being a second system, of course it was made horribly complicated, so
> as to meet all of the requirements that might be dreamed up by the
> industry committee that put it together.
>

Not quite. EFI was the second system. It didn't try to retain any backwards
compatibility with the original IBM BIOS. It was completely different. Early
Intel macs used it to boot. Intel and Apple worked on it together. Intel
wrote it
for the Itanium IA64 fiasco because it couldn't use the IBM BIOS interfaces
and such to boot its new machine because IA64 lacked many of the low-level
tricks that you used to interface. to the BIOS. It was an attempt to redo
things
from scratch.

Intel did then turn it over to something more public to manage, but it was
well on
its way to being an insane mess before it morphed into UEFI. But even in EFI
times it was clearly on the way to being the trainwreck it became.


> Normally, once the OS has gotten to a certain point in its OS
> initialization, the OS can "terminate UEFI services".  At that point,
> the OS can no longer call into the UEFI functions --- but it also
> doesn't have to worry about the UEFI code servicing an interrupt take
> taking control of the CPU away from the OS.
>

There's a 'however' here. The OS may call UEFI Runtime Services from
time to time (to interact with non-volatile variables), and it has to
carefully
arrange things to do this (it has to swap in a special memory map, etc). So
UEFI isn't completely gone away.. You just don't have to worry about it
fielding interrupts after you exit boot services.


> However, the OS can still execute various ACPI functions, assuming the
> OS has its own AML interpreter.  (Of course, the UEFI BIOS has its own
> AML interpreter --- and scheduler --- and everything else that a
> simple OS might have.)  Ron Minnich has a number of really good
> presentations about the "joys" of working with UEFI.  See the YouTube
> video, "Replace your Expoit-ridden Firmware with Linux"[1] and prepare to
> be horrified about what horrors lurks in the heart of UEFI....
>
> [1] https://www.youtube.com/watch?v=iffTJ1vPCSo
>
> So you can very much be *not* using UEFI, and still be using ACPI ---
> either if you are using a pre-2004 computer, or if you are using a
> more modern platform which uses LinuxBoot.
>

Indeed. I got to deal with all of that, and more. I have finished writing
LinuxBoot support for FreeBSD. The normal kexec-tools, u-root, etc aren't
sufficient for FreeBSD because FreeBSD's kernel expects the boot loader
to setup a number of meta-data items that go with the kernel that include
all the information about the system that the kernel simply can't get once
you've entered long mode...

Even with LinuxBoot, you are booting with UEFI, albeit with a much small
much smaller UEFI. The PEI is still there (the pre-efi initialization phase:
that runs the CPU vendor's binary blobs, and does other baisc bringup
of the machine). In addition, UEFI's runtime services remain. Linux's EFI
capsule replaces almost all fo the DEX drivers, though, which is a huge win
It's a stub that acts as the first EFI program that runs before UEFI gets
too
far along (MDS on x86, Shell,efi on all the others). It completely takes
over,
modulo the callbacks to Runtime Services.

Why am I on about Runtime Services so much? That was one of the pain
points of my port. ExitBootServices had been called, and SetVirtualMapping
had been as well by the time my boot loader got to run. So I had
to pass all that into FreeBSD's kernel (along with an interesting amount
of metadata to work around chip errata) and the FreeBSD kernel expected 1:1
PA:VA mapping, but the Linux kernel doesn't do that and if I want to run
there I have to cope. Thankfully, the kernel only needed one bug fix and
two asserts removed to work...

So even with LinuxBoot you still have a tiny bit of UEFI left behind...
Though
the huge ugly mess of it finding what to boot, executing another boot
loader,
and that loading the kernel is thankfully gone.

My BSDcan talk, my EuroBSDcon talk and a forthcoming article on
booting FreeBSD in a LinuxBoot environment are all on this.

https://2023.eurobsdcon.org/slides/eurobsdcon2023-warner_losh-kboot.pdf
are the slides, but the videos aren't up for some reason....  It was very
eye-opening doing this work...

Warner

[-- Attachment #2: Type: text/html, Size: 8240 bytes --]

[TUHS] Re: Question about BSD disklabel history

[Warner Losh]

[-- Attachment #1: Type: text/plain, Size: 287 bytes --]

On Tue, Jan 2, 2024 at 8:57 PM Warner Losh <imp@bsdimp.com> wrote:

> . It didn't try to retain any backwards
> compatibility with the original IBM BIOS.
>

I should have added... Later, UEFI also added CSM booting mode to boot
legacy mode that was a bit compatible.

Warner

[-- Attachment #2: Type: text/html, Size: 713 bytes --]

[TUHS] Re: Question about BSD disklabel history

[Theodore Ts'o]

On Tue, Jan 02, 2024 at 08:57:34PM -0700, Warner Losh wrote:
> 
> Indeed. I got to deal with all of that, and more. I have finished writing
> LinuxBoot support for FreeBSD. The normal kexec-tools, u-root, etc aren't
> sufficient for FreeBSD because FreeBSD's kernel expects the boot loader
> to setup a number of meta-data items that go with the kernel that include
> all the information about the system that the kernel simply can't get once
> you've entered long mode...
> 
> Even with LinuxBoot, you are booting with UEFI, albeit with a much small
> much smaller UEFI.

Yeah, one of the older names of LinuxBoot was NERF (Non-Extensible
Reduced Firmware).  I was confusing LinuxBoot with coreboot, which is
used on all ChromeOS devices after 2012, and which completely doesn't
use any magic binary blobs supplied by the mainbord vendor.  The
tradeoff is that coreboot only supports a very restricted set of
hardware, since it has to do all of the things that are "normally"
done by the vendor's binary blobs to initialize the hardware devices,
etc.  This only works if you have very tight control over hardware,
and you have enough influence that you can lean on the mainboard
vendors to allow the low-level programming details of their devices to
be released in open source code which that can be independently
verified and digitally signed by the OS vendor (such as Google in the
case of ChromeOS).

Many hyper-scale cloud companies will tend to use coreboot or related
software instead of UEFI.  A public/published example of this is
Facebook's Open Compute Project.

It *is* nice not to have to deal with UEFI at all, if you're lucky
enough to be able to use hardware where it's not necessary....  Of
course you won't be able to run Windows on those systems, but some
would consider that a feature.  :-)

					- Ted

[TUHS] Re: Question about BSD disklabel history

[Warner Losh]

[-- Attachment #1: Type: text/plain, Size: 3245 bytes --]

On Tue, Jan 2, 2024 at 9:30 PM Theodore Ts'o <tytso@mit.edu> wrote:

> On Tue, Jan 02, 2024 at 08:57:34PM -0700, Warner Losh wrote:
> >
> > Indeed. I got to deal with all of that, and more. I have finished writing
> > LinuxBoot support for FreeBSD. The normal kexec-tools, u-root, etc aren't
> > sufficient for FreeBSD because FreeBSD's kernel expects the boot loader
> > to setup a number of meta-data items that go with the kernel that include
> > all the information about the system that the kernel simply can't get
> once
> > you've entered long mode...
> >
> > Even with LinuxBoot, you are booting with UEFI, albeit with a much small
> > much smaller UEFI.
>
> Yeah, one of the older names of LinuxBoot was NERF (Non-Extensible
> Reduced Firmware).


I love the secondary meaning for nerf from gaming too:
"to reconfigure (an existing character or weapon), making it less powerful."


> I was confusing LinuxBoot with coreboot, which is
> used on all ChromeOS devices after 2012, and which completely doesn't
> use any magic binary blobs supplied by the mainbord vendor.  The
> tradeoff is that coreboot only supports a very restricted set of
> hardware, since it has to do all of the things that are "normally"
> done by the vendor's binary blobs to initialize the hardware devices,
> etc.  This only works if you have very tight control over hardware,
> and you have enough influence that you can lean on the mainboard
> vendors to allow the low-level programming details of their devices to
> be released in open source code which that can be independently
> verified and digitally signed by the OS vendor (such as Google in the
> case of ChromeOS).
>

I think that some of the open laptops might have a binary blob or two
buried deep in their coreboot implementation. But it is far more open
system than EDK2 is... Most of the open parts of EDK2 are what are nerfed
by LinuxBoot. The private hidden ones are all that remain, alas.

So if you are large enough, you can get all the sources. If not, then you
have to put up with the binary blob, or not run on some computers. :(.

There's also a rewrite of coreboot in Rust called 'oreboot' which is
coreboot
without the 'C', since they want it to be even safer.

Many hyper-scale cloud companies will tend to use coreboot or related
> software instead of UEFI.  A public/published example of this is
> Facebook's Open Compute Project.
>

Yes. It was a concerted effort of all these stakeholders that got the
Linux kernel so it could tolerate such an early handoff.


> It *is* nice not to have to deal with UEFI at all, if you're lucky
> enough to be able to use hardware where it's not necessary....  Of
> course you won't be able to run Windows on those systems, but some
> would consider that a feature.  :-)
>

Yea... There's several experimental attempts at booting windows on
at least the UEFI-remnant machines. And it's (a) too F'ing weird and
(b) to far afield to get into here...

https://www.osfc.io/2019/talks/booting-windows-on-linuxboot/

has all the horrors of one such attempt. I think this is the 'create an
exec handler in Linux that can run .efi programs and emulate UEFI'
version...

Warner

[-- Attachment #2: Type: text/html, Size: 4550 bytes --]

[TUHS] Re: Question about BSD disklabel history

[Dan Cross]

On Tue, Jan 2, 2024 at 10:33 PM Theodore Ts'o <tytso@mit.edu> wrote:
> On Tue, Jan 02, 2024 at 03:48:28PM -0500, Dan Cross wrote:
> > On Sun, Dec 31, 2023 at 6:25 PM Larry McVoy <lm@mcvoy.com> wrote:
> >
> > Ironically, the UEFI people have done something _similar_ to OF in the
> > form of AML (ACPI Machine Language), which is a byte-code
> > serialization ASL (ACPI Source Language); presumably that's system
> > independent. The idea of a p-code representation is about where the
> > similarity ends, though: AML exposes a mechanism to talk to the UEFI
> > OS for a whole slew of stuff, which is rather unlike what OF did
>
> [Ted's excellent history of ACPI/UEFI elided for brevity.]
>
> Normally, once the OS has gotten to a certain point in its OS
> initialization, the OS can "terminate UEFI services".  At that point,
> the OS can no longer call into the UEFI functions --- but it also
> doesn't have to worry about the UEFI code servicing an interrupt take
> taking control of the CPU away from the OS.

Warner corrected this: UEFI remains resident and callable.

> However, the OS can still execute various ACPI functions, assuming the
> OS has its own AML interpreter.  (Of course, the UEFI BIOS has its own
> AML interpreter --- and scheduler --- and everything else that a
> simple OS might have.)  Ron Minnich has a number of really good
> presentations about the "joys" of working with UEFI.  See the YouTube
> video, "Replace your Expoit-ridden Firmware with Linux"[1] and prepare to
> be horrified about what horrors lurks in the heart of UEFI....
>
> [1] https://www.youtube.com/watch?v=iffTJ1vPCSo

I'm well aware.  :-)  But I'll see your LinuxBoot and raise you this:
https://www.osfc.io/2022/talks/i-have-come-to-bury-the-bios-not-to-open-it-the-need-for-holistic-systems/

Of course, we at Oxide have the luxury of controlling the hardware in
addition to the software that runs [directly] on it, so we can get
away with doing our own thing. It's not so easy in the larger world,
where integrating components from disparate vendors demands _some_
kind of interface. It's just unfortunate that UEFI is the interface
that we got; see also Mothy Roscoe's OSDI'21 keynote about
rediscovering hardware.

> So you can very much be *not* using UEFI, and still be using ACPI ---
> either if you are using a pre-2004 computer, or if you are using a
> more modern platform which uses LinuxBoot.

Indeed; caveat the details that have already been mentioned about
coreboot/oreboot/holistic boot vs LinuxBoot (which replaces the UEFI
DXE layer). However, note that the ACPI spec has fallen under the
auspices of the UEFI forum for the last ten years or so, which is what
I meant (e.g., that one finds the ACPI spec on uefi.org). It's true,
though, that I was being unacceptably imprecise by not mentioning the
history at all and that ACPI came before UEFI.

        - Dan C.

[TUHS] Re: Question about BSD disklabel history

[Dan Cross]

On Wed, Jan 3, 2024 at 12:10 AM Warner Losh <imp@bsdimp.com> wrote:
> On Tue, Jan 2, 2024 at 9:30 PM Theodore Ts'o <tytso@mit.edu> wrote:
>> [snip]
>> I was confusing LinuxBoot with coreboot, which is
>> used on all ChromeOS devices after 2012, and which completely doesn't
>> use any magic binary blobs supplied by the mainbord vendor.  The
>> tradeoff is that coreboot only supports a very restricted set of
>> hardware, since it has to do all of the things that are "normally"
>> done by the vendor's binary blobs to initialize the hardware devices,
>> etc.  This only works if you have very tight control over hardware,
>> and you have enough influence that you can lean on the mainboard
>> vendors to allow the low-level programming details of their devices to
>> be released in open source code which that can be independently
>> verified and digitally signed by the OS vendor (such as Google in the
>> case of ChromeOS).
>
> I think that some of the open laptops might have a binary blob or two
> buried deep in their coreboot implementation. But it is far more open
> system than EDK2 is... Most of the open parts of EDK2 are what are nerfed
> by LinuxBoot. The private hidden ones are all that remain, alas.

Sadly the situation is even more complex than this.

Consider AMD's EPYC processors: before the x86 cores start up, the PSP
(Platform Security Processor) starts up and does a lot of
pre-pre-initialization: it does DRAM timing training, for instance.
It's also responsible for loading the x86 payload out of the local
flash and setting up the x86 environment so that when those cores come
out of reset, they're running whatever was loaded (for instance, they
can load %cs on the BSC so that it starts somewhere other than the
architecturally-defined segment right below 4GiB). While cool in some
ways ("I don't have to train DRAM? Score!") the PSP is embedded in the
SoC and the firmware is a signed blob you get from AMD. I know there's
an ARM Cortex-A5 in there, but don't know much more about it and even
if I did, I have no way to generate signed images for it. :-/

The point is, even if you've got a completely open stack running on
x86 from the reset vector, there's almost certainly something else
somewhere that's not open (yet).

> So if you are large enough, you can get all the sources. If not, then you
> have to put up with the binary blob, or not run on some computers. :(.

Yup. :-(

        - Dan C.

[TUHS] Re: Question about BSD disklabel history

[Theodore Ts'o]

On Wed, Jan 03, 2024 at 10:56:14AM -0500, Dan Cross wrote:
> Sadly the situation is even more complex than this.
> 
> Consider AMD's EPYC processors: before the x86 cores start up, the PSP
> (Platform Security Processor) starts up and does a lot of
> pre-pre-initialization: it does DRAM timing training, for instance.
> It's also responsible for loading the x86 payload out of the local
> flash and setting up the x86 environment so that when those cores come
> out of reset, they're running whatever was loaded (for instance, they
> can load %cs on the BSC so that it starts somewhere other than the
> architecturally-defined segment right below 4GiB). While cool in some
> ways ("I don't have to train DRAM? Score!") the PSP is embedded in the
> SoC and the firmware is a signed blob you get from AMD. I know there's
> an ARM Cortex-A5 in there, but don't know much more about it and even
> if I did, I have no way to generate signed images for it. :-/
> 
> The point is, even if you've got a completely open stack running on
> x86 from the reset vector, there's almost certainly something else
> somewhere that's not open (yet).

Or there's something running on a completely different x86 core with
unpatched securiy bugs in the Minix and Apache cores that you can't
even disable (unless you are the National Security Agency)....  Sadly,
Intel refuses to make it available the magic bits to disable the Intel
ME to anyone else.  :-(

     	     	     	     	 	  	   - Ted

[TUHS] Re: Question about BSD disklabel history

[Dan Cross]

On Wed, Jan 3, 2024 at 11:37 AM Theodore Ts'o <tytso@mit.edu> wrote:
> On Wed, Jan 03, 2024 at 10:56:14AM -0500, Dan Cross wrote:
> > Sadly the situation is even more complex than this.
> >
> > Consider AMD's EPYC processors: before the x86 cores start up, the PSP
> > (Platform Security Processor) starts up and does a lot of
> > pre-pre-initialization: it does DRAM timing training, for instance.
> > It's also responsible for loading the x86 payload out of the local
> > flash and setting up the x86 environment so that when those cores come
> > out of reset, they're running whatever was loaded (for instance, they
> > can load %cs on the BSC so that it starts somewhere other than the
> > architecturally-defined segment right below 4GiB). While cool in some
> > ways ("I don't have to train DRAM? Score!") the PSP is embedded in the
> > SoC and the firmware is a signed blob you get from AMD. I know there's
> > an ARM Cortex-A5 in there, but don't know much more about it and even
> > if I did, I have no way to generate signed images for it. :-/
> >
> > The point is, even if you've got a completely open stack running on
> > x86 from the reset vector, there's almost certainly something else
> > somewhere that's not open (yet).
>
> Or there's something running on a completely different x86 core with
> unpatched securiy bugs in the Minix and Apache cores that you can't
> even disable (unless you are the National Security Agency)....  Sadly,
> Intel refuses to make it available the magic bits to disable the Intel
> ME to anyone else.  :-(

...and let's not even get started on SMM. :-(

        - Dan C.

[TUHS] Re: Question about BSD disklabel history

[arnold]

Things have wandered a little far afield... :-)

"Theodore Ts'o" <tytso@mit.edu> wrote:

> Or there's something running on a completely different x86 core with
> unpatched securiy bugs in the Minix and Apache cores that you can't
> even disable (unless you are the National Security Agency)....  Sadly,
> Intel refuses to make it available the magic bits to disable the Intel
> ME to anyone else.  :-(

I worked for a number of years in the design center where the firmware
and software for the ME were develped.  Although it's possible that
the firmware developers were sworn to secrecy, I never heard anything
about back doors for the NSA or anyone else.

Intel took security and code quality in the ME very seriously,
and during my tenure the quality of the ME firmware improved a lot.

ISTR that the BIOS had settings for disabling the ME. Is that
no longer true?

I know there are lots of people who despise the ME, which I never
understood. It was designed to solve the very real problem of remote
PC management, and for that it works.  My own feeling is, if you don't
want the ME, buy a processor without it; there are plenty from Intel
and AMD.

Quite seriously, and with no animosity, I'd be happy to learn what
I'm missing here.

Thanks,

Arnold

[TUHS] Re: Question about BSD disklabel history

[Kevin Bowling]

On Thu, Jan 4, 2024 at 1:42 AM <arnold@skeeve.com> wrote:
>
> Things have wandered a little far afield... :-)
>
> "Theodore Ts'o" <tytso@mit.edu> wrote:
>
> > Or there's something running on a completely different x86 core with
> > unpatched securiy bugs in the Minix and Apache cores that you can't
> > even disable (unless you are the National Security Agency)....  Sadly,
> > Intel refuses to make it available the magic bits to disable the Intel
> > ME to anyone else.  :-(
>
> I worked for a number of years in the design center where the firmware
> and software for the ME were develped.  Although it's possible that
> the firmware developers were sworn to secrecy, I never heard anything
> about back doors for the NSA or anyone else.
>
> Intel took security and code quality in the ME very seriously,
> and during my tenure the quality of the ME firmware improved a lot.
>
> ISTR that the BIOS had settings for disabling the ME. Is that
> no longer true?
>
> I know there are lots of people who despise the ME, which I never
> understood. It was designed to solve the very real problem of remote
> PC management, and for that it works.  My own feeling is, if you don't
> want the ME, buy a processor without it; there are plenty from Intel
> and AMD.

I have tried out the AMT stuff for OS development and it is a mess.  I
am skeptical anyone seriously uses it.  Laptops already have
microcontrollers for various functionality so it is hard to see why
the already standardized NC-SI and IPMI couldn't be applied to the
problem space in some way that is secure, standardized, and doesn't
significantly change the BOM cost.

For whatever reason, intel makes it difficult to impossible to remove
the ME in later generations.  It seems more than accidental
incompetence since people have figured out how to force it into a
brain dead state (coreboot with me_cleaner).  It is doubly suspicious
that the US government has a killswitch for it that the commercial and
general public do not.

Which are the intel CPUs without the ME?  Just because a CPU doesn't
have vPro licensed doesn't mean the ME isn't there.

> Quite seriously, and with no animosity, I'd be happy to learn what
> I'm missing here.

https://en.wikipedia.org/wiki/Intel_Management_Engine has a good
enough survey and links to other soruces.

It's a complete mess on the NIC too, the firmware on e1000 NICs has
all sorts of issues and much of it is related to the insane errata and
complexity of transitioning to and fro Management mode and different
interpretations of who is responsible for power management.

>
> Thanks,
>
> Arnold

[TUHS] Re: Question about BSD disklabel history

[Norman Wilson]

Disk sections (I don't think anyone in Research called them
partitions--certainly the Research manuals didn't) were
originally defined in the device driver, not by data on the
disk.  In those days, system management included recompiling
stuff, including the OS kernel, and it was not unusual for
sites to edit hp.c or whatnot to adjust things to local
preference.

There was nothing magic about the mapping between device
names and minor device numbers either; the system came with
certain conventions on the original tape, but it was not
at all uncommon to change them.

By the time I arrived at the first Unix site I ever helped
run, in a physics group at Caltech, we already used a different
naming convention: a BSD-like ddNs, where dd was a driver
name, N the physical drive unit number, s a section letter.
I don't know whether that was borrowed from BSD (it must have
started during the 3BSD era, since I started there in mid-1980
and 4BSD appears to have been released late in that year).

Looking at my archival copy of that much-locally-hacked
source tree, I see that we later moved the definitions of
all the disk-section tables to a single file compiled at
system-configuration time (we used a USG-like scheme that
compiled most of the system into libraries, rather than
compiling every file separately for each target system a
la V7 and BSD).  That simplified handling our somewhat-
complicated disk topology: all but system disks were connected
through System Industries 9400 disk controllers, which were
a neat design (each controller could interface to as many as
four hosts and four disks) but in practice were not always
reliable.  On one hand, we arranged for one disk to be used
in parts by our main time-sharing VAX and a subsidiary PDP-11/45,
making the 11/45 cheaper to keep around; on the other, the
main VAX had two paths to each of its disks, through different
SI controllers, so when an SI controller conked out we could
run without it until the service guys fixed it.  (Each disk
was dual-ported, as was common in the SMD world, hence
connected to two controllers.)

Reliability took rather more work in those days.

A different data point: by the time I moved from California
to New Jersey and joined 1127, Research was also using a
different naming scheme for disk sections.  By then the
internal naming convention was e.g. ra17 for physical unit
1, section 7; by further convention 7 (the highest-numbered
section).  At some point a little later we added an ioctl
to set the starting block and size of a particular section
on a particular drive, but we never went to having the OS
itself try to find a label and trust its contents (something
that still makes the 1980s part of me feel a little creepy,
though 21st century me has come to terms with it).

Norman Wilson
Certified old fart
Toronto ON