[9fans] MBR messed up on installation

[9fans] MBR messed up on installation

[tlaronde]

Hello,

Just a note for the record (I think I will have the time tomorrow to
track down the problem and give explanations/patches) for the ones
installing Plan 9 on a i386 with a NetBSD partionned disk. 


When finishing the install, and setting (there is not the choice to "do
nothing") the active partition to Plan 9, apparently all the slots are
rewritten in the table, and the chs and xstart for previous alien
partitions are recomputed (modified), leading obviously to the
impossibility to launch NetBSD and to find its disklabel.

	At first look it seems that Plan 9 recomputes the beginning of the
	partition to match a cylinder border using BIOS values (number of
	sectors by cylinder) while NetBSD uses a disklabel, kernel provided
	value for the same task:
	
$ fdisk wd0

Disk: /dev/rwd0d
NetBSD disklabel disk geometry:
cylinders: 38792, heads: 16, sectors/track: 63 (1008 sectors/cylinder)
total sectors: 39102336

BIOS disk geometry:
cylinders: 1024, heads: 255, sectors/track: 63 (16065 sectors/cylinder)
total sectors: 39102336

Partition table:
0: Plan 9, or Theos spanned (sysid 57)
    bootmenu: Plan9
    start 63, size 2232972 (1090 MB, Cyls 0-139), Active
1: NetBSD (sysid 169)
    start 2241792, size 14329728 (6997 MB, Cyls 139/139/1-1031/135/1)
2: NetBSD (sysid 169)
    bootmenu: NetBSD
    start 16571520, size 8192016 (4000 MB, Cyls 1031/135/1-1541/117/1)
3: <UNUSED>
Bootselector enabled, timeout 10 seconds.

i.e. (for partition 1) 2241792 % 1008 == 0 

while Plan 9 gives 

xstart = 139 * 16065 = 2233035  for the same partition (p2)


I will verify this later (a priori, there is a non identical conversion

xstart -> chs
chs -> xstart

and probably the only solution is to take as is the values for the
partitions about which Plan 9 fdisk did not modify the _geometry_.
)

Note 2: could someone update the Wiki page about the installation to
note that the minimum of 300 Mbytes is, for the current distribution and
replica settings, for the sole fossil partition, and not for the whole 
chunk dedicated to Plan 9?

Cheers,
-- 
Thierry Laronde (Alceste) <tlaronde +AT+ polynum +dot+ com>
                 http://www.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] MBR messed up on installation

[erik quanstrom]

why is netbsd making up chs numbers?  it would seem to me that that is
the problem.  i'm not sure where the fdisk partition table standard is,
so i can't say definatively this is wrong.  but it seems like trouble, making
up one's on chs numbers.

- erik

> Disk: /dev/rwd0d
> NetBSD disklabel disk geometry:
> cylinders: 38792, heads: 16, sectors/track: 63 (1008 sectors/cylinder)
> total sectors: 39102336
> 
> BIOS disk geometry:
> cylinders: 1024, heads: 255, sectors/track: 63 (16065 sectors/cylinder)
> total sectors: 39102336

Re: [9fans] MBR messed up on installation

[Paweł Lasek]

On 5/24/07, erik quanstrom <quanstro@coraid.com> wrote:

> > Disk: /dev/rwd0d
> > NetBSD disklabel disk geometry:
> > cylinders: 38792, heads: 16, sectors/track: 63 (1008 sectors/cylinder)
> > total sectors: 39102336
> >
> > BIOS disk geometry:
> > cylinders: 1024, heads: 255, sectors/track: 63 (16065 sectors/cylinder)
> > total sectors: 39102336

> why is netbsd making up chs numbers?  it would seem to me that that is
> the problem.  i'm not sure where the fdisk partition table standard is,
> so i can't say definatively this is wrong.  but it seems like trouble, making
> up one's on chs numbers.



BIOS geometry seems a little off to me, although It is voodoo magick
after all... Especially with a ~20 GB drive. It looks like NetBSD
placed LBA-like values into it's disklabel, propably received by it's
own drivers, while MBR partition table is made up in ... weird way?
Maybe it has that weird geometry to allow booting from every partition
without using LBA, as BIOS had 1024 cylinders set.

Anyway I'd recommend staying away from Plan9 fdisk if you use other
OS, especially other than those from MS (And NT might take it badly
too). I'd recommend setting Plan 9 partition from BSD or GNU fdisk and
then use plan 9 to make it's own partition table in it.

-- 
Paul Lasek

[9fans] Re: [explanations] MBR messed up on installation

[tlaronde]

Hello,

The time for me to discover Plan 9 and to be able to dig/edit/compile
under Plan 9 for the patches here are, at least, explanations about the
process on i386 involving CHS addressing, the BIOS and the partition
table.

Position of the problem
1. After installing Plan 9 on an i386 with already NetBSD installed,
the partition table hold by the MBR was changed: the absolute start
(sector in LBA) for the NetBSD partition was changed, the CHS values too
(but irrelevant in part because out of reach of CHS addressing), leading
to the impossibility to boot in NetBSD.

The story

ATA CHS addressing scheme

2. The ATA standard specified a CHS based addressing scheme on 28
bits, allowing theorically 128 GiB of disk space (2^28 for sectors, with
512 bytes per sector).
The mapping was this:
C	H	S
16	4	8

3. The PC BIOS provides services. The INT 13H is dedicated to disk
access, and one of its function deals with CHS addressing.
The CHS addressing passed to the BIOS was supposed, in legacy PC, to be
passed as is to the ATA controler.
But, since it's the PC, the CHS addressing used by the BIOS is distinct
from the CHS addressing used by ATA...

In the BIOS:
C	H	S
10	8	6

that is only 24 bits.

Furthermore, to make things even more interesting, while cylinders and
heads are indexed starting at 0, sectors are indexed starting at 1:

C: 0..1023
H: 0..255
S: 1..63

To continue in the same mood, Microsoft made a fault in the initial
programmation, and didn't even allow the maximum of 256 heads, leading
to:

C: 0..1023
H: 0..254
S: 1..63

4. The main result, since the legacy BIOS passes the CHS values as is to
the ATA controler, is that only the minimal common denominator between
the two specification can be used:

C	H	S
10	4	6	

that is the limit: 1024 * 16 * 63 sectors hence the 504MiB.

5. Since the disk finally exceeded this 504MiB, a hack was designed. The
BIOS was modified to _not_ pass the specification as is to the ATA
controler, but to do a mapping between BIOS CHS addressing and ATA CHS
addressing, allowing the use of the full range (minus the
idiosyncrasies) of the 24 bits.

This is called ECHS, and this works even with legacy boot loaders using
INT 13H services, since this is the BIOS that does the job. This has
nothing to do with LBA, but this allows disk up to:

1024 * 254 * 63 sectors that is 7.8GiB.

The first problem is that the cylinder boundary in BIOS CHS addressing
may not match the hardware ATA CHS cylinder boundary. The "correct"
value is to align on a hardware boundary (if this makes sense).

Some fdisk(8) programs try to find a match between BIOS "cylinders" and
ATA (hardware) cylinders (that is what NetBSD seems to do):

------ fdisk on NetBSD
Disk: /dev/rwd0d
NetBSD disklabel disk geometry:
cylinders: 38792, heads: 16, sectors/track: 63 (1008 sectors/cylinder)
total sectors: 39102336

BIOS disk geometry:
cylinders: 1024, heads: 255, sectors/track: 63 (16065 sectors/cylinder)
total sectors: 39102336
------

The first specification is the hardware description (ATA).
The second is an ECHS description.

6. If the disk capacity exceeds the BIOS CHS addressing capabilities,
the only option is for the boot loader to use LBA services (and for the
BIOS to provide these services).

7. The only value that shall not be touched for a chunk (partition)
already in use in the absolute sector value (LBA) [and the problem is
that Plan 9 fdisk modifies this value].

The partition table.

8. Since the MBR is a "BIOS" program---by a signature, and because, due
to its size, it has to use BIOS services---the CHS values are already
computed in the partition table for a direct use by the BIOS services,
i.e. in the partition table one finds BIOS CHS values.

Here is the description in pseudo-code (pseudo because it is not
portable, assumes packing of fields and little endianness):

/* 3 bytes i.e. 24 bits */
struct Chs {
	unsigned int head: 8;
	unsigned int sector: 6;
	unsigned int cylh: 2; /* high two bits of cylinder */
	unsigned int cyll: 8; /* low eight bits of cylinder */
};

/* 16 bytes */
struc Part {
	uint8_t status;
	struct Chs Chs_start;
	uint8_t type;
	struct Chs Chs_end;
	uint32_t lba_start;
	uint32_t size_in_sectors;
};

9. In the MBR the partition table starts at offset 0x1BE (446), with 
4 Part entries (4 * 16 = 64) and the last two byte with the signature.

Hence, it is easy to see the details of the computation (here using
NetBSD tool hexdump(1) ---same can be done with xd(1)--- and awk(1)):

------awk script
{
	print "partition:", NR
	print "status:", ($1 == 128) ? "ACTIVE" : "NOT ACTIVE"
	print "start CHS:", int($3 / 64) * 256 + $4, $2, $3 % 64
	printf "type: %#x\n", $5 
	print "end CHS:", int($7 / 64) * 256 + $8, $6, $7 % 64
	print "start LBA:", $9 + $10 * 256 + $11 * 65536 + $12 * 16777216
	print "size in sectors:", $13 + $14 * 256 + $15 * 65536 + $16 * 16777216
	print ""
}
-----

Looking at the NetBSD computed mbr :

$ hexdump -s 0x1BE -n 64 -e '16/1 "%u " "\n"' /tmp/mbr | awk -f mbrtbl.awk

partition: 1
status: NOT ACTIVE
start CHS: 0 1 1
type: 0x39
end CHS: 138 254 63
start LBA: 63
size in sectors: 2232972

partition: 2
status: NOT ACTIVE
start CHS: 139 139 1
type: 0xa9
end CHS: 1023 134 63
start LBA: 2241792
size in sectors: 14329728

partition: 3
status: ACTIVE
start CHS: 1023 135 1
type: 0xa9
end CHS: 1023 116 63
start LBA: 16571520
size in sectors: 8192016

partition: 4
status: NOT ACTIVE
start CHS: 0 0 0
type: 0
end CHS: 0 0 0
start LBA: 0
size in sectors: 0

As can be seen the values are ECHS (more than 16 heads).
Have no sense when we cross the 7.8 GiB border.

TODO: track the Plan 9 sources to see the manipulation done.
Found where the hardware description is obtained (for the ATA CHS
values at least).
Problem of portability (this is i386 specific).
Why is fdisk(8) recomputing the absolute sector to put it in a BIOS
cylinder boundary (that has almost no sense). And why does it redefines
the starting sector of a partition it has neither created nor modified
(for the geometry). It seems that this is modified when setting the
ACTIVE flag, that is all the record is overwritten, including recomputed
CHS and LBA values.

To be continued.
-- 
Thierry Laronde (Alceste) <tlaronde +AT+ polynum +dot+ com>
                 http://www.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] Re: [explanations] MBR messed up on installation

[Paweł Lasek]

On 5/28/07, tlaronde@polynum.com <tlaronde@polynum.com> wrote:
> Hello,
[cut]
> TODO: track the Plan 9 sources to see the manipulation done.
> Found where the hardware description is obtained (for the ATA CHS
> values at least).
> Problem of portability (this is i386 specific).
> Why is fdisk(8) recomputing the absolute sector to put it in a BIOS
> cylinder boundary (that has almost no sense). And why does it redefines
> the starting sector of a partition it has neither created nor modified
> (for the geometry). It seems that this is modified when setting the
> ACTIVE flag, that is all the record is overwritten, including recomputed
> CHS and LBA values.

At the moment, fdisk also redefines partition numbers, rearranging
them as it finds them on the disk (So if you have first partition
number 3, then number 1, then number 2, it gets changed to 1-2-3).

It would be a fine thing if Plan 9's fdisk would rather follow the way
of GNU/BSD fdisk and _NOT_ touch any partition it didn't modify. While
the usage of plan 9 named partitions makes it irrevelant to p9, many
people have other OS on their hard drive which don't add another layer
(like disklabel or LVM) or aren't configured to do so.

Booting another OS just to run fdisk isn't a good thing. Pity that I
don't have access to a Plan 9 machine (or time) to prepare a patch for
this :-/

> To be continued.
> --
> Thierry Laronde (Alceste) <tlaronde +AT+ polynum +dot+ com>
>                  http://www.kergis.com/
> Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C
>

-- 
Paul Lasek

Re: [9fans] Re: [explanations] MBR messed up on installation

[tlaronde]

On Tue, May 29, 2007 at 01:31:48AM +0200, Pawe? Lasek wrote:
> 
> At the moment, fdisk also redefines partition numbers, rearranging
> them as it finds them on the disk (So if you have first partition
> number 3, then number 1, then number 2, it gets changed to 1-2-3).
> 
> It would be a fine thing if Plan 9's fdisk would rather follow the way
> of GNU/BSD fdisk and _NOT_ touch any partition it didn't modify. While
> the usage of plan 9 named partitions makes it irrevelant to p9, many
> people have other OS on their hard drive which don't add another layer
> (like disklabel or LVM) or aren't configured to do so.
> 
> Booting another OS just to run fdisk isn't a good thing. Pity that I
> don't have access to a Plan 9 machine (or time) to prepare a patch for
> this :-/

I will probably take the task.

Just, as usual, a matter of time: I have to tackle with business needs
just in order to have incomes ;)

But I will try to dedicate a slice of time in June for this.
-- 
Thierry Laronde (Alceste) <tlaronde +AT+ polynum +dot+ com>
                 http://www.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C