Behold, the power of 2!

Behold, the power of 2!

[Eric J Bowman]

[-- Attachment #1: Type: text/plain, Size: 1101 bytes --]

With few exceptions, 'mkisofs -pad' behaves the same way across tools in that it's an arbitrary, static value. The others dial the wrong number. Here's how I've partitioned my 1/2TB boot drive:



8.00 MiB

8.00 GiB

64.00 MiB

32.00 GiB

320.00 MiB

256.00 GiB

96.00 GiB

64.00 GiB

8.00 MiB



What those all have in common, is a byte count that's a multiple of 65536. If your partitioning yields x.00, x.25, x.50 or x.75, your sector alignment is 4K, 8K, or 16K (not in that order, just sayin'). This is important when creating a ZFS filesystem -- gotta account for the offset. On my setup, a root pool w/ 8K and a zone pool of 16K have the same offset as a storage pool with a 64K blocksize -- zero. What's the offset for 4K Windows/Linux partitions on the same drive? Zero.



Suggest new interop guideline in general -- partition in MiB/GiB using values which are a power of 2. Anyone got any drives laying around they need partitioned? lol, so easy now... if 'mkisofs -pad' behavior were to be "pad to multiple of 65536" would it cause anyone any harm?



-Eric

[-- Attachment #2: Type: text/html, Size: 1567 bytes --]

Re: [discuss] Behold, the power of 2!

[Udo Grabowski (IMK)]

[-- Attachment #1: Type: text/plain, Size: 611 bytes --]

On 10/06/2024 19:24, Eric J Bowman via illumos-discuss wrote:
> With few exceptions, 'mkisofs -pad' behaves the same way across tools in 
> that it's an arbitrary, static value. The others dial the wrong number. 
> ...
> Suggest new interop guideline in general -- partition in MiB/GiB using 
> values which are a power of 2. Anyone got any drives laying around they 
> need partitioned? lol, so easy now... if 'mkisofs -pad' behavior were to 
> be "pad to multiple of 65536" would it cause anyone any harm?

In xa1 mode (muiltisession CD), sectors are 2056 bytes, so that won't
fit into 65536 ...


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5804 bytes --]

Re: [discuss] Behold, the power of 2!

[Eric J Bowman]

[-- Attachment #1: Type: text/plain, Size: 1160 bytes --]

Indeed. But that's the fs inside the .iso, we're trying to fit the .iso into a partition UEFI requires to be at least 1 MiB (which should probably be changed to 2).



-Eric






---- On Mon, 10 Jun 2024 10:45:51 -0700 Udo Grabowski (IMK) <udo.grabowski@kit.edu> wrote ---



On 10/06/2024 19:24, Eric J Bowman via illumos-discuss wrote: 
> With few exceptions, 'mkisofs -pad' behaves the same way across tools in 
> that it's an arbitrary, static value. The others dial the wrong number. 
> ... 
> Suggest new interop guideline in general -- partition in MiB/GiB using 
> values which are a power of 2. Anyone got any drives laying around they 
> need partitioned? lol, so easy now... if 'mkisofs -pad' behavior were to 
> be "pad to multiple of 65536" would it cause anyone any harm? 
 
In xa1 mode (muiltisession CD), sectors are 2056 bytes, so that won't 
fit into 65536 ... 
 
 
------------------------------------------ 
illumos: illumos-discuss 
Permalink: https://illumos.topicbox.com/groups/discuss/Te6d81d754118b912-Mbc2cf1cf480d4f3cb435f197 
Delivery options: https://illumos.topicbox.com/groups/discuss/subscription

[-- Attachment #2: Type: text/html, Size: 2085 bytes --]

Re: [discuss] Behold, the power of 2!

[Eric J Bowman]

[-- Attachment #1: Type: text/plain, Size: 2720 bytes --]

CSM isn't compatible with DDR-6. UEFI's won't have CSM, soon. My only multi-session optical media that works with UEFI-CSM is BD-RE which was never bootable to begin with. If I had some other multi-session media format, I don't think it would work without CSM any better than distro.iso's which forgot -no-emul. Whole lotta Rock Ridge options won't be valid moving forward, due to incompatibility at the "alignment" level, like offsetting an overlapping partition containing the real distro.iso -- at best, gets you 16-bit real mode that only the Intel 8088 MINIX Experience doesn't care about -- due to NVRAM; it just loses access to system RAM w/o CSM.



UEFI says Rock Ridge. UEFI* lists some things which translate to, "won't work without CSM" while hewing to Joliet without saying so, but comes as no surprise because Joliet's Windows. I don't think Joliet's multi-session compatible, but I also note, UEFI allows booting a standard .iso in a partition with an MBR -- not that said MBR is required to point to anything actually existing, which defaults to pointing to -- Windows.


-Eric






---- On Mon, 10 Jun 2024 10:54:02 -0700 Eric J Bowman via illumos-discuss <discuss@lists.illumos.org> wrote ---



Indeed. But that's the fs inside the .iso, we're trying to fit the .iso into a partition UEFI requires to be at least 1 MiB (which should probably be changed to 2).



-Eric







---- On Mon, 10 Jun 2024 10:45:51 -0700 Udo Grabowski (IMK) <mailto:udo.grabowski@kit.edu> wrote ---









https://illumos.topicbox.com/latest / illumos-discuss / see https://illumos.topicbox.com/groups/discuss + https://illumos.topicbox.com/groups/discuss/members + https://illumos.topicbox.com/groups/discuss/subscription https://illumos.topicbox.com/groups/discuss/Te6d81d754118b912-M29fc6009e23320813eff56e3


On 10/06/2024 19:24, Eric J Bowman via illumos-discuss wrote: 
> With few exceptions, 'mkisofs -pad' behaves the same way across tools in 
> that it's an arbitrary, static value. The others dial the wrong number. 
> ... 
> Suggest new interop guideline in general -- partition in MiB/GiB using 
> values which are a power of 2. Anyone got any drives laying around they 
> need partitioned? lol, so easy now... if 'mkisofs -pad' behavior were to 
> be "pad to multiple of 65536" would it cause anyone any harm? 

In xa1 mode (muiltisession CD), sectors are 2056 bytes, so that won't 
fit into 65536 ... 


------------------------------------------ 
illumos: illumos-discuss 
Permalink: https://illumos.topicbox.com/groups/discuss/Te6d81d754118b912-Mbc2cf1cf480d4f3cb435f197
Delivery options: https://illumos.topicbox.com/groups/discuss/subscription

[-- Attachment #2: Type: text/html, Size: 4669 bytes --]

Re: [discuss] Behold, the power of 2!

[Gordon Ross]

I gather the topic here is partition alignment.  (If not, sorry, nevermind:)
My previous experience when looking into this for: fdisk, format, etc.
is that with "modern disks" (pretty much anything that survives today)
partitioning systems should simply use megabyte alignment, and
SHOULD NOT bother about cylinders etc.

On Mon, Jun 10, 2024 at 2:44 PM Eric J Bowman via illumos-discuss
<discuss@lists.illumos.org> wrote:
>
> CSM isn't compatible with DDR-6. UEFI's won't have CSM, soon. My only multi-session optical media that works with UEFI-CSM is BD-RE which was never bootable to begin with. If I had some other multi-session media format, I don't think it would work without CSM any better than distro.iso's which forgot -no-emul. Whole lotta Rock Ridge options won't be valid moving forward, due to incompatibility at the "alignment" level, like offsetting an overlapping partition containing the real distro.iso -- at best, gets you 16-bit real mode that only the Intel 8088 MINIX Experience doesn't care about -- due to NVRAM; it just loses access to system RAM w/o CSM.
>
> UEFI says Rock Ridge. UEFI* lists some things which translate to, "won't work without CSM" while hewing to Joliet without saying so, but comes as no surprise because Joliet's Windows. I don't think Joliet's multi-session compatible, but I also note, UEFI allows booting a standard .iso in a partition with an MBR -- not that said MBR is required to point to anything actually existing, which defaults to pointing to -- Windows.
>
> -Eric
>
>
>
> ---- On Mon, 10 Jun 2024 10:54:02 -0700 Eric J Bowman via illumos-discuss <discuss@lists.illumos.org> wrote ---
>
> Indeed. But that's the fs inside the .iso, we're trying to fit the .iso into a partition UEFI requires to be at least 1 MiB (which should probably be changed to 2).
>
> -Eric
>
>
>
> ---- On Mon, 10 Jun 2024 10:45:51 -0700 Udo Grabowski (IMK) <udo.grabowski@kit.edu> wrote ---
>
>
>
> On 10/06/2024 19:24, Eric J Bowman via illumos-discuss wrote:
> > With few exceptions, 'mkisofs -pad' behaves the same way across tools in
> > that it's an arbitrary, static value. The others dial the wrong number.
> > ...
> > Suggest new interop guideline in general -- partition in MiB/GiB using
> > values which are a power of 2. Anyone got any drives laying around they
> > need partitioned? lol, so easy now... if 'mkisofs -pad' behavior were to
> > be "pad to multiple of 65536" would it cause anyone any harm?
>
> In xa1 mode (muiltisession CD), sectors are 2056 bytes, so that won't
> fit into 65536 ...
>
>
> ------------------------------------------
> illumos: illumos-discuss
> Permalink: https://illumos.topicbox.com/groups/discuss/Te6d81d754118b912-Mbc2cf1cf480d4f3cb435f197
> Delivery options: https://illumos.topicbox.com/groups/discuss/subscription
>
>
> illumos / illumos-discuss / see discussions + participants + delivery options Permalink

Re: [discuss] Behold, the power of 2!

[Gordon Ross]

Oh, and I almost forgot I opened an issue for this:
https://www.illumos.org/issues/2949

On Mon, Jun 10, 2024 at 3:49 PM Gordon Ross <gordon.w.ross@gmail.com> wrote:
>
> I gather the topic here is partition alignment.  (If not, sorry, nevermind:)
> My previous experience when looking into this for: fdisk, format, etc.
> is that with "modern disks" (pretty much anything that survives today)
> partitioning systems should simply use megabyte alignment, and
> SHOULD NOT bother about cylinders etc.
>
> On Mon, Jun 10, 2024 at 2:44 PM Eric J Bowman via illumos-discuss
> <discuss@lists.illumos.org> wrote:
> >
> > CSM isn't compatible with DDR-6. UEFI's won't have CSM, soon. My only multi-session optical media that works with UEFI-CSM is BD-RE which was never bootable to begin with. If I had some other multi-session media format, I don't think it would work without CSM any better than distro.iso's which forgot -no-emul. Whole lotta Rock Ridge options won't be valid moving forward, due to incompatibility at the "alignment" level, like offsetting an overlapping partition containing the real distro.iso -- at best, gets you 16-bit real mode that only the Intel 8088 MINIX Experience doesn't care about -- due to NVRAM; it just loses access to system RAM w/o CSM.
> >
> > UEFI says Rock Ridge. UEFI* lists some things which translate to, "won't work without CSM" while hewing to Joliet without saying so, but comes as no surprise because Joliet's Windows. I don't think Joliet's multi-session compatible, but I also note, UEFI allows booting a standard .iso in a partition with an MBR -- not that said MBR is required to point to anything actually existing, which defaults to pointing to -- Windows.
> >
> > -Eric
> >
> >
> >
> > ---- On Mon, 10 Jun 2024 10:54:02 -0700 Eric J Bowman via illumos-discuss <discuss@lists.illumos.org> wrote ---
> >
> > Indeed. But that's the fs inside the .iso, we're trying to fit the .iso into a partition UEFI requires to be at least 1 MiB (which should probably be changed to 2).
> >
> > -Eric
> >
> >
> >
> > ---- On Mon, 10 Jun 2024 10:45:51 -0700 Udo Grabowski (IMK) <udo.grabowski@kit.edu> wrote ---
> >
> >
> >
> > On 10/06/2024 19:24, Eric J Bowman via illumos-discuss wrote:
> > > With few exceptions, 'mkisofs -pad' behaves the same way across tools in
> > > that it's an arbitrary, static value. The others dial the wrong number.
> > > ...
> > > Suggest new interop guideline in general -- partition in MiB/GiB using
> > > values which are a power of 2. Anyone got any drives laying around they
> > > need partitioned? lol, so easy now... if 'mkisofs -pad' behavior were to
> > > be "pad to multiple of 65536" would it cause anyone any harm?
> >
> > In xa1 mode (muiltisession CD), sectors are 2056 bytes, so that won't
> > fit into 65536 ...
> >
> >
> > ------------------------------------------
> > illumos: illumos-discuss
> > Permalink: https://illumos.topicbox.com/groups/discuss/Te6d81d754118b912-Mbc2cf1cf480d4f3cb435f197
> > Delivery options: https://illumos.topicbox.com/groups/discuss/subscription
> >
> >
> > illumos / illumos-discuss / see discussions + participants + delivery options Permalink

Re: [discuss] Behold, the power of 2!

[Eric J Bowman]

[-- Attachment #1: Type: text/plain, Size: 3638 bytes --]

Whether we're talking about multiboot, or distro.iso, or NVMe vs. SATA NGFF, or HDD vs. SSD, the over-arching problem is fitting a round peg in a square hole such that the edges of the square are tangents of the circumference of the circle -- in "sector alignment". Illumos uses GPT-approved PMBR partitioning, i.e. "SOLARIS2" for compatibility; format> partition asks for cylinders. Using EFI partitioning still results in PMBR, but it's easy to work around; in which case format> partition asks for sectors or MiB/GiB.



-Eric







---- On Mon, 10 Jun 2024 12:49:59 -0700 Gordon Ross <gordon.w.ross@gmail.com> wrote ---



I gather the topic here is partition alignment.  (If not, sorry, nevermind:) 
My previous experience when looking into this for: fdisk, format, etc. 
is that with "modern disks" (pretty much anything that survives today) 
partitioning systems should simply use megabyte alignment, and 
SHOULD NOT bother about cylinders etc. 
 
On Mon, Jun 10, 2024 at 2:44 PM Eric J Bowman via illumos-discuss 
<mailto:discuss@lists.illumos.org> wrote: 
> 
> CSM isn't compatible with DDR-6. UEFI's won't have CSM, soon. My only multi-session optical media that works with UEFI-CSM is BD-RE which was never bootable to begin with. If I had some other multi-session media format, I don't think it would work without CSM any better than distro.iso's which forgot -no-emul. Whole lotta Rock Ridge options won't be valid moving forward, due to incompatibility at the "alignment" level, like offsetting an overlapping partition containing the real distro.iso -- at best, gets you 16-bit real mode that only the Intel 8088 MINIX Experience doesn't care about -- due to NVRAM; it just loses access to system RAM w/o CSM. 
> 
> UEFI says Rock Ridge. UEFI* lists some things which translate to, "won't work without CSM" while hewing to Joliet without saying so, but comes as no surprise because Joliet's Windows. I don't think Joliet's multi-session compatible, but I also note, UEFI allows booting a standard .iso in a partition with an MBR -- not that said MBR is required to point to anything actually existing, which defaults to pointing to -- Windows. 
> 
> -Eric 
> 
> 
> 
> ---- On Mon, 10 Jun 2024 10:54:02 -0700 Eric J Bowman via illumos-discuss <mailto:discuss@lists.illumos.org> wrote --- 
> 
> Indeed. But that's the fs inside the .iso, we're trying to fit the .iso into a partition UEFI requires to be at least 1 MiB (which should probably be changed to 2). 
> 
> -Eric 
> 
> 
> 
> ---- On Mon, 10 Jun 2024 10:45:51 -0700 Udo Grabowski (IMK) <mailto:udo.grabowski@kit.edu> wrote --- 
> 
> 
> 
> On 10/06/2024 19:24, Eric J Bowman via illumos-discuss wrote: 
> > With few exceptions, 'mkisofs -pad' behaves the same way across tools in 
> > that it's an arbitrary, static value. The others dial the wrong number. 
> > ... 
> > Suggest new interop guideline in general -- partition in MiB/GiB using 
> > values which are a power of 2. Anyone got any drives laying around they 
> > need partitioned? lol, so easy now... if 'mkisofs -pad' behavior were to 
> > be "pad to multiple of 65536" would it cause anyone any harm? 
> 
> In xa1 mode (muiltisession CD), sectors are 2056 bytes, so that won't 
> fit into 65536 ... 
> 
> illumos / illumos-discuss / see discussions + participants + delivery options Permalink 
 
------------------------------------------ 
illumos: illumos-discuss 
Permalink: https://illumos.topicbox.com/groups/discuss/Te6d81d754118b912-Mfca249b30c9df2563e52f29e 
Delivery options: https://illumos.topicbox.com/groups/discuss/subscription

[-- Attachment #2: Type: text/html, Size: 4915 bytes --]

Re: [discuss] Behold, the power of 2!

[Eric J Bowman]

[-- Attachment #1: Type: text/plain, Size: 533 bytes --]

I had the wrong analogy before, when I was drawing my circle around the square such that the vertices met the circumfrence, and that the "alignment spokes" were corner-to-corner. But no, the circle needs to go in the square, and the spokes need to be perpindicular to the edges of the square.



This leaves us with not only squares of different sizes, but different rotations i.e. offsets. What 64K sector alignment does, is lines up the squares such that there's no offset, i.e. everyone's spokes are in alignment.



-Eric

[-- Attachment #2: Type: text/html, Size: 946 bytes --]

Re: [discuss] Behold, the power of 2!

[Eric J Bowman]

[-- Attachment #1: Type: text/plain, Size: 1646 bytes --]

Circling back around to smartos.img having the same i/o error, but not attributable to 16-bit math, tells me it means "VTOC's maths don't add up." IOW, "sector misalignment." Apparently they didn't get Peter's memo on installing ZFS rpool to a slice. ;-) G'head, burn it, boot it, tell me about s9, bork it, won't reboot... now, boot any other illumos including smartos, and point "format -e" at it, now:



format> q

# reboot -p



See, it's already fixed! Except you have to move s9 manually, either before or after rebooting... unless you pointed parted at it and answered "yes" to "fix" in which case it's fixed and s9 relocates.



What I was thinkin' before about those hogs, was wrong. Has nothing to do with the template you use when partitioning. What causes the desired behavior is, when you "pack your bags" so to speak, you set s9 to 0xFFFF's (not that I've figured out how as yet). This = 2TB, or wherever the new "end of drive" is. With my partitioning on a 512GB SATA, I was able to clone to a 500GiB NVMe with no room to spare, because somehow it "just worked" while it doesn't budge when I clone to a larger drive.



What isn't UEFI compatible, is PMBR-SMI. Only PMBR-GPT is UEFI approved, and if you zero out s0 and s2 you're fully interoperable so long as you're not using UFS, because ZFS puts "Solaris Backup" inside the pool. The reason the Wizard was confused between PMBR-SMI/PMBR-EFI, was simply not knowing what to make of 16-bit math. So, PMBR is something you don't need at all, if your firmware boots partitions instead of devices -- the partitions are GUID "either way you slice it."



-Eric

[-- Attachment #2: Type: text/html, Size: 2061 bytes --]

Re: [discuss] Behold, the power of 2!

[Eric J Bowman]

[-- Attachment #1: Type: text/plain, Size: 2416 bytes --]

Goes for RAMdisk, too. I got a new little computer this week; seeing as how all my UEFI boxes are Intel, I went AMD. Diskless, using the NIC to access CAINE on netboot.xyz, fails initramfs:



losetup: /cdrom/casper/root.squashfs: Warning: file does not fit into a 512-byte sector; the end of the file will be ignored.



New DDR5 mobo, no CSM. YMMV by firmware, but this looks to me like an endian issue. What doesn't fit, is one byte of extra padding, which breaks the checksum. Searching the error text yields much discussion about offsets (of physical storage media not RAMdisk). 64K sector alignment, of the .img file, solved the problem when I re-created the issue with an .img on my router. OS doesn't matter.



A couple other distros exhibit the same behavior, in that they work on Intel firmware but not AMD, Insyde vs. AMI UEFI.


-Eric







---- On Mon, 10 Jun 2024 10:24:19 -0700 Eric J Bowman via illumos-discuss <discuss@lists.illumos.org> wrote ---



With few exceptions, 'mkisofs -pad' behaves the same way across tools in that it's an arbitrary, static value. The others dial the wrong number. Here's how I've partitioned my 1/2TB boot drive:



8.00 MiB

8.00 GiB

64.00 MiB

32.00 GiB

320.00 MiB

256.00 GiB

96.00 GiB

64.00 GiB

8.00 MiB



What those all have in common, is a byte count that's a multiple of 65536. If your partitioning yields x.00, x.25, x.50 or x.75, your sector alignment is 4K, 8K, or 16K (not in that order, just sayin'). This is important when creating a ZFS filesystem -- gotta account for the offset. On my setup, a root pool w/ 8K and a zone pool of 16K have the same offset as a storage pool with a 64K blocksize -- zero. What's the offset for 4K Windows/Linux partitions on the same drive? Zero.



Suggest new interop guideline in general -- partition in MiB/GiB using values which are a power of 2. Anyone got any drives laying around they need partitioned? lol, so easy now... if 'mkisofs -pad' behavior were to be "pad to multiple of 65536" would it cause anyone any harm?



-Eric






https://illumos.topicbox.com/latest / illumos-discuss / see https://illumos.topicbox.com/groups/discuss + https://illumos.topicbox.com/groups/discuss/members + https://illumos.topicbox.com/groups/discuss/subscription https://illumos.topicbox.com/groups/discuss/Te6d81d754118b912-M772bc4944a71b200edc4b174

[-- Attachment #2: Type: text/html, Size: 3857 bytes --]

Re: [discuss] Behold, the power of 2!

[Eric J Bowman]

[-- Attachment #1: Type: text/plain, Size: 2738 bytes --]

On x86 UEFI systems (CMS or not), proper recordsize for ZFS RAMdisk is 4096. Because you're aligning to page size, not sector, and 'auto' isn't accounting for RAM.



-Eric






---- On Sat, 15 Jun 2024 01:12:52 -0700 Eric J Bowman via illumos-discuss <discuss@lists.illumos.org> wrote ---



Goes for RAMdisk, too. I got a new little computer this week; seeing as how all my UEFI boxes are Intel, I went AMD. Diskless, using the NIC to access CAINE on netboot.xyz, fails initramfs:



losetup: /cdrom/casper/root.squashfs: Warning: file does not fit into a 512-byte sector; the end of the file will be ignored.



New DDR5 mobo, no CSM. YMMV by firmware, but this looks to me like an endian issue. What doesn't fit, is one byte of extra padding, which breaks the checksum. Searching the error text yields much discussion about offsets (of physical storage media not RAMdisk). 64K sector alignment, of the .img file, solved the problem when I re-created the issue with an .img on my router. OS doesn't matter.



A couple other distros exhibit the same behavior, in that they work on Intel firmware but not AMD, Insyde vs. AMI UEFI.



-Eric







---- On Mon, 10 Jun 2024 10:24:19 -0700 Eric J Bowman via illumos-discuss <mailto:discuss@lists.illumos.org> wrote ---









https://illumos.topicbox.com/latest / illumos-discuss / see https://illumos.topicbox.com/groups/discuss + https://illumos.topicbox.com/groups/discuss/members + https://illumos.topicbox.com/groups/discuss/subscription https://illumos.topicbox.com/groups/discuss/Te6d81d754118b912-M7f76120a4f6ff8dbacf6f388


With few exceptions, 'mkisofs -pad' behaves the same way across tools in that it's an arbitrary, static value. The others dial the wrong number. Here's how I've partitioned my 1/2TB boot drive:



8.00 MiB

8.00 GiB

64.00 MiB

32.00 GiB

320.00 MiB

256.00 GiB

96.00 GiB

64.00 GiB

8.00 MiB



What those all have in common, is a byte count that's a multiple of 65536. If your partitioning yields x.00, x.25, x.50 or x.75, your sector alignment is 4K, 8K, or 16K (not in that order, just sayin'). This is important when creating a ZFS filesystem -- gotta account for the offset. On my setup, a root pool w/ 8K and a zone pool of 16K have the same offset as a storage pool with a 64K blocksize -- zero. What's the offset for 4K Windows/Linux partitions on the same drive? Zero.



Suggest new interop guideline in general -- partition in MiB/GiB using values which are a power of 2. Anyone got any drives laying around they need partitioned? lol, so easy now... if 'mkisofs -pad' behavior were to be "pad to multiple of 65536" would it cause anyone any harm?



-Eric

[-- Attachment #2: Type: text/html, Size: 4783 bytes --]