Kernel and rootfs tinkering

Kernel and rootfs tinkering

[678yym]

[-- Attachment #1: Type: text/plain, Size: 433 bytes --]

Hello list,

I'm new to illumos, but I have a bit of experience with Linux and the *BSDs.

I was wondering if it was possible to embed a rootfs image into the kernel, as a binary blob, and either mount as a RO fs, or mmap() it from somewhere in the kernel, and use it as a tmpfs.

I was thinking the second option would be better.

Is this possible? Where in the source would I need to make the modifications?

Thanks list.

[-- Attachment #2: Type: text/html, Size: 615 bytes --]

Re: [discuss] Kernel and rootfs tinkering

[Peter Tribble]

[-- Attachment #1: Type: text/plain, Size: 1491 bytes --]

On Fri, Jul 26, 2024 at 12:28 PM <678yym@gmail.com> wrote:

> Hello list,
>
> I'm new to illumos, but I have a bit of experience with Linux and the
> *BSDs.
>
> I was wondering if it was possible to embed a rootfs image into the
> kernel, as a binary blob, and either mount as a RO fs, or mmap() it from
> somewhere in the kernel, and use it as a tmpfs.
>

First, an observation: illumos has a modular kernel. Therefore, in order
for it to access a file system
(the rootfs image in this case) it needs to load the file system driver
modules. which it does by reading
the filesystem... Normally, it has external help, specifically from the
bootloader.

Which leads to the second thing, which is a question: what are you trying
to do here? It sounds like
an interesting experiment, but I'm not sure what the objective is.


> I was thinking the second option would be better.
>
> Is this possible? Where in the source would I need to make the
> modifications?
>
> Thanks list.
> *illumos <https://illumos.topicbox.com/latest>* / illumos-discuss / see
> discussions <https://illumos.topicbox.com/groups/discuss> + participants
> <https://illumos.topicbox.com/groups/discuss/members> + delivery options
> <https://illumos.topicbox.com/groups/discuss/subscription> Permalink
> <https://illumos.topicbox.com/groups/discuss/Tc9bfa679c7294ee7-M584c7f98cc4045f5d7aa6799>
>


-- 
-Peter Tribble
http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/

[-- Attachment #2: Type: text/html, Size: 2712 bytes --]

Re: [discuss] Kernel and rootfs tinkering

[Bryce]

[-- Attachment #1: Type: text/plain, Size: 2261 bytes --]

Okay, I didn't realize that. So the bootloader loads the modules into
memory and informs the kernel where the modules are?

If that's the case, then would it be possible to hijack the bootloader to
load a custom module which would just mount a tmpfs? (Can a module mount a
filesystem by itself, without intervention from the main kernel?)

As for why, I've been burned too many times by file system corruption from
sudden power loss etc (I know ZFS is super good, but Im pretty sick of fs
issues).

If there are any detailed man pages on the boot process I'd love to read
them, ive already read the boot(8) man page though.

On Fri, Jul 26, 2024, 9:01 AM Peter Tribble <peter.tribble@gmail.com> wrote:

> On Fri, Jul 26, 2024 at 12:28 PM <678yym@gmail.com> wrote:
>
>> Hello list,
>>
>> I'm new to illumos, but I have a bit of experience with Linux and the
>> *BSDs.
>>
>> I was wondering if it was possible to embed a rootfs image into the
>> kernel, as a binary blob, and either mount as a RO fs, or mmap() it from
>> somewhere in the kernel, and use it as a tmpfs.
>>
>
> First, an observation: illumos has a modular kernel. Therefore, in order
> for it to access a file system
> (the rootfs image in this case) it needs to load the file system driver
> modules. which it does by reading
> the filesystem... Normally, it has external help, specifically from the
> bootloader.
>
> Which leads to the second thing, which is a question: what are you trying
> to do here? It sounds like
> an interesting experiment, but I'm not sure what the objective is.
>
>
>> I was thinking the second option would be better.
>>
>> Is this possible? Where in the source would I need to make the
>> modifications?
>>
>> Thanks list.
>>
>
>
> --
> -Peter Tribble
> http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/
> *illumos <https://illumos.topicbox.com/latest>* / illumos-discuss / see
> discussions <https://illumos.topicbox.com/groups/discuss> + participants
> <https://illumos.topicbox.com/groups/discuss/members> + delivery options
> <https://illumos.topicbox.com/groups/discuss/subscription> Permalink
> <https://illumos.topicbox.com/groups/discuss/Tc9bfa679c7294ee7-Me0ac74d996d2040cc8947ece>
>

[-- Attachment #2: Type: text/html, Size: 3884 bytes --]

Re: [discuss] Kernel and rootfs tinkering

[Toomas Soome]

> On 26. Jul 2024, at 14:28, 678yym@gmail.com wrote:
> 
> Hello list,
> 
> I'm new to illumos, but I have a bit of experience with Linux and the *BSDs.
> 
> I was wondering if it was possible to embed a rootfs image into the kernel, as a binary blob, and either mount as a RO fs, or mmap() it from somewhere in the kernel, and use it as a tmpfs.
> 
> I was thinking the second option would be better.
> 
> Is this possible? Where in the source would I need to make the modifications?
> 

Our current setup is loading kernel and boot_archive, which is initial rootfs in memory — depending on file system type, you can end up presenting it to userland or switching to some other file system - one can use ufs to build boot_archive and then use it as rootfs (this is scenario smartos is using and how the live boot/install images are working).

rgds,
toomas

Re: [discuss] Kernel and rootfs tinkering

[Bryce]

[-- Attachment #1: Type: text/plain, Size: 1877 bytes --]

Thanks toomas

I tried some googling but I couldn't find out what's usually in
`boot_archive', nor could I find anything in illumos-gate relating to a
default, nor in oi-userland. Is boot_archive generated per distro or is it
in illumos-gate?

I read the bootadm(8) man page hoping for some answers but none came.
Although it did mention the available archive formats, does that mean that
boot_archive is archived into an (eg cpio) archive, and then mounted RO by
the kernel so it can load the modules necessary to talk to the hardware?

sorry for the silly questions.

On Fri, Jul 26, 2024, 9:36 AM Toomas Soome via illumos-discuss <
discuss@lists.illumos.org> wrote:

>
>
> > On 26. Jul 2024, at 14:28, 678yym@gmail.com wrote:
> >
> > Hello list,
> >
> > I'm new to illumos, but I have a bit of experience with Linux and the
> *BSDs.
> >
> > I was wondering if it was possible to embed a rootfs image into the
> kernel, as a binary blob, and either mount as a RO fs, or mmap() it from
> somewhere in the kernel, and use it as a tmpfs.
> >
> > I was thinking the second option would be better.
> >
> > Is this possible? Where in the source would I need to make the
> modifications?
> >
>
> Our current setup is loading kernel and boot_archive, which is initial
> rootfs in memory — depending on file system type, you can end up presenting
> it to userland or switching to some other file system - one can use ufs to
> build boot_archive and then use it as rootfs (this is scenario smartos is
> using and how the live boot/install images are working).
>
> rgds,
> toomas
> ------------------------------------------
> illumos: illumos-discuss
> Permalink:
> https://illumos.topicbox.com/groups/discuss/Tc9bfa679c7294ee7-M9beb4930631f6fb9c52e7725
> Delivery options: https://illumos.topicbox.com/groups/discuss/subscription
>

[-- Attachment #2: Type: text/html, Size: 2742 bytes --]

Re: [discuss] Kernel and rootfs tinkering

[Udo Grabowski (IMK)]

[-- Attachment #1: Type: text/plain, Size: 1561 bytes --]

On 26/07/2024 15:32, Bryce wrote:
> ...
> As for why, I've been burned too many times by file system corruption from
> sudden power loss etc (I know ZFS is super good, but Im pretty sick of fs issues).

NO OS will help you with these issues, since they are usually generated
by disk hardware without powerloss protection (which applies to almost
all consumer disks and (shame!) even to a significant amount of
server disks). Since disks caches are large, non-negligible portions
of data in transfer go to Nirwana in case of a powerloss without protection.

The resilience of ZFS here is, that, even when no full atomic write
could be finished due to a powerloss, or cached atmic writes were lost,
ZFS can rewind a couple of write groups to an earlier consistent state,
so you will loose some data (which is usually no problem on an rpool since
that's mostly log and state data, and the loss covers only a span of a few
seconds), but the filesystem is always consistent, so ZFS can cope with lousy
hardware pretty well.

But in a datacenter, the few seconds lost usually correspond to a
pretty large amount of data lost, and in that case, you never want
cache loss and invest in proper enterprise hardware.

Hope that convinces you to drop some more coins on proper hardware.
-- 
Dr.Udo Grabowski  Inst.of Meteorology & Climate Research IMK-ASF-SAT
https://www.imk-asf.kit.edu/english/sat.php
KIT - Karlsruhe Institute of Technology          https://www.kit.edu
Postfach 3640,76021 Karlsruhe,Germany T:(+49)721 608-26026 F:-926026


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5804 bytes --]

Re: [discuss] Kernel and rootfs tinkering

[Bryce]

[-- Attachment #1: Type: text/plain, Size: 1760 bytes --]

Thanks Peter, I just found your project MVI, it's exactly what i was
looking for. I'm quite new here, how should I close this thread?

On Fri, Jul 26, 2024, 9:01 AM Peter Tribble <peter.tribble@gmail.com> wrote:

> On Fri, Jul 26, 2024 at 12:28 PM <678yym@gmail.com> wrote:
>
>> Hello list,
>>
>> I'm new to illumos, but I have a bit of experience with Linux and the
>> *BSDs.
>>
>> I was wondering if it was possible to embed a rootfs image into the
>> kernel, as a binary blob, and either mount as a RO fs, or mmap() it from
>> somewhere in the kernel, and use it as a tmpfs.
>>
>
> First, an observation: illumos has a modular kernel. Therefore, in order
> for it to access a file system
> (the rootfs image in this case) it needs to load the file system driver
> modules. which it does by reading
> the filesystem... Normally, it has external help, specifically from the
> bootloader.
>
> Which leads to the second thing, which is a question: what are you trying
> to do here? It sounds like
> an interesting experiment, but I'm not sure what the objective is.
>
>
>> I was thinking the second option would be better.
>>
>> Is this possible? Where in the source would I need to make the
>> modifications?
>>
>> Thanks list.
>>
>
>
> --
> -Peter Tribble
> http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/
> *illumos <https://illumos.topicbox.com/latest>* / illumos-discuss / see
> discussions <https://illumos.topicbox.com/groups/discuss> + participants
> <https://illumos.topicbox.com/groups/discuss/members> + delivery options
> <https://illumos.topicbox.com/groups/discuss/subscription> Permalink
> <https://illumos.topicbox.com/groups/discuss/Tc9bfa679c7294ee7-Me0ac74d996d2040cc8947ece>
>

[-- Attachment #2: Type: text/html, Size: 3338 bytes --]

Re: [discuss] Kernel and rootfs tinkering

[Bill Sommerfeld]

On 7/26/24 07:14, Bryce wrote:
> I tried some googling but I couldn't find out what's usually in 
> `boot_archive', nor could I find anything in illumos-gate relating to a 
> default, nor in oi-userland. Is boot_archive generated per distro or is 
> it in illumos-gate?

The boot archive is generated by 'bootadm update-archive', which can be 
run manually by the administrator but it generally gets run 
automatically when needed at reboot time and by the 
svc:/system/boot-archive-update SMF service (which runs at boot once the 
relevant filesystems are mounted read-write).

Under the covers, 'bootadm update-archive' runs the 
/boot/solaris/bin/extract_boot_filelist script to enumerate the files 
and directories from the real root that should be included in the 
archive.   Note that if a directory is named, the entire tree rooted at 
that directory is included.

As others have mentioned, SmartOS does things somewhat differently but 
the above applies to most general-purpose illumos distributions.

> I read the bootadm(8) man page hoping for some answers but none came. 
> Although it did mention the available archive formats, does that mean 
> that boot_archive is archived into an (eg cpio) archive, and then 
> mounted RO by the kernel so it can load the modules necessary to talk to 
> the hardware?

Yes.

Re: [discuss] Kernel and rootfs tinkering

[Bryce]

[-- Attachment #1: Type: text/plain, Size: 2430 bytes --]

I see. So the third stage bootloader loads the kernel into memory, as well
as the boot_archive, informs the kernel where the archive is (in memory),
then the kernel uses the drivers and such in the programs to load the root
and execute /sbin/init?

I checked the filelist document in
https://github.com/illumos/illumos-gate/blob/master/usr/src/cmd/boot/filelist/i386/filelist.ramdisk
and didn't see an /init, do does that mean that whatever the kernel does at
that time is completely controlled by the drivers (as I would expect)?

If so, then could it be as simple as sliding an /init into the boot_archive
and then avoid mounting any disk filesystems?

On Fri, Jul 26, 2024, 12:58 PM Bill Sommerfeld via illumos-discuss <
discuss@lists.illumos.org> wrote:

> On 7/26/24 07:14, Bryce wrote:
> > I tried some googling but I couldn't find out what's usually in
> > `boot_archive', nor could I find anything in illumos-gate relating to a
> > default, nor in oi-userland. Is boot_archive generated per distro or is
> > it in illumos-gate?
>
> The boot archive is generated by 'bootadm update-archive', which can be
> run manually by the administrator but it generally gets run
> automatically when needed at reboot time and by the
> svc:/system/boot-archive-update SMF service (which runs at boot once the
> relevant filesystems are mounted read-write).
>
> Under the covers, 'bootadm update-archive' runs the
> /boot/solaris/bin/extract_boot_filelist script to enumerate the files
> and directories from the real root that should be included in the
> archive.   Note that if a directory is named, the entire tree rooted at
> that directory is included.
>
> As others have mentioned, SmartOS does things somewhat differently but
> the above applies to most general-purpose illumos distributions.
>
> > I read the bootadm(8) man page hoping for some answers but none came.
> > Although it did mention the available archive formats, does that mean
> > that boot_archive is archived into an (eg cpio) archive, and then
> > mounted RO by the kernel so it can load the modules necessary to talk to
> > the hardware?
>
> Yes.
>
>
>
> ------------------------------------------
> illumos: illumos-discuss
> Permalink:
> https://illumos.topicbox.com/groups/discuss/Tc9bfa679c7294ee7-M95d858b764d4fd4a52eafa67
> Delivery options: https://illumos.topicbox.com/groups/discuss/subscription
>

[-- Attachment #2: Type: text/html, Size: 3296 bytes --]

Re: [discuss] Kernel and rootfs tinkering

[Joshua M. Clulow]

On Fri, 26 Jul 2024 at 10:49, Bryce <678yym@gmail.com> wrote:
> I see. So the third stage bootloader loads the kernel into memory, as well as the boot_archive, informs the kernel where the archive is (in memory), then the kernel uses the drivers and such in the programs to load the root and execute /sbin/init?
>
> I checked the filelist document in https://github.com/illumos/illumos-gate/blob/master/usr/src/cmd/boot/filelist/i386/filelist.ramdisk and didn't see an /init, do does that mean that whatever the kernel does at that time is completely controlled by the drivers (as I would expect)?

There are two distinct styles of boot archive:

  (a) Archive style; e.g., CPIO.  Essentially a cache of files from
      the real root file system, kept in sync by bootadm(8).  This
      cache only needs to contain:

        - kernel modules required to locate and mount the root
          file system from whichever devices house it

        - any data files (e.g., path_to_inst(5) and system(5) and
          so on) that are needed before the root file system is
          mounted

  (b) Ramdisk style; i.e., a ufs(4FS) image.  This is an entire
      root file system, made available to the kernel by the boot
      loader.  There are a minimal set of routines in the base
      kernel that can pick files out of a UFS image like they can
      for a CPIO archive, before the file system is actually
      mounted.

In addition to the kernel ("unix") and the boot archive (described
above) the boot loader provides either or both of a command line
string, and/or an environment (a list of string key-value pairs).
In this environment, we provide some additional properties that tell
the kernel which file system driver to use and how to locate the
file system (e.g., a /devices path to the disk, or some information
about the network for NFS root, etc).

In the archive case, the loader must provide enough detail that when
the kernel calls vfs_mountroot(), we can locate and mount the root
file system.  Modules and data needed before that point come from
the archive, and after that point, come from the newly mounted file
system.  The init process is started _after_ that, so we don't need
to include it in the archive.

In the ramdisk case, the kernel first treats the UFS image as an
archive up until vfs_mountroot(), when it "mounts" the UFS image
in-place in RAM and things proceed normally as if it were accessing
a physical disk -- with the obvious caveat that changes will be
destroyed on reboot.  This is how install images work today; they
boot from an ISO or a USB image into this kind of ramdisk and then
load other software by finding and mounting the boot device later
from usermode.  It's also how SmartOS works.

At Oxide, we've got a prototype for a new kind of booting, using a
combination of a CPIO boot archive that contains drivers for disks
and the ZFS modules, and then having the kernel load a ramdisk image
from a slice on an NVMe device into RAM.  To do this, I had to add
new hooks into the main() routine of the kernel just prior to
vfs_mountroot() so that we could inject some custom behaviour from a
separate kernel module we build.  It's designed to be customisable
without changing the base kernel source, but also not quite ready to
upstream to mainline illumos -- though I'm keen to get it up there
eventually!

If you want to get started with a minimal boot-to-ramdisk system, I
would probably look at SmartOS, as that's what it is!  It uses the
UFS ramdisk approach today, and that ramdisk is able to locate other
resources on physical disks as needed, etc.  We also have some tools
the allow building images automatically from templates:

    https://github.com/illumos/image-builder

There is no documentation there right now, but I have some examples
of how to use it up here:

    https://github.com/jclulow/omnios-image-builder

I have, in my home directory, other templates I'm yet to add there
which can construct a custom OmniOS-based ramdisk image that you can
add software and custom boot-time behaviour to.  If you want me to
throw that up in there, let me know.

Happy to answer any questions, as I know this is a lot to take in!
It would help to have more concrete specifics about how you're
looking to operate the system you're building, what software you're
hoping to include, how autonomous it should be, how you're hoping to
update it, etc.


Cheers.

-- 
Joshua M. Clulow
http://blog.sysmgr.org

Re: [discuss] Kernel and rootfs tinkering

[Bryce]

[-- Attachment #1: Type: text/plain, Size: 6701 bytes --]

Wow, thanks Joshua

that makes a lot of sense. I'm trying to build a monolithic illumos kernel
that includes its own root file system. I want to do this because I see
that illumos is really only established on x86 with a weak standing in
SPARC, and I'd like to try and port Illumos to other arch's. I'd like to
get a minimal image working on x86 first though, and troubleshoot my way
from there on arm64 hopefully then aarch and maybe risc v. (yes, I know
it's a lot of work, but I enjoy tinkering in assembly and id like to help
the community).

so, the bootloader passes a command line string and perhaps an environment
listing (which I assume is also a string?) to the kerbel, does it pass
these as pointers? If so, are they pushed onto the stack? and if that is
so, then does the bootloader load the kernel and set up SS and BP in the
process?

speaking of which, does the loader switch into protected mode before
loading the kerbel (I assume so, as to load a ufs/cpio archive with drivers
I'd assume you'd need more than a meg of memory.

Also, can the loader be configured to load the kernel from a known address
on disk, instead of from a file system? if not that's fine, I could patch
the source to support that. I ask because I used legacy grub a long time
ago, and I'm pretty sure it doesn't have that feature, but idk how illumos
has patched their grub 0.98.

I'll definitely refer to omnios to see how the big guys do it, but it's not
quite what I'm looking for.

thank you for offering the templates, but I was looking for a slightly
lower level modification.

On Fri, Jul 26, 2024, 2:26 PM Joshua M. Clulow via illumos-discuss <
discuss@lists.illumos.org> wrote:

> On Fri, 26 Jul 2024 at 10:49, Bryce <678yym@gmail.com> wrote:
> > I see. So the third stage bootloader loads the kernel into memory, as
> well as the boot_archive, informs the kernel where the archive is (in
> memory), then the kernel uses the drivers and such in the programs to load
> the root and execute /sbin/init?
> >
> > I checked the filelist document in
> https://github.com/illumos/illumos-gate/blob/master/usr/src/cmd/boot/filelist/i386/filelist.ramdisk
> and didn't see an /init, do does that mean that whatever the kernel does at
> that time is completely controlled by the drivers (as I would expect)?
>
> There are two distinct styles of boot archive:
>
>   (a) Archive style; e.g., CPIO.  Essentially a cache of files from
>       the real root file system, kept in sync by bootadm(8).  This
>       cache only needs to contain:
>
>         - kernel modules required to locate and mount the root
>           file system from whichever devices house it
>
>         - any data files (e.g., path_to_inst(5) and system(5) and
>           so on) that are needed before the root file system is
>           mounted
>
>   (b) Ramdisk style; i.e., a ufs(4FS) image.  This is an entire
>       root file system, made available to the kernel by the boot
>       loader.  There are a minimal set of routines in the base
>       kernel that can pick files out of a UFS image like they can
>       for a CPIO archive, before the file system is actually
>       mounted.
>
> In addition to the kernel ("unix") and the boot archive (described
> above) the boot loader provides either or both of a command line
> string, and/or an environment (a list of string key-value pairs).
> In this environment, we provide some additional properties that tell
> the kernel which file system driver to use and how to locate the
> file system (e.g., a /devices path to the disk, or some information
> about the network for NFS root, etc).
>
> In the archive case, the loader must provide enough detail that when
> the kernel calls vfs_mountroot(), we can locate and mount the root
> file system.  Modules and data needed before that point come from
> the archive, and after that point, come from the newly mounted file
> system.  The init process is started _after_ that, so we don't need
> to include it in the archive.
>
> In the ramdisk case, the kernel first treats the UFS image as an
> archive up until vfs_mountroot(), when it "mounts" the UFS image
> in-place in RAM and things proceed normally as if it were accessing
> a physical disk -- with the obvious caveat that changes will be
> destroyed on reboot.  This is how install images work today; they
> boot from an ISO or a USB image into this kind of ramdisk and then
> load other software by finding and mounting the boot device later
> from usermode.  It's also how SmartOS works.
>
> At Oxide, we've got a prototype for a new kind of booting, using a
> combination of a CPIO boot archive that contains drivers for disks
> and the ZFS modules, and then having the kernel load a ramdisk image
> from a slice on an NVMe device into RAM.  To do this, I had to add
> new hooks into the main() routine of the kernel just prior to
> vfs_mountroot() so that we could inject some custom behaviour from a
> separate kernel module we build.  It's designed to be customisable
> without changing the base kernel source, but also not quite ready to
> upstream to mainline illumos -- though I'm keen to get it up there
> eventually!
>
> If you want to get started with a minimal boot-to-ramdisk system, I
> would probably look at SmartOS, as that's what it is!  It uses the
> UFS ramdisk approach today, and that ramdisk is able to locate other
> resources on physical disks as needed, etc.  We also have some tools
> the allow building images automatically from templates:
>
>     https://github.com/illumos/image-builder
>
> There is no documentation there right now, but I have some examples
> of how to use it up here:
>
>     https://github.com/jclulow/omnios-image-builder
>
> I have, in my home directory, other templates I'm yet to add there
> which can construct a custom OmniOS-based ramdisk image that you can
> add software and custom boot-time behaviour to.  If you want me to
> throw that up in there, let me know.
>
> Happy to answer any questions, as I know this is a lot to take in!
> It would help to have more concrete specifics about how you're
> looking to operate the system you're building, what software you're
> hoping to include, how autonomous it should be, how you're hoping to
> update it, etc.
>
>
> Cheers.
>
> --
> Joshua M. Clulow
> http://blog.sysmgr.org
>
> ------------------------------------------
> illumos: illumos-discuss
> Permalink:
> https://illumos.topicbox.com/groups/discuss/Tc9bfa679c7294ee7-Ma01a51d0c7fe2e4bc5c02ceb
> Delivery options: https://illumos.topicbox.com/groups/discuss/subscription
>

[-- Attachment #2: Type: text/html, Size: 8653 bytes --]

Re: [discuss] Kernel and rootfs tinkering

[Joshua M. Clulow]

On Fri, 26 Jul 2024 at 11:57, Bryce <678yym@gmail.com> wrote:
> that makes a lot of sense. I'm trying to build a monolithic illumos kernel that includes its own root file system. I want to do this because I see that illumos is really only established on x86 with a weak standing in SPARC, and I'd like to try and port Illumos to other arch's. I'd like to get a minimal image working on x86 first though, and troubleshoot my way from there

I do think that's a good place to start, to get familiar with building
OS bits and assembling and configuring them, etc.  Indeed, most people
are using i86pc (x86) these days, so that's a good place to experiment
and it's easy enough to do in a VM on most systems.

> so, the bootloader passes a command line string and perhaps an environment listing (which I assume is also a string?) to the kerbel, does it pass these as pointers? If so, are they pushed onto the stack? and if that is so, then does the bootloader load the kernel and set up SS and BP in the process?
>
> speaking of which, does the loader switch into protected mode before loading the kerbel (I assume so, as to load a ufs/cpio archive with drivers I'd assume you'd need more than a meg of memory.

Our boot loader is actually a Multiboot2 loader:

    https://www.gnu.org/software/grub/manual/multiboot2/multiboot.html

The specification outlines a partially defined expected machine state,
and the layout of control structures that allow us to pass arguments
and modules to the kernel.  On i86pc, we then have "dboot", which is
our Multiboot2 entrypoint and which handles getting the system the
rest of the way up from that predefined state.  The dboot object is
built and stuffed into "unix" along with the rest of the core of the
kernel.

You don't strictly need to use our boot loader, though.  Folks in the
SmartOS community make extensive use of iPXE, from their fork:

    https://github.com/tritonDataCenter/ipxe

At Oxide, we're actually working on a new architecture, "oxide", that
sits as a peer alongside the existing "i86pc".  In our world, there is
no BIOS/EFI firmware at all, just a ROM on the motherboard.  We put
our kernel and CPIO archive into the ROM, and then the kernel itself
loads the root file system from an NVMe disk once it's running.  In
this model, we've had to fold all the things a BIOS would do to get us
to the boot loader, and then what the boot loader would have done,
into our single ROM image.  None of that is as yet available in
illumos-gate, but we're building it to be upstream-ready in the long
term.

    https://github.com/oxidecomputer/illumos-gate/tree/stlouis

> Also, can the loader be configured to load the kernel from a known address on disk, instead of from a file system? if not that's fine, I could patch the source to support that. I ask because I used legacy grub a long time ago, and I'm pretty sure it doesn't have that feature, but idk how illumos has patched their grub 0.98.

We actually don't use our legacy GRUB anymore.  It hasn't been deleted
from the source tree, but it will eventually be when someone finds
time to do that gracefully.  Our current boot loader is in the tree
under "usr/src/boot", and is one we imported from FreeBSD.  I don't
believe that loader can read directly from a disk slice, but it does
have support for several file systems including pcfs (aka FAT), hsfs
(aka ISO), UFS, ZFS, etc.

> I'll definitely refer to omnios to see how the big guys do it, but it's not quite what I'm looking for.
>
> thank you for offering the templates, but I was looking for a slightly lower level modification.

Alright!  You're always welcome to assemble the pieces yourself, of
course.  The templates are just a sequence of steps for doing that, so
even if you don't use them, it might help from time to time to refer
to what they're doing.  The packaging system (which the image builder
will drive) is also responsible for assembling certain data files like
"/etc/driver_aliases" that define which drivers get attached to which
hardware, etc.  I expect Peter has reworked at least some of that in
his "minimal viable illumos" project, so that may be work looking at
also to see what's required in place to boot.


Cheers.

-- 
Joshua M. Clulow
http://blog.sysmgr.org

Re: [discuss] Kernel and rootfs tinkering

[Toomas Soome]

> On 26. Jul 2024, at 22:16, Joshua M. Clulow via illumos-discuss <discuss@lists.illumos.org> wrote:
> 
> 
> We actually don't use our legacy GRUB anymore.  It hasn't been deleted
> from the source tree, but it will eventually be when someone finds
> time to do that gracefully.  Our current boot loader is in the tree
> under "usr/src/boot", and is one we imported from FreeBSD.  I don't
> believe that loader can read directly from a disk slice, but it does
> have support for several file systems including pcfs (aka FAT), hsfs
> (aka ISO), UFS, ZFS, etc.
> 

reading from “raw” disk is nothing impossible - we can actually even script it, but it is another question, why….  With BIOS setup, we actually read stage1 (gptzfsboot) from either boot slice or from zfs label boot block area, and the reason is, stage0 is small an can not have file system driver built in (we can use 446 bytes max;). The cost of it is that you have to have infrastructure to install and maintain your program in raw disk space…

rgds,
toomas

Re: [discuss] Kernel and rootfs tinkering

[Bryce]

[-- Attachment #1: Type: text/plain, Size: 1903 bytes --]

Toomas, you mentioned that you can "script" booting from the unformatted
disk... could you elaborate on this?

Im aware of the sector zero (mbr or vbr) limitations, so i underatand that
stage 1 must be loaded using bios int 13h. But what about stage 1 loading
stage 2? could you point me to the relevant loop in stage 1 that deals with
passing control to stage 2? specifically, how stage 1 finds stage 2 on disk?

Thanks

On Fri, Jul 26, 2024, 4:39 PM Toomas Soome via illumos-discuss <
discuss@lists.illumos.org> wrote:

>
>
> > On 26. Jul 2024, at 22:16, Joshua M. Clulow via illumos-discuss <
> discuss@lists.illumos.org> wrote:
> >
> >
> > We actually don't use our legacy GRUB anymore.  It hasn't been deleted
> > from the source tree, but it will eventually be when someone finds
> > time to do that gracefully.  Our current boot loader is in the tree
> > under "usr/src/boot", and is one we imported from FreeBSD.  I don't
> > believe that loader can read directly from a disk slice, but it does
> > have support for several file systems including pcfs (aka FAT), hsfs
> > (aka ISO), UFS, ZFS, etc.
> >
>
> reading from “raw” disk is nothing impossible - we can actually even
> script it, but it is another question, why….  With BIOS setup, we actually
> read stage1 (gptzfsboot) from either boot slice or from zfs label boot
> block area, and the reason is, stage0 is small an can not have file system
> driver built in (we can use 446 bytes max;). The cost of it is that you
> have to have infrastructure to install and maintain your program in raw
> disk space…
>
> rgds,
> toomas
>
>
> ------------------------------------------
> illumos: illumos-discuss
> Permalink:
> https://illumos.topicbox.com/groups/discuss/Tc9bfa679c7294ee7-M7326ab2012b69062f05d8abf
> Delivery options: https://illumos.topicbox.com/groups/discuss/subscription
>

[-- Attachment #2: Type: text/html, Size: 2693 bytes --]

Re: [discuss] Kernel and rootfs tinkering

[Bryce]

[-- Attachment #1: Type: text/plain, Size: 5216 bytes --]

Thanks for all this, Joshua, I really appreciate it. I'll definitely be
checking out your links, thank God for FOSS.

I think I'll try tinkering with stage 1 and see if I can skip the file
system checks and jump straight to loader (putting loader in memory through
int 13h) and see what can happen from there.

Thanks for the help.

On Fri, Jul 26, 2024, 3:18 PM Joshua M. Clulow via illumos-discuss <
discuss@lists.illumos.org> wrote:

> On Fri, 26 Jul 2024 at 11:57, Bryce <678yym@gmail.com> wrote:
> > that makes a lot of sense. I'm trying to build a monolithic illumos
> kernel that includes its own root file system. I want to do this because I
> see that illumos is really only established on x86 with a weak standing in
> SPARC, and I'd like to try and port Illumos to other arch's. I'd like to
> get a minimal image working on x86 first though, and troubleshoot my way
> from there
>
> I do think that's a good place to start, to get familiar with building
> OS bits and assembling and configuring them, etc.  Indeed, most people
> are using i86pc (x86) these days, so that's a good place to experiment
> and it's easy enough to do in a VM on most systems.
>
> > so, the bootloader passes a command line string and perhaps an
> environment listing (which I assume is also a string?) to the kerbel, does
> it pass these as pointers? If so, are they pushed onto the stack? and if
> that is so, then does the bootloader load the kernel and set up SS and BP
> in the process?
> >
> > speaking of which, does the loader switch into protected mode before
> loading the kerbel (I assume so, as to load a ufs/cpio archive with drivers
> I'd assume you'd need more than a meg of memory.
>
> Our boot loader is actually a Multiboot2 loader:
>
>     https://www.gnu.org/software/grub/manual/multiboot2/multiboot.html
>
> The specification outlines a partially defined expected machine state,
> and the layout of control structures that allow us to pass arguments
> and modules to the kernel.  On i86pc, we then have "dboot", which is
> our Multiboot2 entrypoint and which handles getting the system the
> rest of the way up from that predefined state.  The dboot object is
> built and stuffed into "unix" along with the rest of the core of the
> kernel.
>
> You don't strictly need to use our boot loader, though.  Folks in the
> SmartOS community make extensive use of iPXE, from their fork:
>
>     https://github.com/tritonDataCenter/ipxe
>
> At Oxide, we're actually working on a new architecture, "oxide", that
> sits as a peer alongside the existing "i86pc".  In our world, there is
> no BIOS/EFI firmware at all, just a ROM on the motherboard.  We put
> our kernel and CPIO archive into the ROM, and then the kernel itself
> loads the root file system from an NVMe disk once it's running.  In
> this model, we've had to fold all the things a BIOS would do to get us
> to the boot loader, and then what the boot loader would have done,
> into our single ROM image.  None of that is as yet available in
> illumos-gate, but we're building it to be upstream-ready in the long
> term.
>
>     https://github.com/oxidecomputer/illumos-gate/tree/stlouis
>
> > Also, can the loader be configured to load the kernel from a known
> address on disk, instead of from a file system? if not that's fine, I could
> patch the source to support that. I ask because I used legacy grub a long
> time ago, and I'm pretty sure it doesn't have that feature, but idk how
> illumos has patched their grub 0.98.
>
> We actually don't use our legacy GRUB anymore.  It hasn't been deleted
> from the source tree, but it will eventually be when someone finds
> time to do that gracefully.  Our current boot loader is in the tree
> under "usr/src/boot", and is one we imported from FreeBSD.  I don't
> believe that loader can read directly from a disk slice, but it does
> have support for several file systems including pcfs (aka FAT), hsfs
> (aka ISO), UFS, ZFS, etc.
>
> > I'll definitely refer to omnios to see how the big guys do it, but it's
> not quite what I'm looking for.
> >
> > thank you for offering the templates, but I was looking for a slightly
> lower level modification.
>
> Alright!  You're always welcome to assemble the pieces yourself, of
> course.  The templates are just a sequence of steps for doing that, so
> even if you don't use them, it might help from time to time to refer
> to what they're doing.  The packaging system (which the image builder
> will drive) is also responsible for assembling certain data files like
> "/etc/driver_aliases" that define which drivers get attached to which
> hardware, etc.  I expect Peter has reworked at least some of that in
> his "minimal viable illumos" project, so that may be work looking at
> also to see what's required in place to boot.
>
>
> Cheers.
>
> --
> Joshua M. Clulow
> http://blog.sysmgr.org
>
> ------------------------------------------
> illumos: illumos-discuss
> Permalink:
> https://illumos.topicbox.com/groups/discuss/Tc9bfa679c7294ee7-Me6bd44b7d0839e1c58ed16f4
> Delivery options: https://illumos.topicbox.com/groups/discuss/subscription
>

[-- Attachment #2: Type: text/html, Size: 6668 bytes --]

Re: [discuss] Kernel and rootfs tinkering

[Toomas Soome]

[-- Attachment #1: Type: text/plain, Size: 3002 bytes --]



> On 27. Jul 2024, at 01:20, Bryce <678yym@gmail.com> wrote:
> 
> Toomas, you mentioned that you can "script" booting from the unformatted disk... could you elaborate on this?

since loader is using forth as command interpreter and we have forth words to open, close, read from files, so we can do something like open “disk0p1:” and then read data from it.


> 
> Im aware of the sector zero (mbr or vbr) limitations, so i underatand that stage 1 must be loaded using bios int 13h. But what about stage 1 loading stage 2? could you point me to the relevant loop in stage 1 that deals with passing control to stage 2? specifically, how stage 1 finds stage 2 on disk?
> 

we are just recording starting lba and size in sectors in pmbr binary and it is reading those blocks into memory and jumps there. See usr/src/boot/i386/pmbr/pmbr.s. The values are set by installboot program. To avoid browsing partition tables to guess where our rootfs might be, installboot also does record the lba of rootfs partition in next stage.

rgds,
toomas


> Thanks
> 
> On Fri, Jul 26, 2024, 4:39 PM Toomas Soome via illumos-discuss <discuss@lists.illumos.org <mailto:discuss@lists.illumos.org>> wrote:
>> 
>> 
>> > On 26. Jul 2024, at 22:16, Joshua M. Clulow via illumos-discuss <discuss@lists.illumos.org <mailto:discuss@lists.illumos.org>> wrote:
>> > 
>> > 
>> > We actually don't use our legacy GRUB anymore.  It hasn't been deleted
>> > from the source tree, but it will eventually be when someone finds
>> > time to do that gracefully.  Our current boot loader is in the tree
>> > under "usr/src/boot", and is one we imported from FreeBSD.  I don't
>> > believe that loader can read directly from a disk slice, but it does
>> > have support for several file systems including pcfs (aka FAT), hsfs
>> > (aka ISO), UFS, ZFS, etc.
>> > 
>> 
>> reading from “raw” disk is nothing impossible - we can actually even script it, but it is another question, why….  With BIOS setup, we actually read stage1 (gptzfsboot) from either boot slice or from zfs label boot block area, and the reason is, stage0 is small an can not have file system driver built in (we can use 446 bytes max;). The cost of it is that you have to have infrastructure to install and maintain your program in raw disk space…
>> 
>> rgds,
>> toomas
>> 
>> 
>> ------------------------------------------
>> illumos: illumos-discuss
>> Permalink: https://illumos.topicbox.com/groups/discuss/Tc9bfa679c7294ee7-M7326ab2012b69062f05d8abf
>> Delivery options: https://illumos.topicbox.com/groups/discuss/subscription
> 
> illumos <https://illumos.topicbox.com/latest> / illumos-discuss / see discussions <https://illumos.topicbox.com/groups/discuss> + participants <https://illumos.topicbox.com/groups/discuss/members> + delivery options <https://illumos.topicbox.com/groups/discuss/subscription>Permalink <https://illumos.topicbox.com/groups/discuss/Tc9bfa679c7294ee7-Mf77ff3bfd58e2fa6e6b04724>

[-- Attachment #2: Type: text/html, Size: 4874 bytes --]

Re: [discuss] Kernel and rootfs tinkering

[Bryce]

[-- Attachment #1: Type: text/plain, Size: 3330 bytes --]

Toomas, I may be misunderstanding, but I believe pmbr.s is stage 0 pointing
to stage 1, as determined by the linker. but what about stage 1 to stage 2?
where is the stage 1 code?

thanks

On Fri, Jul 26, 2024, 6:39 PM Toomas Soome via illumos-discuss <
discuss@lists.illumos.org> wrote:

>
>
> On 27. Jul 2024, at 01:20, Bryce <678yym@gmail.com> wrote:
>
> Toomas, you mentioned that you can "script" booting from the unformatted
> disk... could you elaborate on this?
>
>
> since loader is using forth as command interpreter and we have forth words
> to open, close, read from files, so we can do something like open
> “disk0p1:” and then read data from it.
>
>
>
> Im aware of the sector zero (mbr or vbr) limitations, so i underatand that
> stage 1 must be loaded using bios int 13h. But what about stage 1 loading
> stage 2? could you point me to the relevant loop in stage 1 that deals with
> passing control to stage 2? specifically, how stage 1 finds stage 2 on disk?
>
>
> we are just recording starting lba and size in sectors in pmbr binary and
> it is reading those blocks into memory and jumps there. See usr/src/boot/i386/pmbr/pmbr.s.
> The values are set by installboot program. To avoid browsing partition
> tables to guess where our rootfs might be, installboot also does record the
> lba of rootfs partition in next stage.
>
> rgds,
> toomas
>
>
> Thanks
>
> On Fri, Jul 26, 2024, 4:39 PM Toomas Soome via illumos-discuss <
> discuss@lists.illumos.org> wrote:
>
>>
>>
>> > On 26. Jul 2024, at 22:16, Joshua M. Clulow via illumos-discuss <
>> discuss@lists.illumos.org> wrote:
>> >
>> >
>> > We actually don't use our legacy GRUB anymore.  It hasn't been deleted
>> > from the source tree, but it will eventually be when someone finds
>> > time to do that gracefully.  Our current boot loader is in the tree
>> > under "usr/src/boot", and is one we imported from FreeBSD.  I don't
>> > believe that loader can read directly from a disk slice, but it does
>> > have support for several file systems including pcfs (aka FAT), hsfs
>> > (aka ISO), UFS, ZFS, etc.
>> >
>>
>> reading from “raw” disk is nothing impossible - we can actually even
>> script it, but it is another question, why….  With BIOS setup, we actually
>> read stage1 (gptzfsboot) from either boot slice or from zfs label boot
>> block area, and the reason is, stage0 is small an can not have file system
>> driver built in (we can use 446 bytes max;). The cost of it is that you
>> have to have infrastructure to install and maintain your program in raw
>> disk space…
>>
>> rgds,
>> toomas
>>
>>
>> ------------------------------------------
>> illumos: illumos-discuss
>> Permalink:
>> https://illumos.topicbox.com/groups/discuss/Tc9bfa679c7294ee7-M7326ab2012b69062f05d8abf
>> Delivery options:
>> https://illumos.topicbox.com/groups/discuss/subscription
>>
>
> *illumos <https://illumos.topicbox.com/latest>* / illumos-discuss / see
> discussions <https://illumos.topicbox.com/groups/discuss> + participants
> <https://illumos.topicbox.com/groups/discuss/members> + delivery options
> <https://illumos.topicbox.com/groups/discuss/subscription> Permalink
> <https://illumos.topicbox.com/groups/discuss/Tc9bfa679c7294ee7-M8a383c71a51fe01800536ffe>
>

[-- Attachment #2: Type: text/html, Size: 5617 bytes --]

Re: [discuss] Kernel and rootfs tinkering

[Toomas Soome]

[-- Attachment #1: Type: text/plain, Size: 3605 bytes --]



> On 27. Jul 2024, at 04:20, Bryce <678yym@gmail.com> wrote:
> 
> Toomas, I may be misunderstanding, but I believe pmbr.s is stage 0 pointing to stage 1, as determined by the linker. but what about stage 1 to stage 2? where is the stage 1 code?

usr/src/boot/i386/gptzfsboot/zfsboot.c

rgds,
toomas

> 
> thanks
> 
> On Fri, Jul 26, 2024, 6:39 PM Toomas Soome via illumos-discuss <discuss@lists.illumos.org <mailto:discuss@lists.illumos.org>> wrote:
>> 
>> 
>>> On 27. Jul 2024, at 01:20, Bryce <678yym@gmail.com <mailto:678yym@gmail.com>> wrote:
>>> 
>>> Toomas, you mentioned that you can "script" booting from the unformatted disk... could you elaborate on this?
>> 
>> since loader is using forth as command interpreter and we have forth words to open, close, read from files, so we can do something like open “disk0p1:” and then read data from it.
>> 
>> 
>>> 
>>> Im aware of the sector zero (mbr or vbr) limitations, so i underatand that stage 1 must be loaded using bios int 13h. But what about stage 1 loading stage 2? could you point me to the relevant loop in stage 1 that deals with passing control to stage 2? specifically, how stage 1 finds stage 2 on disk?
>>> 
>> 
>> we are just recording starting lba and size in sectors in pmbr binary and it is reading those blocks into memory and jumps there. See usr/src/boot/i386/pmbr/pmbr.s. The values are set by installboot program. To avoid browsing partition tables to guess where our rootfs might be, installboot also does record the lba of rootfs partition in next stage.
>> 
>> rgds,
>> toomas
>> 
>> 
>>> Thanks
>>> 
>>> On Fri, Jul 26, 2024, 4:39 PM Toomas Soome via illumos-discuss <discuss@lists.illumos.org <mailto:discuss@lists.illumos.org>> wrote:
>>>> 
>>>> 
>>>> > On 26. Jul 2024, at 22:16, Joshua M. Clulow via illumos-discuss <discuss@lists.illumos.org <mailto:discuss@lists.illumos.org>> wrote:
>>>> > 
>>>> > 
>>>> > We actually don't use our legacy GRUB anymore.  It hasn't been deleted
>>>> > from the source tree, but it will eventually be when someone finds
>>>> > time to do that gracefully.  Our current boot loader is in the tree
>>>> > under "usr/src/boot", and is one we imported from FreeBSD.  I don't
>>>> > believe that loader can read directly from a disk slice, but it does
>>>> > have support for several file systems including pcfs (aka FAT), hsfs
>>>> > (aka ISO), UFS, ZFS, etc.
>>>> > 
>>>> 
>>>> reading from “raw” disk is nothing impossible - we can actually even script it, but it is another question, why….  With BIOS setup, we actually read stage1 (gptzfsboot) from either boot slice or from zfs label boot block area, and the reason is, stage0 is small an can not have file system driver built in (we can use 446 bytes max;). The cost of it is that you have to have infrastructure to install and maintain your program in raw disk space…
>>>> 
>>>> rgds,
>>>> toomas
>>>> 
>>>> 
>>>> ------------------------------------------
>>>> illumos: illumos-discuss
>>>> Permalink: https://illumos.topicbox.com/groups/discuss/Tc9bfa679c7294ee7-M7326ab2012b69062f05d8abf
>>>> Delivery options: https://illumos.topicbox.com/groups/discuss/subscription
>> 
> 
> illumos <https://illumos.topicbox.com/latest> / illumos-discuss / see discussions <https://illumos.topicbox.com/groups/discuss> + participants <https://illumos.topicbox.com/groups/discuss/members> + delivery options <https://illumos.topicbox.com/groups/discuss/subscription>Permalink <https://illumos.topicbox.com/groups/discuss/Tc9bfa679c7294ee7-Mc217b4a13817c5c0be2ce90f>

[-- Attachment #2: Type: text/html, Size: 6025 bytes --]

Aw: Re: [discuss] Kernel and rootfs tinkering

[Ignacio Soriano Hernandez]

[-- Attachment #1: Type: text/plain, Size: 7356 bytes --]

Hi Bryce, let me first complain as I cannot help you with your request. Reading this posting I just ask myself, why is this the next try to make something where illumos and several distros based on it still lack core capabilities (see the driver support, power management, graphics subsystem, etc, etc.) there even have been several proof of concepts with regards to having an illumos based distro on arm64 (yes, several projects made it up to prompt and some core capabilities) while someone with that deep knowledge would be a huge asset to get some things done to close some gaps? And don't get me wrong please, I like that there are projects one wants and has to do because it THAT what gathers your interest :-) So if you like to get into the core core there are lots of areas where someone knowledgeable like you could deliver so much value to the whole community . Thanks for your passion and motiviation and welcome in the rabbithole :-) Cheers from Germany Iggi Bryce <678yym@gmail.com> schrieb am 26. Juli 2024 um 20:59: Wow, thanks Joshua that makes a lot of sense. I'm trying to build a monolithic illumos kernel that includes its own root file system. I want to do this because I see that illumos is really only established on x86 with a weak standing in SPARC, and I'd like to try and port Illumos to other arch's. I'd like to get a minimal image working on x86 first though, and troubleshoot my way from there on arm64 hopefully then aarch and maybe risc v. (yes, I know it's a lot of work, but I enjoy tinkering in assembly and id like to help the community). so, the bootloader passes a command line string and perhaps an environment listing (which I assume is also a string?) to the kerbel, does it pass these as pointers? If so, are they pushed onto the stack? and if that is so, then does the bootloader load the kernel and set up SS and BP in the process? speaking of which, does the loader switch into protected mode before loading the kerbel (I assume so, as to load a ufs/cpio archive with drivers I'd assume you'd need more than a meg of memory. Also, can the loader be configured to load the kernel from a known address on disk, instead of from a file system? if not that's fine, I could patch the source to support that. I ask because I used legacy grub a long time ago, and I'm pretty sure it doesn't have that feature, but idk how illumos has patched their grub 0.98. I'll definitely refer to omnios to see how the big guys do it, but it's not quite what I'm looking for. thank you for offering the templates, but I was looking for a slightly lower level modification. On Fri, Jul 26, 2024, 2:26 PM Joshua M. Clulow via illumos-discuss < discuss@lists.illumos.org > wrote: On Fri, 26 Jul 2024 at 10:49, Bryce < 678yym@gmail.com > wrote: > I see. So the third stage bootloader loads the kernel into memory, as well as the boot_archive, informs the kernel where the archive is (in memory), then the kernel uses the drivers and such in the programs to load the root and execute /sbin/init? > > I checked the filelist document in https://github.com/illumos/illumos-gate/blob/master/usr/src/cmd/boot/filelist/i386/filelist.ramdisk and didn't see an /init, do does that mean that whatever the kernel does at that time is completely controlled by the drivers (as I would expect)? There are two distinct styles of boot archive: (a) Archive style; e.g., CPIO. Essentially a cache of files from the real root file system, kept in sync by bootadm(8). This cache only needs to contain: - kernel modules required to locate and mount the root file system from whichever devices house it - any data files (e.g., path_to_inst(5) and system(5) and so on) that are needed before the root file system is mounted (b) Ramdisk style; i.e., a ufs(4FS) image. This is an entire root file system, made available to the kernel by the boot loader. There are a minimal set of routines in the base kernel that can pick files out of a UFS image like they can for a CPIO archive, before the file system is actually mounted. In addition to the kernel ("unix") and the boot archive (described above) the boot loader provides either or both of a command line string, and/or an environment (a list of string key-value pairs). In this environment, we provide some additional properties that tell the kernel which file system driver to use and how to locate the file system (e.g., a /devices path to the disk, or some information about the network for NFS root, etc). In the archive case, the loader must provide enough detail that when the kernel calls vfs_mountroot(), we can locate and mount the root file system. Modules and data needed before that point come from the archive, and after that point, come from the newly mounted file system. The init process is started _after_ that, so we don't need to include it in the archive. In the ramdisk case, the kernel first treats the UFS image as an archive up until vfs_mountroot(), when it "mounts" the UFS image in-place in RAM and things proceed normally as if it were accessing a physical disk -- with the obvious caveat that changes will be destroyed on reboot. This is how install images work today; they boot from an ISO or a USB image into this kind of ramdisk and then load other software by finding and mounting the boot device later from usermode. It's also how SmartOS works. At Oxide, we've got a prototype for a new kind of booting, using a combination of a CPIO boot archive that contains drivers for disks and the ZFS modules, and then having the kernel load a ramdisk image from a slice on an NVMe device into RAM. To do this, I had to add new hooks into the main() routine of the kernel just prior to vfs_mountroot() so that we could inject some custom behaviour from a separate kernel module we build. It's designed to be customisable without changing the base kernel source, but also not quite ready to upstream to mainline illumos -- though I'm keen to get it up there eventually! If you want to get started with a minimal boot-to-ramdisk system, I would probably look at SmartOS, as that's what it is! It uses the UFS ramdisk approach today, and that ramdisk is able to locate other resources on physical disks as needed, etc. We also have some tools the allow building images automatically from templates: https://github.com/illumos/image-builder There is no documentation there right now, but I have some examples of how to use it up here: https://github.com/jclulow/omnios-image-builder I have, in my home directory, other templates I'm yet to add there which can construct a custom OmniOS-based ramdisk image that you can add software and custom boot-time behaviour to. If you want me to throw that up in there, let me know. Happy to answer any questions, as I know this is a lot to take in! It would help to have more concrete specifics about how you're looking to operate the system you're building, what software you're hoping to include, how autonomous it should be, how you're hoping to update it, etc. Cheers. -- Joshua M. Clulow http://blog.sysmgr.org ------------------------------------------ illumos: illumos-discuss Permalink: https://illumos.topicbox.com/groups/discuss/Tc9bfa679c7294ee7-Ma01a51d0c7fe2e4bc5c02ceb Delivery options: https://illumos.topicbox.com/groups/discuss/subscription illumos / illumos-discuss / see discussions + participants + delivery options Permalink

[-- Attachment #2.1: Type: text/html, Size: 11599 bytes --]

Re: Re: [discuss] Kernel and rootfs tinkering

[Bryce]

[-- Attachment #1: Type: text/plain, Size: 8947 bytes --]

Hi, Iggi:

I certainly would not mind contributing wherever I could. I've been reading
through the source for the bootloader, if there are any issues that need to
be addressed id love to see if I could help.

I'd like to port Illumos because, while there do exist several ports to
arm64, they are all incomplete or don't work for some people. id like to
try fixing this.

If there's anything that I could contribute I'd love to know.


On Sat, Jul 27, 2024, 12:09 PM Ignacio Soriano Hernandez via
illumos-discuss <discuss@lists.illumos.org> wrote:

> Hi Bryce,
>
> let me first complain as I cannot help you with your request.
>
> Reading this posting I just ask myself, why is this the next try to make
> something where illumos and several distros based on it still lack core
> capabilities (see the driver support, power management, graphics subsystem,
> etc, etc.) there even have been several proof of concepts with regards to
> having an illumos based distro on arm64 (yes, several projects made it up
> to prompt and some core capabilities) while someone with that deep
> knowledge would be a huge asset to get some things done to close some gaps?
> And don't get me wrong please, I like that there are projects one wants
> and has to do because it THAT what gathers your interest :-)
> So if you like to get into the core core there are lots of areas where
> someone knowledgeable like you could deliver so much value to the whole
> community .
>
> Thanks for your passion and motiviation and welcome in the rabbithole :-)
>
> Cheers from Germany
>
> Iggi
>
> Bryce <678yym@gmail.com> schrieb am 26. Juli 2024 um 20:59:
>
>
> Wow, thanks Joshua
>
> that makes a lot of sense. I'm trying to build a monolithic illumos kernel
> that includes its own root file system. I want to do this because I see
> that illumos is really only established on x86 with a weak standing in
> SPARC, and I'd like to try and port Illumos to other arch's. I'd like to
> get a minimal image working on x86 first though, and troubleshoot my way
> from there on arm64 hopefully then aarch and maybe risc v. (yes, I know
> it's a lot of work, but I enjoy tinkering in assembly and id like to help
> the community).
>
> so, the bootloader passes a command line string and perhaps an environment
> listing (which I assume is also a string?) to the kerbel, does it pass
> these as pointers? If so, are they pushed onto the stack? and if that is
> so, then does the bootloader load the kernel and set up SS and BP in the
> process?
>
> speaking of which, does the loader switch into protected mode before
> loading the kerbel (I assume so, as to load a ufs/cpio archive with drivers
> I'd assume you'd need more than a meg of memory.
>
> Also, can the loader be configured to load the kernel from a known address
> on disk, instead of from a file system? if not that's fine, I could patch
> the source to support that. I ask because I used legacy grub a long time
> ago, and I'm pretty sure it doesn't have that feature, but idk how illumos
> has patched their grub 0.98.
>
> I'll definitely refer to omnios to see how the big guys do it, but it's
> not quite what I'm looking for.
>
> thank you for offering the templates, but I was looking for a slightly
> lower level modification.
>
> On Fri, Jul 26, 2024, 2:26 PM Joshua M. Clulow via illumos-discuss <
> discuss@lists.illumos.org> wrote:
>
>> On Fri, 26 Jul 2024 at 10:49, Bryce <678yym@gmail.com> wrote:
>> > I see. So the third stage bootloader loads the kernel into memory, as
>> well as the boot_archive, informs the kernel where the archive is (in
>> memory), then the kernel uses the drivers and such in the programs to load
>> the root and execute /sbin/init?
>> >
>> > I checked the filelist document in
>> https://github.com/illumos/illumos-gate/blob/master/usr/src/cmd/boot/filelist/i386/filelist.ramdisk
>> and didn't see an /init, do does that mean that whatever the kernel does at
>> that time is completely controlled by the drivers (as I would expect)?
>>
>> There are two distinct styles of boot archive:
>>
>>   (a) Archive style; e.g., CPIO.  Essentially a cache of files from
>>       the real root file system, kept in sync by bootadm(8).  This
>>       cache only needs to contain:
>>
>>         - kernel modules required to locate and mount the root
>>           file system from whichever devices house it
>>
>>         - any data files (e.g., path_to_inst(5) and system(5) and
>>           so on) that are needed before the root file system is
>>           mounted
>>
>>   (b) Ramdisk style; i.e., a ufs(4FS) image.  This is an entire
>>       root file system, made available to the kernel by the boot
>>       loader.  There are a minimal set of routines in the base
>>       kernel that can pick files out of a UFS image like they can
>>       for a CPIO archive, before the file system is actually
>>       mounted.
>>
>> In addition to the kernel ("unix") and the boot archive (described
>> above) the boot loader provides either or both of a command line
>> string, and/or an environment (a list of string key-value pairs).
>> In this environment, we provide some additional properties that tell
>> the kernel which file system driver to use and how to locate the
>> file system (e.g., a /devices path to the disk, or some information
>> about the network for NFS root, etc).
>>
>> In the archive case, the loader must provide enough detail that when
>> the kernel calls vfs_mountroot(), we can locate and mount the root
>> file system.  Modules and data needed before that point come from
>> the archive, and after that point, come from the newly mounted file
>> system.  The init process is started _after_ that, so we don't need
>> to include it in the archive.
>>
>> In the ramdisk case, the kernel first treats the UFS image as an
>> archive up until vfs_mountroot(), when it "mounts" the UFS image
>> in-place in RAM and things proceed normally as if it were accessing
>> a physical disk -- with the obvious caveat that changes will be
>> destroyed on reboot.  This is how install images work today; they
>> boot from an ISO or a USB image into this kind of ramdisk and then
>> load other software by finding and mounting the boot device later
>> from usermode.  It's also how SmartOS works.
>>
>> At Oxide, we've got a prototype for a new kind of booting, using a
>> combination of a CPIO boot archive that contains drivers for disks
>> and the ZFS modules, and then having the kernel load a ramdisk image
>> from a slice on an NVMe device into RAM.  To do this, I had to add
>> new hooks into the main() routine of the kernel just prior to
>> vfs_mountroot() so that we could inject some custom behaviour from a
>> separate kernel module we build.  It's designed to be customisable
>> without changing the base kernel source, but also not quite ready to
>> upstream to mainline illumos -- though I'm keen to get it up there
>> eventually!
>>
>> If you want to get started with a minimal boot-to-ramdisk system, I
>> would probably look at SmartOS, as that's what it is!  It uses the
>> UFS ramdisk approach today, and that ramdisk is able to locate other
>> resources on physical disks as needed, etc.  We also have some tools
>> the allow building images automatically from templates:
>>
>>     https://github.com/illumos/image-builder
>>
>> There is no documentation there right now, but I have some examples
>> of how to use it up here:
>>
>>     https://github.com/jclulow/omnios-image-builder
>>
>> I have, in my home directory, other templates I'm yet to add there
>> which can construct a custom OmniOS-based ramdisk image that you can
>> add software and custom boot-time behaviour to.  If you want me to
>> throw that up in there, let me know.
>>
>> Happy to answer any questions, as I know this is a lot to take in!
>> It would help to have more concrete specifics about how you're
>> looking to operate the system you're building, what software you're
>> hoping to include, how autonomous it should be, how you're hoping to
>> update it, etc.
>>
>>
>> Cheers.
>>
>> --
>> Joshua M. Clulow
>> http://blog.sysmgr.org
>>
>> ------------------------------------------
>> illumos: illumos-discuss
>> Permalink:
>> https://illumos.topicbox.com/groups/discuss/Tc9bfa679c7294ee7-Ma01a51d0c7fe2e4bc5c02ceb
>> Delivery options:
>> https://illumos.topicbox.com/groups/discuss/subscription
>>
>
>
> *illumos <https://illumos.topicbox.com/latest>* / illumos-discuss / see
> discussions <https://illumos.topicbox.com/groups/discuss> + participants
> <https://illumos.topicbox.com/groups/discuss/members> + delivery options
> <https://illumos.topicbox.com/groups/discuss/subscription> Permalink
> <https://illumos.topicbox.com/groups/discuss/Ta3de79b601a98cac-M322da0570ac087beba9e72dc>
>

[-- Attachment #2: Type: text/html, Size: 12813 bytes --]

Aw: Re: Re: [discuss] Kernel and rootfs tinkering

[Ignacio Soriano Hernandez]

[-- Attachment #1: Type: text/plain, Size: 8875 bytes --]

Hi Bryce, that would be awesome, if you just look into: https://www.illumos.org/projects There is so much for your taste that you be overwhelmed by the things that are pending in all areas and due to capacity and knowledge are more in "bug fixing" mode than anything else :-) graphicsdrm, illumos-gate, OpenIndiana you can select depending on your taste .. and yes, the arm64 approaches had more an experimental approach with lots and lots of "custom" workarounds (don't want to call them "hacks" :-)) Have you installed one of the illumos based dirtros on baremetal (or running in a VM)? .. then you will instantly find out what is missing .. because you will be joining from an operating system that has basically support for everything .. Btw. Toasterson (nick) is one of the maintainers of OI, nice guy, very helpful and he arranged some co-working sessions to tackle the most "urgent" topics. Cheers Iggi Bryce < 678yym@gmail.com > schrieb am 27. Juli 2024 um 18:58: Hi, Iggi: I certainly would not mind contributing wherever I could. I've been reading through the source for the bootloader, if there are any issues that need to be addressed id love to see if I could help. I'd like to port Illumos because, while there do exist several ports to arm64, they are all incomplete or don't work for some people. id like to try fixing this. If there's anything that I could contribute I'd love to know. On Sat, Jul 27, 2024, 12:09 PM Ignacio Soriano Hernandez via illumos-discuss < discuss@lists.illumos.org > wrote: Hi Bryce, let me first complain as I cannot help you with your request. Reading this posting I just ask myself, why is this the next try to make something where illumos and several distros based on it still lack core capabilities (see the driver support, power management, graphics subsystem, etc, etc.) there even have been several proof of concepts with regards to having an illumos based distro on arm64 (yes, several projects made it up to prompt and some core capabilities) while someone with that deep knowledge would be a huge asset to get some things done to close some gaps? And don't get me wrong please, I like that there are projects one wants and has to do because it THAT what gathers your interest :-) So if you like to get into the core core there are lots of areas where someone knowledgeable like you could deliver so much value to the whole community . Thanks for your passion and motiviation and welcome in the rabbithole :-) Cheers from Germany Iggi Bryce < 678yym@gmail.com > schrieb am 26. Juli 2024 um 20:59: Wow, thanks Joshua that makes a lot of sense. I'm trying to build a monolithic illumos kernel that includes its own root file system. I want to do this because I see that illumos is really only established on x86 with a weak standing in SPARC, and I'd like to try and port Illumos to other arch's. I'd like to get a minimal image working on x86 first though, and troubleshoot my way from there on arm64 hopefully then aarch and maybe risc v. (yes, I know it's a lot of work, but I enjoy tinkering in assembly and id like to help the community). so, the bootloader passes a command line string and perhaps an environment listing (which I assume is also a string?) to the kerbel, does it pass these as pointers? If so, are they pushed onto the stack? and if that is so, then does the bootloader load the kernel and set up SS and BP in the process? speaking of which, does the loader switch into protected mode before loading the kerbel (I assume so, as to load a ufs/cpio archive with drivers I'd assume you'd need more than a meg of memory. Also, can the loader be configured to load the kernel from a known address on disk, instead of from a file system? if not that's fine, I could patch the source to support that. I ask because I used legacy grub a long time ago, and I'm pretty sure it doesn't have that feature, but idk how illumos has patched their grub 0.98. I'll definitely refer to omnios to see how the big guys do it, but it's not quite what I'm looking for. thank you for offering the templates, but I was looking for a slightly lower level modification. On Fri, Jul 26, 2024, 2:26 PM Joshua M. Clulow via illumos-discuss < discuss@lists.illumos.org > wrote: On Fri, 26 Jul 2024 at 10:49, Bryce < 678yym@gmail.com > wrote: > I see. So the third stage bootloader loads the kernel into memory, as well as the boot_archive, informs the kernel where the archive is (in memory), then the kernel uses the drivers and such in the programs to load the root and execute /sbin/init? > > I checked the filelist document in https://github.com/illumos/illumos-gate/blob/master/usr/src/cmd/boot/filelist/i386/filelist.ramdisk and didn't see an /init, do does that mean that whatever the kernel does at that time is completely controlled by the drivers (as I would expect)? There are two distinct styles of boot archive: (a) Archive style; e.g., CPIO. Essentially a cache of files from the real root file system, kept in sync by bootadm(8). This cache only needs to contain: - kernel modules required to locate and mount the root file system from whichever devices house it - any data files (e.g., path_to_inst(5) and system(5) and so on) that are needed before the root file system is mounted (b) Ramdisk style; i.e., a ufs(4FS) image. This is an entire root file system, made available to the kernel by the boot loader. There are a minimal set of routines in the base kernel that can pick files out of a UFS image like they can for a CPIO archive, before the file system is actually mounted. In addition to the kernel ("unix") and the boot archive (described above) the boot loader provides either or both of a command line string, and/or an environment (a list of string key-value pairs). In this environment, we provide some additional properties that tell the kernel which file system driver to use and how to locate the file system (e.g., a /devices path to the disk, or some information about the network for NFS root, etc). In the archive case, the loader must provide enough detail that when the kernel calls vfs_mountroot(), we can locate and mount the root file system. Modules and data needed before that point come from the archive, and after that point, come from the newly mounted file system. The init process is started _after_ that, so we don't need to include it in the archive. In the ramdisk case, the kernel first treats the UFS image as an archive up until vfs_mountroot(), when it "mounts" the UFS image in-place in RAM and things proceed normally as if it were accessing a physical disk -- with the obvious caveat that changes will be destroyed on reboot. This is how install images work today; they boot from an ISO or a USB image into this kind of ramdisk and then load other software by finding and mounting the boot device later from usermode. It's also how SmartOS works. At Oxide, we've got a prototype for a new kind of booting, using a combination of a CPIO boot archive that contains drivers for disks and the ZFS modules, and then having the kernel load a ramdisk image from a slice on an NVMe device into RAM. To do this, I had to add new hooks into the main() routine of the kernel just prior to vfs_mountroot() so that we could inject some custom behaviour from a separate kernel module we build. It's designed to be customisable without changing the base kernel source, but also not quite ready to upstream to mainline illumos -- though I'm keen to get it up there eventually! If you want to get started with a minimal boot-to-ramdisk system, I would probably look at SmartOS, as that's what it is! It uses the UFS ramdisk approach today, and that ramdisk is able to locate other resources on physical disks as needed, etc. We also have some tools the allow building images automatically from templates: https://github.com/illumos/image-builder There is no documentation there right now, but I have some examples of how to use it up here: https://github.com/jclulow/omnios-image-builder I have, in my home directory, other templates I'm yet to add there which can construct a custom OmniOS-based ramdisk image that you can add software and custom boot-time behaviour to. If you want me to throw that up in there, let me know. Happy to answer any questions, as I know this is a lot to take in! It would help to have more concrete specifics about how you're looking to operate the system you're building, what software you're hoping to include, how autonomous it should be, how you're hoping to update it, etc. Cheers. -- Joshua M. Clulow http://blog.sysmgr.org ------------------------------------------ illumos: illumos-discuss Permalink: https://illumos.topicbox.com/groups/discuss/Tc9bfa679c7294ee7-Ma01a51d0c7fe2e4bc5c02ceb Delivery options: https://illumos.topicbox.com/groups/discuss/subscription illumos / illumos-discuss / see discussions + participants + delivery options Permalink

[-- Attachment #2.1: Type: text/html, Size: 14145 bytes --]

Re: Re: Re: [discuss] Kernel and rootfs tinkering

[Bryce]

[-- Attachment #1: Type: text/plain, Size: 10505 bytes --]

Hi Iggi,

Alright, I'll take a look at the Issues and such under the GitHub repos.

I have OI running on an amd64 uefi+csm machine, and an amd64 bios-only
machine.

On Sat, Jul 27, 2024, 1:23 PM Ignacio Soriano Hernandez via illumos-discuss
<discuss@lists.illumos.org> wrote:

>
> Hi Bryce,
>
> that would be awesome, if you just look into:
>
> https://www.illumos.org/projects
>
> There is so much for your taste that you be overwhelmed by the things that
> are pending in all areas and due to capacity and knowledge are more in "bug
> fixing" mode than anything else :-)
>
> graphicsdrm, illumos-gate, OpenIndiana you can select depending on your
> taste ..  and yes, the arm64 approaches had more an experimental approach
> with lots and lots of "custom" workarounds (don't want to call them "hacks"
> :-))
>
> Have you installed one of the illumos based dirtros on baremetal (or
> running in a VM)? .. then you will instantly find out what is missing ..
> because you will be joining from an operating system that has basically
> support for everything ..
>
> Btw. Toasterson (nick) is one of the maintainers of OI, nice guy, very
> helpful and he arranged some co-working sessions to tackle the most
> "urgent" topics.
>
> Cheers
>
> Iggi
>
> Bryce <678yym@gmail.com> schrieb am 27. Juli 2024 um 18:58:
>
>
> Hi, Iggi:
>
> I certainly would not mind contributing wherever I could. I've been
> reading through the source for the bootloader, if there are any issues that
> need to be addressed id love to see if I could help.
>
> I'd like to port Illumos because, while there do exist several ports to
> arm64, they are all incomplete or don't work for some people. id like to
> try fixing this.
>
> If there's anything that I could contribute I'd love to know.
>
>
> On Sat, Jul 27, 2024, 12:09 PM Ignacio Soriano Hernandez via
> illumos-discuss <discuss@lists.illumos.org> wrote:
>
>> Hi Bryce,
>>
>> let me first complain as I cannot help you with your request.
>>
>> Reading this posting I just ask myself, why is this the next try to make
>> something where illumos and several distros based on it still lack core
>> capabilities (see the driver support, power management, graphics subsystem,
>> etc, etc.) there even have been several proof of concepts with regards to
>> having an illumos based distro on arm64 (yes, several projects made it up
>> to prompt and some core capabilities) while someone with that deep
>> knowledge would be a huge asset to get some things done to close some gaps?
>> And don't get me wrong please, I like that there are projects one wants
>> and has to do because it THAT what gathers your interest :-)
>> So if you like to get into the core core there are lots of areas where
>> someone knowledgeable like you could deliver so much value to the whole
>> community .
>>
>> Thanks for your passion and motiviation and welcome in the rabbithole :-)
>>
>> Cheers from Germany
>>
>> Iggi
>>
>> Bryce <678yym@gmail.com> schrieb am 26. Juli 2024 um 20:59:
>>
>>
>> Wow, thanks Joshua
>>
>> that makes a lot of sense. I'm trying to build a monolithic illumos
>> kernel that includes its own root file system. I want to do this because I
>> see that illumos is really only established on x86 with a weak standing in
>> SPARC, and I'd like to try and port Illumos to other arch's. I'd like to
>> get a minimal image working on x86 first though, and troubleshoot my way
>> from there on arm64 hopefully then aarch and maybe risc v. (yes, I know
>> it's a lot of work, but I enjoy tinkering in assembly and id like to help
>> the community).
>>
>> so, the bootloader passes a command line string and perhaps an
>> environment listing (which I assume is also a string?) to the kerbel, does
>> it pass these as pointers? If so, are they pushed onto the stack? and if
>> that is so, then does the bootloader load the kernel and set up SS and BP
>> in the process?
>>
>> speaking of which, does the loader switch into protected mode before
>> loading the kerbel (I assume so, as to load a ufs/cpio archive with drivers
>> I'd assume you'd need more than a meg of memory.
>>
>> Also, can the loader be configured to load the kernel from a known
>> address on disk, instead of from a file system? if not that's fine, I could
>> patch the source to support that. I ask because I used legacy grub a long
>> time ago, and I'm pretty sure it doesn't have that feature, but idk how
>> illumos has patched their grub 0.98.
>>
>> I'll definitely refer to omnios to see how the big guys do it, but it's
>> not quite what I'm looking for.
>>
>> thank you for offering the templates, but I was looking for a slightly
>> lower level modification.
>>
>> On Fri, Jul 26, 2024, 2:26 PM Joshua M. Clulow via illumos-discuss <
>> discuss@lists.illumos.org> wrote:
>>
>>> On Fri, 26 Jul 2024 at 10:49, Bryce <678yym@gmail.com> wrote:
>>> > I see. So the third stage bootloader loads the kernel into memory, as
>>> well as the boot_archive, informs the kernel where the archive is (in
>>> memory), then the kernel uses the drivers and such in the programs to load
>>> the root and execute /sbin/init?
>>> >
>>> > I checked the filelist document in
>>> https://github.com/illumos/illumos-gate/blob/master/usr/src/cmd/boot/filelist/i386/filelist.ramdisk
>>> and didn't see an /init, do does that mean that whatever the kernel does at
>>> that time is completely controlled by the drivers (as I would expect)?
>>>
>>> There are two distinct styles of boot archive:
>>>
>>>   (a) Archive style; e.g., CPIO.  Essentially a cache of files from
>>>       the real root file system, kept in sync by bootadm(8).  This
>>>       cache only needs to contain:
>>>
>>>         - kernel modules required to locate and mount the root
>>>           file system from whichever devices house it
>>>
>>>         - any data files (e.g., path_to_inst(5) and system(5) and
>>>           so on) that are needed before the root file system is
>>>           mounted
>>>
>>>   (b) Ramdisk style; i.e., a ufs(4FS) image.  This is an entire
>>>       root file system, made available to the kernel by the boot
>>>       loader.  There are a minimal set of routines in the base
>>>       kernel that can pick files out of a UFS image like they can
>>>       for a CPIO archive, before the file system is actually
>>>       mounted.
>>>
>>> In addition to the kernel ("unix") and the boot archive (described
>>> above) the boot loader provides either or both of a command line
>>> string, and/or an environment (a list of string key-value pairs).
>>> In this environment, we provide some additional properties that tell
>>> the kernel which file system driver to use and how to locate the
>>> file system (e.g., a /devices path to the disk, or some information
>>> about the network for NFS root, etc).
>>>
>>> In the archive case, the loader must provide enough detail that when
>>> the kernel calls vfs_mountroot(), we can locate and mount the root
>>> file system.  Modules and data needed before that point come from
>>> the archive, and after that point, come from the newly mounted file
>>> system.  The init process is started _after_ that, so we don't need
>>> to include it in the archive.
>>>
>>> In the ramdisk case, the kernel first treats the UFS image as an
>>> archive up until vfs_mountroot(), when it "mounts" the UFS image
>>> in-place in RAM and things proceed normally as if it were accessing
>>> a physical disk -- with the obvious caveat that changes will be
>>> destroyed on reboot.  This is how install images work today; they
>>> boot from an ISO or a USB image into this kind of ramdisk and then
>>> load other software by finding and mounting the boot device later
>>> from usermode.  It's also how SmartOS works.
>>>
>>> At Oxide, we've got a prototype for a new kind of booting, using a
>>> combination of a CPIO boot archive that contains drivers for disks
>>> and the ZFS modules, and then having the kernel load a ramdisk image
>>> from a slice on an NVMe device into RAM.  To do this, I had to add
>>> new hooks into the main() routine of the kernel just prior to
>>> vfs_mountroot() so that we could inject some custom behaviour from a
>>> separate kernel module we build.  It's designed to be customisable
>>> without changing the base kernel source, but also not quite ready to
>>> upstream to mainline illumos -- though I'm keen to get it up there
>>> eventually!
>>>
>>> If you want to get started with a minimal boot-to-ramdisk system, I
>>> would probably look at SmartOS, as that's what it is!  It uses the
>>> UFS ramdisk approach today, and that ramdisk is able to locate other
>>> resources on physical disks as needed, etc.  We also have some tools
>>> the allow building images automatically from templates:
>>>
>>>     https://github.com/illumos/image-builder
>>>
>>> There is no documentation there right now, but I have some examples
>>> of how to use it up here:
>>>
>>>     https://github.com/jclulow/omnios-image-builder
>>>
>>> I have, in my home directory, other templates I'm yet to add there
>>> which can construct a custom OmniOS-based ramdisk image that you can
>>> add software and custom boot-time behaviour to.  If you want me to
>>> throw that up in there, let me know.
>>>
>>> Happy to answer any questions, as I know this is a lot to take in!
>>> It would help to have more concrete specifics about how you're
>>> looking to operate the system you're building, what software you're
>>> hoping to include, how autonomous it should be, how you're hoping to
>>> update it, etc.
>>>
>>>
>>> Cheers.
>>>
>>> --
>>> Joshua M. Clulow
>>> http://blog.sysmgr.org
>>>
>>> ------------------------------------------
>>> illumos: illumos-discuss
>>> Permalink:
>>> https://illumos.topicbox.com/groups/discuss/Tc9bfa679c7294ee7-Ma01a51d0c7fe2e4bc5c02ceb
>>> Delivery options:
>>> https://illumos.topicbox.com/groups/discuss/subscription
>>>
>>
>>
>>
>
>
> *illumos <https://illumos.topicbox.com/latest>* / illumos-discuss / see
> discussions <https://illumos.topicbox.com/groups/discuss> + participants
> <https://illumos.topicbox.com/groups/discuss/members> + delivery options
> <https://illumos.topicbox.com/groups/discuss/subscription> Permalink
> <https://illumos.topicbox.com/groups/discuss/T0d8bd09ba88fa337-M9c78097aa3e6a825ef077f94>
>

[-- Attachment #2: Type: text/html, Size: 15108 bytes --]