Re: [9fans] Announcing: rootless post kernel load startup

Re: [9fans] Announcing: rootless post kernel load startup

[erik quanstrom]

> non-kernel resources. Consequently, if the non-kernel 'root' is lost,
> the dead processes can be killed off, another root acquired, and work
> resumed. If no local resources are available, a srv and mount of
> sources allows access to the tools of the full distribution.

would the dead processes include everything but the boot-time
helpers?  from the perspective of any users of the system, wouldn't
this be equivalent to a reboot?

> As an example of the flexibility of this approach, I have tested this
> scenario: a rootless system starts up with interactive=yes set in the
> plan9.ini. It starts factotum, ipconfig, and then a venti server, and
> then pauses its bootup process. A second machine is then started as
> the fossil file server, and it is instructed to dial the venti and
> begin serving fossil and its normal startup. Once the fossil is
> serving, we return to the machine serving venti, and continue through
> the bootup process, and choose to imitate tcp-booting - and select the
> external fossil for the external service to mount. The fossil is
> dialed and bootup continues as if it was a standard tcp root cpu
> server.  The end result is that even the system hosting the venti is
> still "rooted" via an external fossil which is in turn using the venti
> as a backing store.

have you tested rebooting either machine while the other is
running?  have you tested a three machine scenario?

this reminds me of nfs cross mounting.  how is this different?

- erik

Re: [9fans] Announcing: rootless post kernel load startup

[Iruata Souza]

On Sat, Jan 2, 2010 at 1:56 PM, mycroftiv 9gridchan
<mycroftiv@sphericalharmony.com> wrote:
> More recent work - available on sources in
> contrib/mycroftiv/rootlessboot as both patches and a compiled ready to
> use kernel and optional rootfs.tgz additional tools to be placed in
> 9fat partition.
>
> "Rootless" bootup is a rewrite of the post-kernel load bootup process
> as handled by boot and init. The motivation is severalfold: the basic
> nature of the Plan 9 kernel as a 9p multiplexer means that there is no
> inherent reason to make any given filesystem special and necessary to
> keep the system usable. The particular case of a tcp-booted cpu server
> losing its network root fs is a traditional example. It is sometimes
> the case that a system with local fossil and remote venti can't be
> successfully booted if the IP address of the venti has changed from
> that given in plan9.ini. The overall goal is to improve reliability by
> enabling a running system to always repair and recover itself even if
> it experiences failure of normally criticial resources like a local
> venti+fossil.
>
> The rootless boot method combines boot and init into a much smaller
> program that does much less, and adds the rc shell and a few other
> tools to the kernel's compiled-in /boot. An rc script called plan9rc
> handles the tasks of starting up factotum, ipconfig, venti, fossil,
> and setting up a desired initial filesystem mount and running
> additonal options scripts and/or conventional cpurc or termrc from an
> external fs. Depending on the settings in plan9.ini, this can all be
> done interactively with the ability to drop to an rc shell. The
> plan9rc script can act to imitate the standard bootup process, or
> setup the environment in a different way - for instance by creating a
> ramfs with additional tools loaded from 9fat to create a working
> environment independent of the environment created by the standard
> bootup.
>
> As an example of the flexibility of this approach, I have tested this
> scenario: a rootless system starts up with interactive=yes set in the
> plan9.ini. It starts factotum, ipconfig, and then a venti server, and
> then pauses its bootup process. A second machine is then started as
> the fossil file server, and it is instructed to dial the venti and
> begin serving fossil and its normal startup. Once the fossil is
> serving, we return to the machine serving venti, and continue through
> the bootup process, and choose to imitate tcp-booting - and select the
> external fossil for the external service to mount. The fossil is
> dialed and bootup continues as if it was a standard tcp root cpu
> server.  The end result is that even the system hosting the venti is
> still "rooted" via an external fossil which is in turn using the venti
> as a backing store. This sequence would not be possible using the
> standard bootup method. Rootless booting also enables tcp booted cpu
> servers to be recovered without rebooting. The initial console process
> is usually kept in a protected namespace, with no connections to
> non-kernel resources. Consequently, if the non-kernel 'root' is lost,
> the dead processes can be killed off, another root acquired, and work
> resumed. If no local resources are available, a srv and mount of
> sources allows access to the tools of the full distribution.
>
> contrib/mycroftiv/rootlessboot holds the source for the modifications,
> as well as a compiled kernel. Also available is a rootfs.tgz for use
> in creating an optional ramfs based independent environment. If you
> wish to use this, put it in your 9fat. There is a lot of flexibility
> in exactly what is compiled into /boot and whether or not a ramfs with
> additonal tools loaded. Some users may prefer a light kernel that
> acquires a relatively large rootfs.tgz, whereas others might prefer to
> make a heaver compiled in /boot. The sample kernel on sources and
> rootfs.tgz there are both fairly heavy to try to support most common
> possibilities - venti, fossil, kfs, cfs are all compiled in, for
> instance.
>
> The rootlessboot directory has a fair amount of small modifications
> that are necessary or helpful, such as changing rc to look for
> /boot/rcmain rather than /lib/rcmain, and giving cfs the ability to
> post a srv. contrib/mycroftiv/rootlessboot/bootdir.extras/plan9rc
> controls the boot process and provides many options for controlling
> its behavior via plan9.ini variables. The default behavior is intended
> to closely mimic the standard existing logic for handling plan9.ini.
> The prebuilt kernel is a cpu kernel but setting service=terminal in
> plan9.ini will produce terminal style behavior. As always, I would
> recommend using a plan9.ini menu or setup so that choice of kernel
> used is selectable at boot.
>
> I have read the paper on the 9null kernel load boot process from this
> year's IWP9 and I am hoping that this work is consistent with its
> approach. As I have been working only on the post kernel load actions,
> I believe it to be mostly non-overlapping. This is still very much a
> work-in-progress, although I am using this boot process on my machines
> and VMs and am happy with the additional flexibility and control.
> These patches are most relevant and useful to those working with
> multiple system/VM Plan 9 installations. Although this modified kernel
> can be used to create the same type of base environment as usual, the
> usage model of booting a minimal environment that can be used as a cpu
> server that then creates several externally rooted subenvironments is
> very compelling. A set of provided scripts is intended to make working
> in this manner easier. 'rerootwin' for instance allows you to keep
> your current drawterm connection intact within a window even when
> completely restructuring your namespace to use a different root
> fileserver.
>
> mycroftiv@sphericalharmony.com
> Ben Kidwell
> 9gridchan.org provides a variety of public plan 9 services
> project channel: #plan9chan on irc.freenode.net for 9gridchan
> also in #plan9 for general Plan 9 discussion
>
>

hello,

i guess the work i'm doing overlaps at some extent to yours. we should
definitely talk (off list, perhaps).
i rewrote /boot/boot as a set of rc scripts with a minimal rootfs.tgz.
no big changes were needed.
if you want to take a look http://src.oitobits.net/9null contains the stuff.

thanks,
iru

[9fans] Announcing: rootless post kernel load startup

[mycroftiv 9gridchan]

>would the dead processes include everything but the boot-time
>helpers?  from the perspective of any users of the system, wouldn't
>this be equivalent to a reboot?

>have you tested rebooting either machine while the other is
>running?  have you tested a three machine scenario?
>this reminds me of nfs cross mounting.  how is this different?

Thanks for your interest. Certainly, if the file descriptors being
used by your active applications die, those apps are pretty much dead.
The question of why the scenario is an improvement on rebooting
depends on the specifics - one answer is that in the usage model I
find most optimal, I like to base what I regard as the most essential
services, including the cpu -R listener, either out of boot or a
ramfs, so the system remains available for remote dial-in and
administration. I find that being able to drawterm in, mount sources
for access to the full disttribution, and then work to reset/repair
and environment is a lot more pleasant than plugging in a monitor to a
box I prefer to run headless and then rebooting and hoping that the
issues that forced the reboot (such as unavailable network resources)
are in good enough shape to get things put back together.

The example I gave of starting venti then rooting from a remote fossil
using that venti isn't a particularly important specifically, I simply
wanted to illustrate the fact that moving to a more interactive and
configurable post kernel load boot sequence creates additional
possibilities. The initial motivation was simply that with a grid that
uses separate venti, fossil, and cpu, the imposed boot order
dependency and the fact that the loss of one server could make the
others later in the chain unusable even though they still had running
kernels and control of the hardware seemed suboptimal. I love the
ability to distribute system components, but I wanted to be able to
keep interacting with my cpu server even if I needed to reboot the
fossil.

However, I don't want to seem defensive, and these bootup modification
patches are not anything I want to try to claim as the best possible
system. Just on the technical level the plan9rc I'm using doesn't
cover 100% of the cases handled by the standard tools - it assumes the
only types of disk drives in the world are sdC0, C1, D0, D1 at the
moment, for instance.

~mycroftiv

[9fans] Announcing: rootless post kernel load startup

[mycroftiv 9gridchan]

More recent work - available on sources in
contrib/mycroftiv/rootlessboot as both patches and a compiled ready to
use kernel and optional rootfs.tgz additional tools to be placed in
9fat partition.

"Rootless" bootup is a rewrite of the post-kernel load bootup process
as handled by boot and init. The motivation is severalfold: the basic
nature of the Plan 9 kernel as a 9p multiplexer means that there is no
inherent reason to make any given filesystem special and necessary to
keep the system usable. The particular case of a tcp-booted cpu server
losing its network root fs is a traditional example. It is sometimes
the case that a system with local fossil and remote venti can't be
successfully booted if the IP address of the venti has changed from
that given in plan9.ini. The overall goal is to improve reliability by
enabling a running system to always repair and recover itself even if
it experiences failure of normally criticial resources like a local
venti+fossil.

The rootless boot method combines boot and init into a much smaller
program that does much less, and adds the rc shell and a few other
tools to the kernel's compiled-in /boot. An rc script called plan9rc
handles the tasks of starting up factotum, ipconfig, venti, fossil,
and setting up a desired initial filesystem mount and running
additonal options scripts and/or conventional cpurc or termrc from an
external fs. Depending on the settings in plan9.ini, this can all be
done interactively with the ability to drop to an rc shell. The
plan9rc script can act to imitate the standard bootup process, or
setup the environment in a different way - for instance by creating a
ramfs with additional tools loaded from 9fat to create a working
environment independent of the environment created by the standard
bootup.

As an example of the flexibility of this approach, I have tested this
scenario: a rootless system starts up with interactive=yes set in the
plan9.ini. It starts factotum, ipconfig, and then a venti server, and
then pauses its bootup process. A second machine is then started as
the fossil file server, and it is instructed to dial the venti and
begin serving fossil and its normal startup. Once the fossil is
serving, we return to the machine serving venti, and continue through
the bootup process, and choose to imitate tcp-booting - and select the
external fossil for the external service to mount. The fossil is
dialed and bootup continues as if it was a standard tcp root cpu
server.  The end result is that even the system hosting the venti is
still "rooted" via an external fossil which is in turn using the venti
as a backing store. This sequence would not be possible using the
standard bootup method. Rootless booting also enables tcp booted cpu
servers to be recovered without rebooting. The initial console process
is usually kept in a protected namespace, with no connections to
non-kernel resources. Consequently, if the non-kernel 'root' is lost,
the dead processes can be killed off, another root acquired, and work
resumed. If no local resources are available, a srv and mount of
sources allows access to the tools of the full distribution.

contrib/mycroftiv/rootlessboot holds the source for the modifications,
as well as a compiled kernel. Also available is a rootfs.tgz for use
in creating an optional ramfs based independent environment. If you
wish to use this, put it in your 9fat. There is a lot of flexibility
in exactly what is compiled into /boot and whether or not a ramfs with
additonal tools loaded. Some users may prefer a light kernel that
acquires a relatively large rootfs.tgz, whereas others might prefer to
make a heaver compiled in /boot. The sample kernel on sources and
rootfs.tgz there are both fairly heavy to try to support most common
possibilities - venti, fossil, kfs, cfs are all compiled in, for
instance.

The rootlessboot directory has a fair amount of small modifications
that are necessary or helpful, such as changing rc to look for
/boot/rcmain rather than /lib/rcmain, and giving cfs the ability to
post a srv. contrib/mycroftiv/rootlessboot/bootdir.extras/plan9rc
controls the boot process and provides many options for controlling
its behavior via plan9.ini variables. The default behavior is intended
to closely mimic the standard existing logic for handling plan9.ini.
The prebuilt kernel is a cpu kernel but setting service=terminal in
plan9.ini will produce terminal style behavior. As always, I would
recommend using a plan9.ini menu or setup so that choice of kernel
used is selectable at boot.

I have read the paper on the 9null kernel load boot process from this
year's IWP9 and I am hoping that this work is consistent with its
approach. As I have been working only on the post kernel load actions,
I believe it to be mostly non-overlapping. This is still very much a
work-in-progress, although I am using this boot process on my machines
and VMs and am happy with the additional flexibility and control.
These patches are most relevant and useful to those working with
multiple system/VM Plan 9 installations. Although this modified kernel
can be used to create the same type of base environment as usual, the
usage model of booting a minimal environment that can be used as a cpu
server that then creates several externally rooted subenvironments is
very compelling. A set of provided scripts is intended to make working
in this manner easier. 'rerootwin' for instance allows you to keep
your current drawterm connection intact within a window even when
completely restructuring your namespace to use a different root
fileserver.

mycroftiv@sphericalharmony.com
Ben Kidwell
9gridchan.org provides a variety of public plan 9 services
project channel: #plan9chan on irc.freenode.net for 9gridchan
also in #plan9 for general Plan 9 discussion