* [TUHS] Unix install & "standalone" package @ 2023-09-04 14:44 Norman Wilson 2023-09-04 14:55 ` [TUHS] " Vincenzo Nicosia ` (3 more replies) 0 siblings, 4 replies; 27+ messages in thread From: Norman Wilson @ 2023-09-04 14:44 UTC (permalink / raw) To: tuhs I don't remember any special many-programs-in-one binary like busybox in any Unix from the days when Unix was simple enough for me to understand. That covers the entire lifetime of the Research systems, but also System V and the BSDs and their sundry offspring up into at least the 1990s. I'm pretty sure OpenBSD at least still has nothing like busybox. The nearest thing was to make sure certain programs were linked statically (or existed in alternate statically- linked versions) so they would work before shared libraries were available. (It seems to be common wisdom that `sbin' means `system bin' these days, but I remember once, long ago, being told it stood for `static bin'.) Perhaps the question to ask is why such a magic program is needed at all. Is it just because programs like the shell have become so large and unwieldy that they won't fit in a small environment suitable for loading into an initramfs? Older UNIXes, even on the VAX, didn't use an initramfs. the boot code had just enough understanding of devices and file systems to load the kernel and to point it at the real root file system. The VAX hardware designers had a clever scheme on many (but, strangely, not all) VAX variants by which the hardware had several little boot ROMs, each containing a bare-bones read-only device driver for a particular device along with a few instructions to read the first sector into memory and start it, with pointers to the boot-rom driver and device ID in specified registers. That was enough to support a device-independent Unix boot block that could read unix (or another file name typed on the console) from the root of the file system at the start of the disk. I know it was because I wrote such a boot block, though I don't know whether anyone else did. (Other systems, I think, just used it to load /boot, a larger and more-capable program.) Maybe it was just that the boot environment was simpler in older systems, without the need to load kernel modules or support multiple locations and means of access for the root? Norman Wilson Toronto ON ^ permalink raw reply [flat|nested] 27+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-04 14:44 [TUHS] Unix install & "standalone" package Norman Wilson @ 2023-09-04 14:55 ` Vincenzo Nicosia 2023-09-04 17:20 ` Warner Losh 2023-09-04 17:18 ` Warner Losh ` (2 subsequent siblings) 3 siblings, 1 reply; 27+ messages in thread From: Vincenzo Nicosia @ 2023-09-04 14:55 UTC (permalink / raw) To: tuhs rather, back then you didn't need the same kernel to run on a wide variety of hardware, with all the possible different combinations of peripherals, requiring all sorts of different drivers. I think that's the only real reason why initramfs came to existence: allowing a selection of kernel modules to be loaded at init time, depending on the hardware at disposal on that machine. Then things went south, and more recent initramfs have everything and the kitchen sink. But that's another story. HND Enzo Nicosia -- ^ permalink raw reply [flat|nested] 27+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-04 14:55 ` [TUHS] " Vincenzo Nicosia @ 2023-09-04 17:20 ` Warner Losh 2023-09-04 19:05 ` Clem Cole 2023-09-04 19:59 ` Theodore Ts'o 0 siblings, 2 replies; 27+ messages in thread From: Warner Losh @ 2023-09-04 17:20 UTC (permalink / raw) To: Vincenzo Nicosia; +Cc: tuhs [-- Attachment #1: Type: text/plain, Size: 1152 bytes --] On Mon, Sep 4, 2023 at 8:56 AM Vincenzo Nicosia <katolaz@freaknet.org> wrote: > rather, back then you didn't need the same kernel to run on a wide > variety of hardware, with all the possible different combinations of > peripherals, requiring all sorts of different drivers. > > I think that's the only real reason why initramfs came to existence: > allowing a selection of kernel modules to be loaded at init time, > depending on the hardware at disposal on that machine. Then things went > south, and more recent initramfs have everything and the kitchen sink. > But that's another story. Yea, it was an effort to move mounting of root out of the kernel. The earliest scripts just mounted the right disk and moved on, and didn't load any new drivers: they just had the logic to pick the desired root. But at the same time, there were a lot of people that were running on 4MB and 8MB systems that noticed they could put all the router software in the initramfs and never pivot to something else and they could have quite the product with that. And those were the first few bricks that paved the road to hell :) Warner [-- Attachment #2: Type: text/html, Size: 1584 bytes --] ^ permalink raw reply [flat|nested] 27+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-04 17:20 ` Warner Losh @ 2023-09-04 19:05 ` Clem Cole 2023-09-05 17:03 ` Paul Winalski 2023-09-04 19:59 ` Theodore Ts'o 1 sibling, 1 reply; 27+ messages in thread From: Clem Cole @ 2023-09-04 19:05 UTC (permalink / raw) To: Warner Losh; +Cc: tuhs [-- Attachment #1.1: Type: text/plain, Size: 4871 bytes --] I can not add too much to what happened later, but I think modern users don't get it why the things like the standalone system were needed back in the day. There were a few simple differences thanks to Moore's law, modern folks really may not grok: 1. The number of peripherals that needed to be supported was small [each firm made only a few for each target system] 2. The capacity/capabilities of those peripherals are minimal compared to today's world 3. Cheap ROMs, as we know it, were coming on the scene [the EPROM is only invented in 1971] One-time programmable and Mask ROMs were around but they were expensive and small. I hope Warren's emailer lets this pass through. Here is a picture of the original root 'ROM' for the PDP-11 M792-xx board, which gives you 32 words (64 bytes) of memory. It used individual diodes in or out arranged an array to represent individual bits with a diode present being a '1' and '0' if missing. As the Gunkies web page suggests, the board came in two versions - customer programmable which was fully populated with diodes, and the user removed unneeded diodes to program it. The other variants were programmed with the designed boot functions - designated M792-Yx (where 'x' is a capital letter, starting with 'A') [image: Read_Only_Memory_DEC_M792_Diode_Matrix_ROM_Aceware_Attribution.jpg] Also, remember that a $3-5k/drive RK05 is only 2.5Mbytes [4872 512 byte 'sectors' *a.k.a*. disk blocks -- 2 hd/disk * 15 sec/cyl * 203 cyl/disk], which is why we had separated /bin and /usr/bin and /lib and /usr/lib. Plus, as Norman points our dynamic linking only comes later, so all of these are static bound programs (*i.e.,* Sun create /bin and /sbin to separate those programs). The whole idea was what was in root was enough to get the system running in /bin, /lib and /etc and everything else was one the mounted file system, which often was another RK05. Many of our V6 and V7 systems ran with 3 RK05 /, /usr /home, and we might have an extra RK05 for /mnt. Tape was the standard thing. DEC tape was cheaper than 9-track, although in my history, we all went for 9-track because it was more portable to other systems, but often used after-market 9-track transports that emulated the DEC ones. Even with the VAX, the front-end runs on an LSI-11 with floppies, so things like microcode were loaded into the VAX and stored in ROM. By then, the RP and RM series had become more normal, and eventually, the RK07 and RL02 replaced the RK05. But we are not talking about substantial capacity in the storage systems. Only ten years later, in the early 1980s, a Fujisu 'Eagle' is just 470Mbytes and costs 12-15K per drive, plus another $6-8K for the controller for your Vax [also often after-market from SI or Emulex]. Sun and Apollo build 'diskless' systems because a 100MB ST-506 style disk was considered too expensive, and was worried about the entry-level price of their systems (I famously made a typo describing how those systems performed BTW with a dyslexic change of the s to c on a whole company email at Masscomp in the mid-1980s - to Sun's credit, it was the best marketing ploy ever - people bought diskless and discovered they sucked and then all added a disk - which cost more $s than the original Masscomp system which was had one built-in). But I digress... Those new workstations did have real ROM chips, unlike the original PDP-11s, but they were small, maybe 8Kbytes of NS2764, so the ROMs were tight and only had enough in them to recognize a couple of peripherals. So to answer more of your question, you build the tools you need that make sense at the time. /stand was a simple solution and was small. For V7 and the original BSD 3 and 4 released, the user needed to read a distribution media, traditionally a 9-track tape, and set up a local disk. The number of peripherals was small. One other thought ... in the case of the PDP-11 and Vaxen, the disks were often dismountable, like the different RK, RL, RP, and RM series drives. We often had more than one system. So once the system was up and running, it was not usual to use a different system with better tools to help repair things if bad stuff occurred. That is also why we often used 9-track tape, just because that was even easier to move back and forth. With the Wokstrations, the disks were generally sealed. Clem A small PS on another note -- I have memories of taking an ST-506 disk with me to France to work with some engineers in Grenoble and explaining to the French authorities that they could not open it up to look inside. BTW: this pre-internet - imagine what we had to do if we flew to Canada, which was taxing SW. Even bringing tapes in and out of Toronto airport could be treacherous. ᐧ [-- Attachment #1.2: Type: text/html, Size: 5769 bytes --] [-- Attachment #2: Read_Only_Memory_DEC_M792_Diode_Matrix_ROM_Aceware_Attribution.jpg --] [-- Type: image/jpeg, Size: 154411 bytes --] ^ permalink raw reply [flat|nested] 27+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-04 19:05 ` Clem Cole @ 2023-09-05 17:03 ` Paul Winalski 2023-09-05 18:02 ` Clem Cole 0 siblings, 1 reply; 27+ messages in thread From: Paul Winalski @ 2023-09-05 17:03 UTC (permalink / raw) To: Clem Cole; +Cc: tuhs On 9/4/23, Clem Cole <clemc@ccc.com> wrote: > > Even with the VAX, the front-end runs on an LSI-11 with floppies, so things > like microcode were loaded into the VAX and stored in ROM. I think you meant either "stored in RAM" or "not stored in ROM". The LSI-11 front end for the VAX-11/780 (and other early VAXen?) was connected to the SBI (Synchronous Backplane Interconnect--the system bus) and performed three main functions: microcode load and system bootstrap initiation, interface for the system console (originally a LA-36 dot matrix terminal), and some system diagnostics. CPU microcode was stored on a 5" floppy disk connected to the LSI-11 and was loaded into RAM at system power-up. -Paul W. ^ permalink raw reply [flat|nested] 27+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-05 17:03 ` Paul Winalski @ 2023-09-05 18:02 ` Clem Cole 0 siblings, 0 replies; 27+ messages in thread From: Clem Cole @ 2023-09-05 18:02 UTC (permalink / raw) To: Paul Winalski; +Cc: tuhs [-- Attachment #1: Type: text/plain, Size: 398 bytes --] On Tue, Sep 5, 2023 at 1:03 PM Paul Winalski <paul.winalski@gmail.com> wrote: > On 9/4/23, Clem Cole <clemc@ccc.com> wrote: > > > > Even with the VAX, the front-end runs on an LSI-11 with floppies, so > things > > like microcode were loaded into the VAX and stored in ROM. > > I think you meant either "stored in RAM" or "not stored in ROM". Yep -- thanks - dyslexia-r-me. ᐧ [-- Attachment #2: Type: text/html, Size: 1330 bytes --] ^ permalink raw reply [flat|nested] 27+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-04 17:20 ` Warner Losh 2023-09-04 19:05 ` Clem Cole @ 2023-09-04 19:59 ` Theodore Ts'o 2023-09-04 23:51 ` Warner Losh 1 sibling, 1 reply; 27+ messages in thread From: Theodore Ts'o @ 2023-09-04 19:59 UTC (permalink / raw) To: Warner Losh; +Cc: tuhs On Mon, Sep 04, 2023 at 11:20:31AM -0600, Warner Losh wrote: > > Yea, it was an effort to move mounting of root out of the > kernel. The earliest scripts just mounted the right disk and moved > on, and didn't load any new drivers: they just had the logic to pick > the desired root. But at the same time, there were a lot of people > that were running on 4MB and 8MB systems that noticed they could put > all the router software in the initramfs and never pivot to > something else and they could have quite the product with that. And > those were the first few bricks that paved the road to hell :) The other reason why you might need something like an initial ramdisk is if you don't yet *have* a root file system on the hardware, and you are trying to install a system using TFTP boot. At that point, you have three choices: (a) Use a NFS root (b) Use a read-only network block device (for example, MIT Project Athena had a read-only Remote Virtual Disk which was a network block device that was used to provide a read-only system image used for installation and update, as well as a read-only /usr image). (c) Use a built-in ramdisk. (And here's not just Linus which has adopted this strategy; OpenBSD's installer does this as well.) The other dynamic at play was that ramdisks were needed when you were installing on a 386 PC with, say, 16 megs of memory, a 320 megabyte hard disk, and a *single* 1.44MB floppy --- and no ethernet, because this was for the home PC user, so the best that you had might be a 38kbps dialup link with PPP --- if you were lucky. So putting the kernel on the first floppy disk, and putting a ramdisk image on the second floppy disk, so you could then eject the floppy disk and insert subsequent floppy disks so you could install rest of the installation binaries (including the shell utilities, the C compiler, X windows systems, etc.) and then reboot onto the HDD with a fully set up system, was a pretty natural evolution. And since we had the ramdisk infrastructure, and most users didn't want to configure their own kernel as part of the installation process, that begat kernel modules and the initial ramdisk being used to store the kernel modules. In the very early days, this made Linux *far* more user friendly for non-system programmers to install, compared to say, FreeBSD during that era, which was still stuck in the BSD 4.x days of a generic kernel, followed by a kernel configuration step, etc. These days, given that some enterprise setups what their servers to be installed from a Fibre Channel Storage Area Network, or iSCSI, perhaps using Kerberos to authenticate to the iSCSI target, it makes absolutely a huge amount of sense to have an initial ramdisk as an option. That being said, for a long time, the initial ramdisk was an *option*. You didn't have to use it, if you were willing to have a custom kernel and you are using a device which has a fixed and stable boot-time enumeration. This tended to only be used by people who knew what they were doing, but for example, my file system test appliance VM system doesn't use an initial ramdisk at all. It uses qemu, and has a built-in kernel configuration and build system, and this works perfectly fine booting into the latest Debian stable distribution, with no initial ramdisk used at all: https://github.com/tytso/xfstests-bld/blob/master/Documentation/kvm-quickstart.md - Ted the root filesystem on the second floppy disk (the first floppy disk contained the kernel), which was then copied into a ramdisk, so you could Trying to fit the kernel and the root file system on a single 1.44MB floppy, was (barely) doable, but you certainly wouldn't have space for a C compiler so you build or link your own custom kernel. So one of the first things that we did was to put the kernel on a single floppy disk, and then but the root on a single floppy disk, and then copy the root to the HDD, and then reboot onto the HDD, and then take the installation procedure from there. But that was awkward and ^ permalink raw reply [flat|nested] 27+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-04 19:59 ` Theodore Ts'o @ 2023-09-04 23:51 ` Warner Losh 0 siblings, 0 replies; 27+ messages in thread From: Warner Losh @ 2023-09-04 23:51 UTC (permalink / raw) To: Theodore Ts'o; +Cc: tuhs [-- Attachment #1: Type: text/plain, Size: 5093 bytes --] On Mon, Sep 4, 2023 at 1:59 PM Theodore Ts'o <tytso@mit.edu> wrote: > On Mon, Sep 04, 2023 at 11:20:31AM -0600, Warner Losh wrote: > > > > Yea, it was an effort to move mounting of root out of the > > kernel. The earliest scripts just mounted the right disk and moved > > on, and didn't load any new drivers: they just had the logic to pick > > the desired root. But at the same time, there were a lot of people > > that were running on 4MB and 8MB systems that noticed they could put > > all the router software in the initramfs and never pivot to > > something else and they could have quite the product with that. And > > those were the first few bricks that paved the road to hell :) > > The other reason why you might need something like an initial ramdisk > is if you don't yet *have* a root file system on the hardware, and you > are trying to install a system using TFTP boot. At that point, you > have three choices: > > (a) Use a NFS root > > (b) Use a read-only network block device (for example, MIT Project > Athena had a read-only Remote Virtual Disk which was a network block > device that was used to provide a read-only system image used for > installation and update, as well as a read-only /usr image). > > (c) Use a built-in ramdisk. (And here's not just Linus which has > adopted this strategy; OpenBSD's installer does this as well.) > Yes. FreeBSD did this originally, and some installers still do that. > The other dynamic at play was that ramdisks were needed when you were > installing on a 386 PC with, say, 16 megs of memory, a 320 megabyte > hard disk, and a *single* 1.44MB floppy --- and no ethernet, because > this was for the home PC user, so the best that you had might be a > 38kbps dialup link with PPP --- if you were lucky. > > So putting the kernel on the first floppy disk, and putting a ramdisk > image on the second floppy disk, so you could then eject the floppy > disk and insert subsequent floppy disks so you could install rest of > the installation binaries (including the shell utilities, the C > compiler, X windows systems, etc.) and then reboot onto the HDD with a > fully set up system, was a pretty natural evolution. > Yes. FreeBSD 2.0 definitely did that, and I think FreeBSD 1.0 as well. > And since we had the ramdisk infrastructure, and most users didn't > want to configure their own kernel as part of the installation > process, that begat kernel modules and the initial ramdisk being used > to store the kernel modules. In the very early days, this made Linux > *far* more user friendly for non-system programmers to install, > compared to say, FreeBSD during that era, which was still stuck in the > BSD 4.x days of a generic kernel, followed by a kernel configuration > step, etc. > GENERIC worked fine in the early days: it was small enough that the savings from removing all the extra devices was small... but that's because the number of devices supported was also relatively small. By the time it became an issue FreeBSD did have dynamic loading of drivers, but retained the GENERIC kernel for install (it should have gone to a minimal kernel, but hasn't yet completed the migration: the savings wasn't big enough due to Moore's Law growing machine memories faster than the kernel grew: FreeBSD's kernel only doubled every 2 years...). But yea, it was a bit of a kludge. And FreeBSD grew the ability to configure the morass of ISA devices in the boot loader early on so you could at least boot the generic kernel, and have the loader remember your choices. > These days, given that some enterprise setups what their servers to be > installed from a Fibre Channel Storage Area Network, or iSCSI, perhaps > using Kerberos to authenticate to the iSCSI target, it makes > absolutely a huge amount of sense to have an initial ramdisk as an > option. That being said, for a long time, the initial ramdisk was an > *option*. You didn't have to use it, if you were willing to have a > custom kernel and you are using a device which has a fixed and stable > boot-time enumeration. > Yea. FreeBSD can use it to this day (and many interesting use cases use it), but generally the loader tells the kernel what root filesystem to use. These smarts in the boot loader have also kept the pressure of the vast majority of cases where initramfs makes sense in Linux land. > This tended to only be used by people who knew what they were doing, > but for example, my file system test appliance VM system doesn't use > an initial ramdisk at all. It uses qemu, and has a built-in kernel > configuration and build system, and this works perfectly fine booting > into the latest Debian stable distribution, with no initial ramdisk > used at all: > > > https://github.com/tytso/xfstests-bld/blob/master/Documentation/kvm-quickstart.md > Ah, cool. Every single distro I've run in the past has had an initramfs. I thought it had become no longer an option... I do similar things for my VM setup with FreeBSD :) Warner [-- Attachment #2: Type: text/html, Size: 6738 bytes --] ^ permalink raw reply [flat|nested] 27+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-04 14:44 [TUHS] Unix install & "standalone" package Norman Wilson 2023-09-04 14:55 ` [TUHS] " Vincenzo Nicosia @ 2023-09-04 17:18 ` Warner Losh 2023-09-04 22:10 ` Steffen Nurpmeso 2023-09-05 1:07 ` Jonathan Gray 3 siblings, 0 replies; 27+ messages in thread From: Warner Losh @ 2023-09-04 17:18 UTC (permalink / raw) To: Norman Wilson; +Cc: tuhs [-- Attachment #1: Type: text/plain, Size: 2061 bytes --] On Mon, Sep 4, 2023 at 8:44 AM Norman Wilson <norman@oclsc.org> wrote: > Maybe it was just that the boot environment was simpler > in older systems, without the need to load kernel modules > or support multiple locations and means of access for > the root? > Older systems had fewer choices. On the PDP-11 you had Q-Bus or Unibus. You booted off a small selection of disks that converged to MSCP(?), so there was THE boot environment and THE driver and THE filesystem. And the kernels were often tuned to be exactly what the machine needed. The VAX, Sun, hp, ibm, etc continued this early trend. Though the boot process was still fairly narrow, with a limited list of supported boot devices. In large part this was because the machine was exactly the same every time you booted. But with USB, PC Card, CardBus, ExpressCard, SCSI, Thunderbolt, and a host of other removable technologies with a dizzying array of cards, like PCI, ISA, etc, the game changed. Also, the number of environments that a FreeBSD GENERIC[*] kernel can boot in is huge due to combinatoric explosion due to three or four boot environments, vm alternate startup paths, several supported root file system, BIOS, UEFI, OpenFirmware, u-boot, coreboot, etc as well as several thousand supported boot controller devices, things got complex, despite there being several common interfaces that made things simpler. I'm running a summer of code project this year to help tame the combinatoric explosion to provide better test coverage because though attempts were made to make things the same, variations exist that can cause unexpected breakage in different environments. Warner [*] I use GENERIC here as a catch all, including the more recent MINIMAL kernels that try to include just the core functionality, and omit the vast majority of drivers. Linux has similar issues, but more, and also solves them in many interesting ways... But I'm the FreeBSD boot loader guy in large part (though there's others that work on it), not a Linux person. [-- Attachment #2: Type: text/html, Size: 2719 bytes --] ^ permalink raw reply [flat|nested] 27+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-04 14:44 [TUHS] Unix install & "standalone" package Norman Wilson 2023-09-04 14:55 ` [TUHS] " Vincenzo Nicosia 2023-09-04 17:18 ` Warner Losh @ 2023-09-04 22:10 ` Steffen Nurpmeso 2023-09-05 15:53 ` Steffen Nurpmeso 2023-09-05 1:07 ` Jonathan Gray 3 siblings, 1 reply; 27+ messages in thread From: Steffen Nurpmeso @ 2023-09-04 22:10 UTC (permalink / raw) To: norman; +Cc: tuhs Norman Wilson wrote in <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org>: |I don't remember any special many-programs-in-one binary |like busybox in any Unix from the days when Unix was simple |enough for me to understand. That covers the entire lifetime |of the Research systems, but also System V and the BSDs and |their sundry offspring up into at least the 1990s. ... |Perhaps the question to ask is why such a magic program is |needed at all. Is it just because programs like the shell |have become so large and unwieldy that they won't fit in |a small environment suitable for loading into an initramfs? AlpineLinux as used on my vserver has busybox by default and can cover most utitilities like that. The lead developer Copa once said something like "The idea is you install explicitly [if you want something better]". (It is a symlink farm that is selectively replaced by installing "real" packages iirc.) For my laptop it allows me easy boot management. To save you the chatter ("Chatten" is the name of my tribe .. most likely; could be Franken, Sueben .. and you know how it is): this approach is much easier and smaller than having lots of static binaries to copy around etc. I do not use secure boot, i have on EFI only a kernel, busybox and cryptsetup, and scripts (the laptop is named "kent"):: ... drwxr-xr-x 4 root root 4096 Jul 15 2021 EFI/ ... -rwxr-xr-x 1 root root 272 Feb 1 2022 kent.sh* -rwxr-xr-x 1 root root 313 Feb 1 2022 kent-direct.sh* drwxr-xr-x 1 root root 252 Oct 9 2022 ../ -rwxr-xr-x 1 root root 4596 Feb 4 2023 linux-init-s1.sh* -rwxr-xr-x 1 root root 3646 Feb 4 2023 linux-init-lib.sh* -rwxr-xr-x 1 root root 5480120 Feb 11 2023 cryptsetup.static* -rwxr-xr-x 1 root root 1978368 Aug 15 18:51 busybox.static* -rwxr-xr-x 1 root root 10112672 Aug 26 18:44 ideapad-stage1.efi* So kent.sh can be init(8) for the ideapad-stage1.efi Linux kernel started via EFI as setup via efibootmgr(8) Boot0001* kent HD(1,GPT,5d6d756b-5de2-4e5d-b043-8d4ae1bb6eb0,0x800,0x82000)/File(\ideapad-stage1.efi)root=/dev/nvme0n1p1 rootfstype=vfat init=/kent.sh #!/busybox.static sh #@ kent, step 1., via EFI. PART_ROOT=/dev/nvme0n1p8 ROOT_DECRYPT='-t btrfs -o defaults,subvol=/crux/kent/root' PART_ROOT1=/dev/nvme0n1p8 ROOT_DECRYPT1='-t btrfs -o defaults,subvol=/crux/kent/root.old' INIT_S2=/boot/kent-2.sh . /linux-init-s1.sh and that allows me to unlock the harddisk. We then boot via $INIT_S2 and kexec(8) a kernel from the encrypted harddisk, so no code from EFI partition keeps on running. (We byte-compare the data from EFI with equal /boot/ files after booting the real system.) This allows nice and easy properties: only three files to track (cryptsetup, busybox, kernel), almost same set of files in /boot/ and /media/efi aka EFI. And ideapad-stage1.efi is the same kernel that later runs, but later we have also additional dynamic modules available. Ie, every few weeks i copy /boot/ideapad-6_1.efi over to be the new -stage1.efi. --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) ^ permalink raw reply [flat|nested] 27+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-04 22:10 ` Steffen Nurpmeso @ 2023-09-05 15:53 ` Steffen Nurpmeso 2023-09-06 17:50 ` Warner Losh 2023-09-08 13:56 ` Michael Kjörling 0 siblings, 2 replies; 27+ messages in thread From: Steffen Nurpmeso @ 2023-09-05 15:53 UTC (permalink / raw) To: norman; +Cc: tuhs Steffen Nurpmeso wrote in <20230904221059.sF2G0%steffen@sdaoden.eu>: |Norman Wilson wrote in | <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org>: ... ||Perhaps the question to ask is why such a magic program is ||needed at all. Is it just because programs like the shell ||have become so large and unwieldy that they won't fit in ||a small environment suitable for loading into an initramfs? ... |For my laptop it allows me easy boot management. ... | -rwxr-xr-x 1 root root 4596 Feb 4 2023 linux-init-s1.sh* | -rwxr-xr-x 1 root root 3646 Feb 4 2023 linux-init-lib.sh* | -rwxr-xr-x 1 root root 5480120 Feb 11 2023 cryptsetup.static* | -rwxr-xr-x 1 root root 1978368 Aug 15 18:51 busybox.static* | -rwxr-xr-x 1 root root 10112672 Aug 26 18:44 ideapad-stage1.efi* Only to add that this is because of Linux and the way it is doing things. If i would use FreeBSD on bare metal, then i would have an EFI boot loader on EFI that knows (only) enough to ask for passphrase (correct me if i am wrong), and can then boot the kernel from FFS or ZFS. (You have to choose dedicated ZFS boot loader iirc, but despite that...) I know GRUB (and maybe other) Linux bootloaders can do all that, but they are huge, are badly maintained, or under-documented, let alone with local manuals, and i am too stupid to configure them (due to all that). refind is ok, however. But.. be aware of typos in the configuration.. But anyhow. With an EFI_STUB Linux kernel i can save me all that, with busybox i get a complete environment (i then even create an initrd in /boot/ on the fly so i do not have to type the password a second time, that can (optionally) be cached, and is, actually -rw------- 1 root root 4495987 May 29 16:29 .kent.initrd.0 Unfortunately cryptsetup is needed even though, i think, the kernel has anything needed; you just cannot access it. cryptsetup is only needed for "$cs open $PART_ROOT p_root --key-file -". Of course i am no real Linux expert but only a do-it-yourself guy. busybox allows me to manage this easily, to answer your question. --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) ^ permalink raw reply [flat|nested] 27+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-05 15:53 ` Steffen Nurpmeso @ 2023-09-06 17:50 ` Warner Losh 2023-09-07 0:11 ` Steffen Nurpmeso 2023-09-08 13:56 ` Michael Kjörling 1 sibling, 1 reply; 27+ messages in thread From: Warner Losh @ 2023-09-06 17:50 UTC (permalink / raw) To: Norman Wilson, tuhs [-- Attachment #1: Type: text/plain, Size: 3147 bytes --] On Tue, Sep 5, 2023 at 9:53 AM Steffen Nurpmeso <steffen@sdaoden.eu> wrote: > Steffen Nurpmeso wrote in > <20230904221059.sF2G0%steffen@sdaoden.eu>: > |Norman Wilson wrote in > | <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org>: > ... > ||Perhaps the question to ask is why such a magic program is > ||needed at all. Is it just because programs like the shell > ||have become so large and unwieldy that they won't fit in > ||a small environment suitable for loading into an initramfs? > ... > |For my laptop it allows me easy boot management. > ... > | -rwxr-xr-x 1 root root 4596 Feb 4 2023 linux-init-s1.sh* > | -rwxr-xr-x 1 root root 3646 Feb 4 2023 linux-init-lib.sh* > | -rwxr-xr-x 1 root root 5480120 Feb 11 2023 cryptsetup.static* > | -rwxr-xr-x 1 root root 1978368 Aug 15 18:51 busybox.static* > | -rwxr-xr-x 1 root root 10112672 Aug 26 18:44 ideapad-stage1.efi* > > Only to add that this is because of Linux and the way it is doing > things. If i would use FreeBSD on bare metal, then i would have > an EFI boot loader on EFI that knows (only) enough to ask for > passphrase (correct me if i am wrong), and can then boot the > kernel from FFS or ZFS. (You have to choose dedicated ZFS boot > loader iirc, but despite that...) > No, you don't have to choose the dedicated ZFS boot loader, at least not anymore. Also, you can use boot1.efi to load loader.efi from the root filesystem to load the kernel, or you could use loader.efi directly on the ESP to load the kernel. boot1 barely knows anything (and has only one choice of what to boot). loader.efi is the full deal, and can do rather a lot of sophisticated things. > I know GRUB (and maybe other) Linux bootloaders can do all that, > but they are huge, are badly maintained, or under-documented, let > alone with local manuals, and i am too stupid to configure them > (due to all that). refind is ok, however. But.. be aware of > typos in the configuration.. > > But anyhow. With an EFI_STUB Linux kernel i can save me all that, > with busybox i get a complete environment (i then even create an > initrd in /boot/ on the fly so i do not have to type the password > a second time, that can (optionally) be cached, and is, actually > > -rw------- 1 root root 4495987 May 29 16:29 .kent.initrd.0 > > Unfortunately cryptsetup is needed even though, i think, the > kernel has anything needed; you just cannot access it. cryptsetup > is only needed for "$cs open $PART_ROOT p_root --key-file -". > Of course i am no real Linux expert but only a do-it-yourself guy. > busybox allows me to manage this easily, to answer your question. > You could do that on FreeBSD with a loader.efi that has a ram disk built into it as well, including a 'beastie box' thing that's akin to busybox. It will boot in one step and no no further I/O to get a running system. Others have used this for secure boot and to boot a small ram disk that's later discarded as userland decides what root should be. But it's much less automated than in Linux... Warner [-- Attachment #2: Type: text/html, Size: 4136 bytes --] ^ permalink raw reply [flat|nested] 27+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-06 17:50 ` Warner Losh @ 2023-09-07 0:11 ` Steffen Nurpmeso 2023-09-07 16:05 ` Warner Losh 0 siblings, 1 reply; 27+ messages in thread From: Steffen Nurpmeso @ 2023-09-07 0:11 UTC (permalink / raw) To: Warner Losh; +Cc: tuhs Hello! Warner Losh wrote in <CANCZdfoy0UJyFdvJ_fO4FDks_dfEOhsJo+7p2RORdk6gQWf7qg@mail.gmail.com>: |On Tue, Sep 5, 2023 at 9:53 AM Steffen Nurpmeso <steffen@sdaoden.eu> wrote: |> Steffen Nurpmeso wrote in |> <20230904221059.sF2G0%steffen@sdaoden.eu>: |>|Norman Wilson wrote in |>| <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org>: |> ... |>||Perhaps the question to ask is why such a magic program is |>||needed at all. Is it just because programs like the shell |>||have become so large and unwieldy that they won't fit in |>||a small environment suitable for loading into an initramfs? |> ... |>|For my laptop it allows me easy boot management. ... |> Only to add that this is because of Linux and the way it is doing |> things. If i would use FreeBSD on bare metal, then i would have |> an EFI boot loader on EFI that knows (only) enough to ask for |> passphrase (correct me if i am wrong), and can then boot the |> kernel from FFS or ZFS. (You have to choose dedicated ZFS boot |> loader iirc, but despite that...) | |No, you don't have to choose the dedicated ZFS boot loader, at least not |anymore. Ah! It seems regarding FreeBSD i am stuck in BIOS world, then. (zfsboot and gptzfsboot.) |Also, you can use boot1.efi to load loader.efi from the root filesystem to The former deprecated i see. |load the kernel, or you could use loader.efi directly on the ESP to load |the kernel. boot1 barely knows anything (and has only one choice of |what to boot). loader.efi is the full deal, and can do rather a lot of |sophisticated things. Great. ... |> But anyhow. With an EFI_STUB Linux kernel i can save me all that, |> with busybox i get a complete environment (i then even create an |> initrd in /boot/ on the fly so i do not have to type the password |> a second time, that can (optionally) be cached, and is, actually ... |> Unfortunately cryptsetup is needed even though, i think, the |> kernel has anything needed; you just cannot access it. cryptsetup |> is only needed for "$cs open $PART_ROOT p_root --key-file -". |> Of course i am no real Linux expert but only a do-it-yourself guy. Well, and user space only, and never having read BIOS or UEFI standards / implementations. I have reread (almost) all of FreeBSDs documentation (loader, uefi etc) today, and it is always great to see such good documentation! Of course some things are beyond what a "normal" user would understand, but at least there is documentation available. That is really great. |> busybox allows me to manage this easily, to answer your question. | |You could do that on FreeBSD with a loader.efi that has a ram disk |built into it as well, including a 'beastie box' thing that's akin to |busybox. |It will boot in one step and no no further I/O to get a running system. |Others have used this for secure boot and to boot a small ram disk that's |later discarded as userland decides what root should be. But it's much less |automated than in Linux... Hm, but wait. Isn't it easier for FreeBSD? I must admit i never used FreeBSD/geli on bare metal, i started using fully encrypted hard disks in mid 2021, no sooner. So when i read [1] it seems FreeBSD's boot procedure is so nicely linked with geli that the boot loader asks itself for the password of the encrypted partition, simply by setting some variables in loader.conf? I mean, the /boot/ must be available? [1] https://man.freebsd.org/cgi/man.cgi?query=geli&apropos=0&sektion=0&manpath=FreeBSD+13.2-RELEASE+and+Ports&arch=default&format=html These simple scripts i use for Linux also do not support true suspend, only suspend-to-ram. Ie, i have no idea how i could "suspend" the encrypted device and then "resume" it again; The "warm" security is only (initiated via ACPI event from root account) auto-unload of SSH and PGP keys etc, unload of encfs fuse volumes, and then slock of all X displays etc. Ie, all programs which run live on the encrypted volume. When the laptop wakes up, X is running etc. One would need the possibility to wake up to "a password prompt". Well, likely it would work to mount EFI vfat before the suspend, create a console and auto-switch to it, and then sit there, ready for reading in the password once the system resumes. That is to say: i always believed in FreeBSD this works just as easy as the boot procedure? --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) ^ permalink raw reply [flat|nested] 27+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-07 0:11 ` Steffen Nurpmeso @ 2023-09-07 16:05 ` Warner Losh 2023-09-08 14:58 ` Theodore Ts'o 0 siblings, 1 reply; 27+ messages in thread From: Warner Losh @ 2023-09-07 16:05 UTC (permalink / raw) To: Warner Losh, Norman Wilson, tuhs [-- Attachment #1: Type: text/plain, Size: 5531 bytes --] On Wed, Sep 6, 2023 at 6:12 PM Steffen Nurpmeso <steffen@sdaoden.eu> wrote: > Hello! > > Warner Losh wrote in > <CANCZdfoy0UJyFdvJ_fO4FDks_dfEOhsJo+7p2RORdk6gQWf7qg@mail.gmail.com>: > |On Tue, Sep 5, 2023 at 9:53 AM Steffen Nurpmeso <steffen@sdaoden.eu> > wrote: > |> Steffen Nurpmeso wrote in > |> <20230904221059.sF2G0%steffen@sdaoden.eu>: > |>|Norman Wilson wrote in > |>| <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org>: > |> ... > |>||Perhaps the question to ask is why such a magic program is > |>||needed at all. Is it just because programs like the shell > |>||have become so large and unwieldy that they won't fit in > |>||a small environment suitable for loading into an initramfs? > |> ... > |>|For my laptop it allows me easy boot management. > ... > |> Only to add that this is because of Linux and the way it is doing > |> things. If i would use FreeBSD on bare metal, then i would have > |> an EFI boot loader on EFI that knows (only) enough to ask for > |> passphrase (correct me if i am wrong), and can then boot the > |> kernel from FFS or ZFS. (You have to choose dedicated ZFS boot > |> loader iirc, but despite that...) > | > |No, you don't have to choose the dedicated ZFS boot loader, at least not > |anymore. > > Ah! It seems regarding FreeBSD i am stuck in BIOS world, then. > (zfsboot and gptzfsboot.) > Yes, for MBR, things are necessarily more complicated because the ZFS boot loader is too big. The first few sectors live in the 8k space that the traditional boot loader lives, and the rest is DD'd to a special area of the ZFS uberblock reserved for boot blocks. For gptboot vs gptzfsboot, they were written to be either or and combining them presented challenges, not least was the partition size that gptboot lives in was created for many releases too small to hold gptzfsboot and making it tricky to upgrade... > |Also, you can use boot1.efi to load loader.efi from the root filesystem > to > > The former deprecated i see. > > |load the kernel, or you could use loader.efi directly on the ESP to load > |the kernel. boot1 barely knows anything (and has only one choice of > |what to boot). loader.efi is the full deal, and can do rather a lot of > |sophisticated things. > > Great. > > ... > |> But anyhow. With an EFI_STUB Linux kernel i can save me all that, > |> with busybox i get a complete environment (i then even create an > |> initrd in /boot/ on the fly so i do not have to type the password > |> a second time, that can (optionally) be cached, and is, actually > ... > |> Unfortunately cryptsetup is needed even though, i think, the > |> kernel has anything needed; you just cannot access it. cryptsetup > |> is only needed for "$cs open $PART_ROOT p_root --key-file -". > |> Of course i am no real Linux expert but only a do-it-yourself guy. > > Well, and user space only, and never having read BIOS or UEFI > standards / implementations. > I have reread (almost) all of FreeBSDs documentation (loader, uefi > etc) today, and it is always great to see such good documentation! > Of course some things are beyond what a "normal" user would > understand, but at least there is documentation available. > That is really great. > > |> busybox allows me to manage this easily, to answer your question. > | > |You could do that on FreeBSD with a loader.efi that has a ram disk > |built into it as well, including a 'beastie box' thing that's akin to > |busybox. > |It will boot in one step and no no further I/O to get a running system. > |Others have used this for secure boot and to boot a small ram disk that's > |later discarded as userland decides what root should be. But it's much > less > |automated than in Linux... > > Hm, but wait. Isn't it easier for FreeBSD? I must admit i never > used FreeBSD/geli on bare metal, i started using fully encrypted > hard disks in mid 2021, no sooner. So when i read [1] it seems > FreeBSD's boot procedure is so nicely linked with geli that the > boot loader asks itself for the password of the encrypted > partition, simply by setting some variables in loader.conf? > I mean, the /boot/ must be available? > > [1] > https://man.freebsd.org/cgi/man.cgi?query=geli&apropos=0&sektion=0&manpath=FreeBSD+13.2-RELEASE+and+Ports&arch=default&format=html > > These simple scripts i use for Linux also do not support true > suspend, only suspend-to-ram. Ie, i have no idea how i could > "suspend" the encrypted device and then "resume" it again; > The "warm" security is only (initiated via ACPI event from root > account) auto-unload of SSH and PGP keys etc, unload of encfs fuse > volumes, and then slock of all X displays etc. Ie, all programs > which run live on the encrypted volume. When the laptop wakes up, > X is running etc. > One would need the possibility to wake up to "a password prompt". > Well, likely it would work to mount EFI vfat before the suspend, > create a console and auto-switch to it, and then sit there, ready > for reading in the password once the system resumes. That is to > say: i always believed in FreeBSD this works just as easy as the > boot procedure? > Yea. I find it easier :). But there's some better automation available in Linux for the 'complicated' situations. I was also trying to be generous to Linux because I know the passion of its supporters in the face of even mild criticism. Warner [-- Attachment #2: Type: text/html, Size: 7144 bytes --] ^ permalink raw reply [flat|nested] 27+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-07 16:05 ` Warner Losh @ 2023-09-08 14:58 ` Theodore Ts'o 0 siblings, 0 replies; 27+ messages in thread From: Theodore Ts'o @ 2023-09-08 14:58 UTC (permalink / raw) To: Warner Losh; +Cc: tuhs On Thu, Sep 07, 2023 at 10:05:37AM -0600, Warner Losh wrote: > > Yea. I find it easier :). But there's some better automation available in > Linux for the 'complicated' situations. Yeah, this is why most of the Linux distributions always use an initramfs, at least by default, because it handles all (or at least most) of these complicated sitautions automatically. So this leads to people assuming that Linux "requires" using a initramfs, when it would be more accurate to say that most Linux distributions makes it super easy to use an initramfs, and much more difficult (because the documentation is a bit scattered, and many may be somewhat out of date) to do things without an Linux distribution-engineered initramfs. And the other clear difference between Linux and FreeBSD is that things like initramfs tends to be distribution-specific, so there is competition between distributions to see who can make a simpler/easier installer and boot sequnce, and so market forces tends to strongly encourage developers (who after all, tend to be experts and who don't need the installer and boot up automation), to improve things much more aggressively. (And since there are product managers involved, often features and time to market often trump issues like technical debt.) The downside with this approach, though, is that distributions tend to reinvent the wheel multiple times, and the initramfs infrastructures are different. So learning how things work in say, Debian, isn't going to help you understand the fine details of how Fedora or Red Hat Enterprise Linux works except at a very basic, high level. And the same is most definitely true if you compare the installers between Debian, Ubuntu, Fedora, etc. FreeBSD has the advantage that it has a much more centralized development model, especially as it relates to kernel/libc integration, the installer, and the boot loader path. But while there is a certain amount of competitive forces of FreeBSD, NetBSD, OpenBSD, et. al., improving and simplifying the boot loader, from an outsider looking in, it would appear that those forces aren't quite as strong, and the level of convergence beween the *BSD seems to be at least as strong, if not stronger, in some areas, than between Linux distributions. Cheers, - Ted ^ permalink raw reply [flat|nested] 27+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-05 15:53 ` Steffen Nurpmeso 2023-09-06 17:50 ` Warner Losh @ 2023-09-08 13:56 ` Michael Kjörling 2023-09-08 23:38 ` Steffen Nurpmeso 1 sibling, 1 reply; 27+ messages in thread From: Michael Kjörling @ 2023-09-08 13:56 UTC (permalink / raw) To: tuhs On 5 Sep 2023 17:53 +0200, from steffen@sdaoden.eu (Steffen Nurpmeso): > Unfortunately cryptsetup is needed even though, i think, the > kernel has anything needed; you just cannot access it. cryptsetup > is only needed for "$cs open $PART_ROOT p_root --key-file -". > Of course i am no real Linux expert but only a do-it-yourself guy. If your need is restricted to a highly specific use case and you are trying to keep it as small as possible, then it should be possible to write a custom wrapper around whatever libcryptsetup functionality you need and avoid the extra code that you get with cryptsetup proper. -- Michael Kjörling 🔗 https://michael.kjorling.se “Remember when, on the Internet, nobody cared that you were a dog?” ^ permalink raw reply [flat|nested] 27+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-08 13:56 ` Michael Kjörling @ 2023-09-08 23:38 ` Steffen Nurpmeso 2023-09-09 22:43 ` Steffen Nurpmeso 2023-09-11 4:10 ` Theodore Ts'o 0 siblings, 2 replies; 27+ messages in thread From: Steffen Nurpmeso @ 2023-09-08 23:38 UTC (permalink / raw) To: Michael Kjörling; +Cc: tuhs Michael Kjörling wrote in <f948d06c-14f4-40bd-8e32-4db1c5b1dd21@home.arpa>: |On 5 Sep 2023 17:53 +0200, from steffen@sdaoden.eu (Steffen Nurpmeso): |> Unfortunately cryptsetup is needed even though, i think, the |> kernel has anything needed; you just cannot access it. cryptsetup |> is only needed for "$cs open $PART_ROOT p_root --key-file -". |> Of course i am no real Linux expert but only a do-it-yourself guy. | |If your need is restricted to a highly specific use case and you are |trying to keep it as small as possible, then it should be possible to |write a custom wrapper around whatever libcryptsetup functionality you |need and avoid the extra code that you get with cryptsetup proper. It is nicely documented, and my Linux distribution ships the static library anyhow. But i am a lazy sort regarding such, i just take the thing of my distribution and copy it over (they do build it statically also by default). You know, things change, and if you do not follow closely, you stand in the rain. I am not a paid Linux engineer that follows this rapidly moving target in the end. For example the (no longer) new random developer chose to disable feeding entropy via /dev/urandom, here (distribution) still is # Load random seed /bin/cat /var/lib/urandom/seed > /dev/urandom for almost two decades (it is a rather young one), but the code path was mutilated (i read the kernel source once he had rewritten that to be blake2/some 32-byte block thing based), now one needs to use some ioctl interface fwiw. Or once here cryptsetup was updated to use OpenSSL 3.0 suddenly ripemd160 was no longer available on EFI (aka purely static, without the filesystem avaialable), even though its release notes explicitly mentioned the problem as solved, and OpenSSL 3's libcrypto.a _had_ ripemd160... I had to switch to sha512 .. then to sha256 once cryptsetup started warning args had to be explicit in the future. Mind you, i in fact use it twice, also for encrypted swap, i only wrongly searched for $cs $2 open --type plain --cipher aes --key-size 256 \ --hash sha256 $PART_SWAP p_swap --key-file - && i said on IRC cryptsetup does EVP_DigestInit_ex(h->md, h->hash_id, NULL), i presume that does load additional things. That surely is it, i did not track it further. So no, to answer you, i have no highly specific use case at all. This is only an encrypted volume with my own boot-style that requires no boot loader but Linux itself. Maybe i should really look deeply in how cryptsetup then attaches a LUKS2 volume to the kernel, maybe it actually _would_ be possible to do this simply in some other way. But truly writing a program? I feel much saver in the horde, with so many people, specialists even, working on Linux, LUKS2, cryptsetup, OpenSSL, .. these are all moving targets. (I mean, i am lucky if i _can_ do a bit of programming on at least the MUA i maintain; so much to do! And roff hopefully somewhere on the horizon, somewhen; today it was zero minutes.) --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) ^ permalink raw reply [flat|nested] 27+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-08 23:38 ` Steffen Nurpmeso @ 2023-09-09 22:43 ` Steffen Nurpmeso 2023-09-11 4:10 ` Theodore Ts'o 1 sibling, 0 replies; 27+ messages in thread From: Steffen Nurpmeso @ 2023-09-09 22:43 UTC (permalink / raw) To: Michael Kjörling; +Cc: tuhs [-- Attachment #1: Type: text/plain, Size: 7275 bytes --] Steffen Nurpmeso wrote in <20230908233854.Xni_j%steffen@sdaoden.eu>: |Michael Kjörling wrote in | <f948d06c-14f4-40bd-8e32-4db1c5b1dd21@home.arpa>: ||On 5 Sep 2023 17:53 +0200, from steffen@sdaoden.eu (Steffen Nurpmeso): ... ||> Of course i am no real Linux expert but only a do-it-yourself guy. || ||If your need is restricted to a highly specific use case and you are ||trying to keep it as small as possible, then it should be possible to ... Actually maybe someone may find it funny, even though it is neither historic nor very sophisticated. Let me post it. In the end it takes some time to get there. I would expect almost all of you will not be interested in the lengthy rest of this mail. Ciao, and a nice Sunday i wish from Germany! All you need are statically linked busybox (and cryptsetup with encryption), and a kernel with EFI_STUB on the EFI partition, so it can be booted directly. The distribution kernels i have (ArchLinux, AlpineLinux) have it enabled; if your EFI is large enough you could simply copy it and all its masses of modules to EFI. Otherwise the kernel needs all the firmware and modules to boot and to access all filesystems where the real stuff is. My own kernel has a lot of things statically built-in, but it can load more via modules (stage2 uses "the same" kernel later, and does). Kernels on stage2 need kexec, i have CONFIG_KEXEC=y CONFIG_KEXEC_FILE=y CONFIG_ARCH_HAS_KEXEC_PURGATORY=y # CONFIG_KEXEC_SIG is not set CONFIG_KEXEC_CORE=y -s1.sh and -s2.sh are docu-commented at the top; they can also be used to create the necessary environment. If i recall correctly cd /boot sh linit-init-s2.sh PATH-TO-BUSYBOX PATH-TO-CRYPTSETUP mount EFI-PARTITION cd EFI-PARTITION sh linit-init-s1.sh PATH-TO-BUSYBOX PATH-TO-CRYPTSETUP This only creates files and directories etc, and beat me if it does any harm otherwise. I use a specific naming scheme, the script reacts in particular on -old and -new to select sensible defaults. #?0|kent:/boot# ll ideapad*.efi -rwxr-xr-x 1 root root 10112672 Aug 26 18:44 ideapad-stage1.efi* -rw-r--r-- 1 root root 10120512 Sep 2 19:52 ideapad-6_1.efi -rw-r--r-- 1 root root 10120512 Sep 9 18:18 ideapad-6_1-old.efi -rw-r--r-- 1 root root 10121376 Sep 9 18:18 ideapad-6_1-new.efi That very kernel (series) is in fact shared in between several computers of a similar series, kent is the one i write this on: #?1|kent:/boot# v kent.sh #!/busybox.static sh #@ kent, step 1., via EFI. PART_ROOT=/dev/nvme0n1p? ROOT_DECRYPT='-t btrfs -o defaults,subvol=/crux/kent/root' PART_ROOT1=/dev/nvme0n1p8 ROOT_DECRYPT1='-t btrfs -o defaults,subvol=/crux/kent/root.old' INIT_S2=/boot/kent-2.sh . /linux-init-s1.sh #?0|kent:/boot# v kent-2.sh #@ kent, stage 2 KERNEL_ID=ideapad ^ restrict kernel selection for kexec: ^ [ "${k##*$KERNEL_ID-*.efi}" != "$k" ] || continue PART_SWAP=/dev/nvme0n1p6 SWAP_INPUT=y ^ you can have additional interactive random for swap encryption ^ (it is a lie i use that in practice) INITRD_PATH=/boot/.kent.initrd KEXEC_ARGS="--append=\"rtw88_pci.disable_aspm=1 rc.hostname=kent\"" FILE_CHECK='kent-direct.sh kent.sh ideapad-stage1.efi' SWITCH_ROOT=media/initrd Stage 2 config file is sourced automatically, so no need to source linux-init-s2.sh yourself (stage 2 uses switch_root). I also have a direct boot, but have not used it for long: #?0|kent:~# v /boot/kent-direct.sh #!/busybox.static sh #@ kent-direct, sole step 1., via EFI. PART_ROOT=/dev/nvme0n1p8 ROOT_DECRYPT='-t btrfs -o defaults,subvol=/crux/kent/root' PART_ROOT1=/dev/nvme0n1p8 ROOT_DECRYPT1='-t btrfs -o defaults,subvol=/crux/kent/root.old' PART_SWAP=/dev/nvme0n1p6 PIVOT_OLDROOT=media/initrd . /linux-init-s1.sh This does not use kexec, but only pivot_root's to the decrypted filesystem, starting sbin/init there. The kernel from EFI continues to run. It is pretty cool how i now can drive several computers easily simply by creating a shell script with some variable assignments. In fact the filesystem is shared in between multiple computers (with only selective partitions being truly unique, but still backed up everywhere), and all those text configs are in /boot. Only the VFAT EFI partition is truly system-specific (ie, in that it only holds the stage1 config file for $HOSTNAME). I like that very much, and hope Linux continues to be so flexible. (Not that one day you need systemd-early to get into Linux to get to systemd-later, or something.) The EFI boot is driven via efibootmgr(8) and has config like #@ /root/hosts/self/efiboot d=/dev/nvme0n1p1 maxno=1 b1=0x01 L1=kent l1=ideapad-stage1 u1='root='${d}' rootfstype=vfat init=/kent.sh' driven by /root/bin/efiboot.sh (shortened): #!/bin/sh : ${HOSTNAME:=$(uname -n)} : ${DBG:=} if [ -f /root/hosts/${HOSTNAME}/efiboot ]; then . /root/hosts/${HOSTNAME}/efiboot else echo >&2 "MISS /root/hosts/${HOSTNAME}/efiboot" exit 1 fi if [ "$1" = create ]; then obo=$(efibootmgr | grep -E ^BootOrder: | sed -E 's/^BootOrder:[[:space:]]*//') if [ -z "$obo" ]; then echo >&2 'Cannot determine previous boot order' exit 1 fi nbo= i=1 while [ $i -le $maxno ]; do eval ${DBG} efibootmgr -c -b \$b$i -L \\\"\$L$i\\\" -d \$d$i -l \\\"\$l$i.efi\\\" -u \\\"\$u$i\\\" #>/dev/null [ -n "$nbo" ] && nbo=$nbo, eval nbo=\$nbo\$b$i i=$((i + 1)) done ${DBG} efibootmgr -o $nbo,$obo #> /dev/null elif [ "$1" = delete ]; then i=1 while [ $i -le $maxno ]; do eval ${DBG} efibootmgr -B -b \$b$i #>/dev/null i=$((i + 1)) done else echo >&2 'Synsopsis: efiboot.sh create|delete' exit 1 fi ${DBG} efibootmgr -v -u This is of course git managed, so i can distribute it easily. (Further even, the entire ideapad series configurations differ in only 193 lines, and most of those are the different SSH keys. The rest are different device names, ie network, display, volume etc, disk partitions, host names.) I am pretty lucky i found a way to manage my systems like this. I have several git branches in root.git, linux.kconfig, bin, iwd.network, and one for each computer. I can then "git merge" selectively together what is necessary. For example my web vserver only needs bin and its own configuration, whereas the laptops also need iwd.network. Etc etc. Like this every computer only knows the selective minimum. The full git repo is on an encrypted volume stored away. And it is not Puppet or Ansible or any of those programs, it is nothing but git and its branches. But now i really left manually driven cryptsetup by far. Ciao! --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) [-- Attachment #2: linux-init-s1.sh --] [-- Type: application/x-sh, Size: 4596 bytes --] #@ /linux-init-s1.sh, simple Linux init, stage 1. #@ To be sourced from a kernel command line init=SCRIPT like: #@ #@ #!/busybox.static sh #@ #@ For kent #@ #@ PART_ROOT=/dev/nvme0n1p8 #@ ROOT_DECRYPT='-t btrfs -o defaults,subvol=/crux/kent/root' #@ PART_SWAP=/dev/nvme0n1p6 #@ INIT_S2=boot/kent-2.sh #@ #@ . /linux-init-s1.sh #@ #@ Alternatively call AS ROOT and FROM WITHIN directory that boots like #@ sh linux-init-s1.sh PATH_TO_BUSYBOX PATH_TO_CRYPTSETUP #@ and is expected to be mounted as / on system start, for init purposes. # Non-empty and we run mdev -s very first (value passed to it via ARGV) : ${MDEV_RUN:=} # Unless ROOT_DECRYPT (only partition then) this can be mount spec, like # PART_ROOT='-t btrfs -o defaults,subvol=/crux/kent/root' [ $# -eq 0 ] && : ${PART_ROOT:?"Need a usable \$PART_ROOT"} # Non-empty and we decrypt ROOT; _it_ should be mount spec then : ${ROOT_DECRYPT:=} # It is possible to specify multiple PART_ROOT[/ROOT_DECRYPT] tuples by # giving PART_ROOT1, 2, 3... A dialog for choosing one to use then appears # Non-empty and we create encrypted swap from the given partition : ${PART_SWAP:=} # Non-empty and we ask for more random bytes for swap encryption : ${SWAP_INPUT:=} # Non-empty and we give up to the stage 2 -s2.sh that is expected to live # in [/dev/mapper/p_root]/boot. ONLY WITH $ROOT_DECRYPT! # The value is the config file of stage 2 as an _absolute_ path within the # newly mounted root. : ${INIT_S2:=} # Were we move old root to when doing pivot_root(8) in non-$INIT_S2 cases : ${PIVOT_OLDROOT:=media/efi} BB=busybox.static CS=cryptsetup.static # -- >8 -- 8< -- xroot() { # {{{ bb=./$BB cs=./$CS slept= # May want to sleep to pass by kernel async console output xbase_mounts "$bb" # (First for completeness sake [not needed]) # PART_SUFFIX=.0 if [ -n "$PART_ROOT1" ]; then # Sleep a bit and hope kernel async console output passed by then [ -z "$slept" ] && $bb sleep 3 slept=y while :; do echo echo '. Multiple $PART_ROOT[/$ROOT_DECRYPT] encountered:' echo " [0] PART_ROOT=$PART_ROOT" [ -n "$ROOT_DECRYPT" ] && echo " ROOT_DECRYPT=$ROOT_DECRYPT" ri=1 while :; do eval rx="\$PART_ROOT$ri" [ -z "$rx" ] && break echo " [$ri] ..ROOT$ri=$rx" eval rx="\$ROOT_DECRYPT$ri" [ -n "$rx" ] && echo " ..CRYPT$ri=$rx" ri=$((ri + 1)) done rmsg= rrt= if (echo y | read -t 1 chck) >/dev/null 2>&1; then rmsg=' (10 seconds until, or empty input for default)' rrt='-t 10' fi printf '. Please choose $PART_ROOT'"$rmsg"'? ' read $rrt PART_SUFFIX if [ $? -ne 0 ] || [ -z "$PART_SUFFIX" ] || [ "$PART_SUFFIX" = 0 ]; then PART_SUFFIX=.0 echo break elif [ "${PART_SUFFIX##*[^0-9]}" != "$PART_SUFFIX" ]; then echo >&2 '! Invalid input' continue fi eval ri=\$PART_ROOT$PART_SUFFIX if [ -z "$ri" ]; then echo >&2 '! No such PART_ROOT: '$PART_SUFFIX else PART_ROOT=$ri eval ROOT_DECRYPT=\$ROOT_DECRYPT$PART_SUFFIX PART_SUFFIX=.$PART_SUFFIX echo break fi done fi # if [ -n "$ROOT_DECRYPT" ]; then [ -z "$slept" ] && $bb sleep 3 slept=y while :; do printf '. Please set free $PART_ROOT: ' $bb stty -echo read ROOT_PASS $bb stty echo printf "%s" "$ROOT_PASS" | $cs open $PART_ROOT p_root --key-file - && break xconfirm 'Cannot cryptsetup open '"$PART_ROOT"', try again' || x 1 done $bb mount $ROOT_DECRYPT /dev/mapper/p_root mnt || x 2 if [ -n "$INIT_S2" ]; then echo '. Going $INIT_S2='"$INIT_S2" cd mnt || x 3 boot/$BB mount --bind ../dev dev || x 10 boot/$BB mount --bind ../sys sys || x 11 boot/$BB mount --bind ../proc proc || x 12 boot/$BB mount --bind ../run run || x 13 exec boot/$BB chroot . /boot/$BB sh \ /boot/linux-init-s2.sh --stage1 "$INIT_S2" \ "$PART_SUFFIX" "$PART_ROOT" \ "$ROOT_DECRYPT" "$ROOT_PASS" || x 'Cannot go $INIT_S2' fi else echo '. Mounting $PART_ROOT' $bb mount $PART_ROOT mnt || x 2 fi xswap $bb $cs echo '. About to pivot_root' cd mnt || x 3 bb=boot/$bb $bb pivot_root . $PIVOT_OLDROOT || x 4 $bb mount -o move $PIVOT_OLDROOT/dev dev || x 10 $bb mount -o move $PIVOT_OLDROOT/sys sys || x 11 $bb mount -o move $PIVOT_OLDROOT/proc proc || x 12 $bb mount -o move $PIVOT_OLDROOT/run run || x 13 echo '. Going sbin/init' $bb stty isig exec $bb chroot . sbin/init || x 'Cannot sbin/init' } # }}} if [ $# -ne 0 ]; then . $($1 dirname "$0")/linux-init-lib.sh xsetup "$1" "$2" else . $(./$BB dirname "$0")/linux-init-lib.sh xroot fi exit 127 # s-sht-mode [-- Attachment #3: linux-init-s2.sh --] [-- Type: application/x-sh, Size: 7186 bytes --] #@ /linux-init-s2.sh, simple Linux init, optional stage 2, called from stage 1. #@ Requires $BB, $CS, as below, and these funs, on EFI / as well as new /boot/. #@ #@ #@ For kent, stage 2 #@ #@ [KERNEL_ID=ideapad] - if set, only $KERNEL_ID-*.efi kernels are examined #@ PART_ROOT=/dev/nvme0n1p8 #@ ROOT_DECRYPT='-t btrfs -o defaults,subvol=/crux-3.6/kent/root' #@ PART_SWAP=/dev/nvme0n1p6 #@ INITRD_PATH=boot/.s2.initrd #@ SWITCH_ROOT=media/initrd #@ #@ Alternatively call AS ROOT and FROM WITHIN directory that boots like #@ sh linux-init-s2.sh PATH_TO_BUSYBOX PATH_TO_CRYPTSETUP #@ for init purposes. # Identical to stage 1. : ${MDEV_RUN:=} #: ${PART_ROOT:=} via stage 1 #: ${ROOT_DECRYPT:=} via stage 1 (ditto ROOT_PASS) : ${PART_SWAP:=} : ${SWAP_INPUT:=} # If non-empty, we try to load the given initrd (relative to root) instead of # newly creating it; also, we will try to save it there if we had to create it # Will get PART_SUFFIX of chosen PART_ROOT/ROOT_DECRYPT tuple! : ${INITRD_PATH:=} # Further arguments when kexec(8)ing the new kernel : ${KEXEC_ARGS:=} : ${KEXEC_EXE:=usr/sbin/kexec} # Running in the kexec(8) kernel, we may apply some file comparisons. # If set it is a list of space separated files which must exist in /boot as # well as $FILE_CHECK_PARTITION (most likely the same as $SWITCH_ROOT), that is # temporarily mounted if it is not yet. # The files $BB, $CS, linux-init-{lib,s1}.sh are implied : ${FILE_CHECK:=} : ${FILE_CHECK_PARTITION:=media/efi} : ${FILE_CHECK_MOUNT_ARGS:=-n -t vfat /dev/nvme0n1p1} # Where we move old root to when doing switch_root(8) : ${SWITCH_ROOT:=media/initrd} BB=busybox.static CS=cryptsetup.static # -- >8 -- 8< -- xkexec() { # $1=config file; assumes we are in / and boot/ "has it" {{{ conf=$1 # Find the possible kernels to boot set -- boot/*.efi ki=1 kdef= knew= for k do if [ -n "$KERNEL_ID" ]; then [ "${k##*$KERNEL_ID-*.efi}" != "$k" ] || continue fi if [ "${k##*-new.efi}" = "$k" ] && [ "${k##*-old.efi}" = "$k" ] && [ "${k##*-stage1.efi}" = "$k" ]; then #[ -z "$kdef" ] && kdef=$k kdef=$k fi if [ -z "$knew" ] && [ "${k##*-new.efi}" != "$k" ]; then knew=$k fi eval k$ki=$k ki=$((ki + 1)) done # Let user choose one trap 'x "Got signal"' INT QUIT TERM echo '+ trapping INT/QUIT/TERM to start a shell' k= k0=- while [ -z "$k" ]; do echo echo '. Available kernels:' echo ' [0] Edit $KEXEC_ARGS' kj=1 while [ $kj -lt $ki ]; do eval echo "\" [$kj] \$k$kj\"" kj=$((kj + 1)) done echo if [ -n "$knew" ] && xconfirm 'Use the new kernel ('"$knew"')?'; then k=$knew else kmsg= krt= if [ -n "$kdef" ] && (echo y | read -t 1 chck) >/dev/null 2>&1; then kmsg=' (10 seconds until, or empty input for '"$kdef"')' krt='-t 10' fi printf '. Please choose kernel'"$kmsg"'? ' read $krt kx if [ $? -ne 0 ] || [ -z "$kx" ]; then echo k=$kdef elif [ "${kx##*[^0-9]}" != "$kx" ]; then echo >&2 '! Invalid input' continue fi eval k=\$k$kx if [ -z "$k" ]; then echo >&2 '! No such kernel' elif [ "$k" = - ]; then printf "%s\n" "$KEXEC_ARGS" > run/.kexec echo '# Only first line counts. Care for quoting! :wq' run/.kexec boot/$BB vi run/.kexec k=$KEXEC_ARGS read KEXEC_ARGS < run/.kexec boot/$BB rm -f run/.kexec xconfirm "New KEXEC_ARGS=$KEXEC_ARGS" || KEXEC_ARGS=$k k= fi fi done # Create or reuse a temporary initrd that contains the password if [ -n "$INITRD_PATH" ] && [ -f "$INITRD_PATH$PART_SUFFIX" ] && boot/$BB cp "$INITRD_PATH$PART_SUFFIX" run/.initrd; then echo '. Reusing initrd '"$INITRD_PATH$PART_SUFFIX" else echo '. Creating initrd' boot/$BB mount -o remount,exec run ( set -e boot/$BB mkdir run/x cd run/x ( xsetup2 "../../boot/$BB" "../../boot/$CS" ) e=$? [ $e -ne 0 ] && exit $e ./$BB mknod dev/console c 5 1 # redundant echo "#!/$BB sh" > ./init ./$BB cat "$conf" >> ./init echo "PART_ROOT='$PART_ROOT'" >> ./init echo "ROOT_DECRYPT='$ROOT_DECRYPT'" >> ./init echo "ROOT_PASS='$ROOT_PASS'" >> ./init echo '. /linux-init-s2.sh' >> ./init ./$BB chmod 0755 ./init { # Microcode update must be uncompressed and first [ -f ../../boot/early-ucode.cpio ] && ./$BB cat ../../boot/early-ucode.cpio # Followed by (possibly compressed) normal initrd ./$BB find . | ./$BB cpio -H newc -o | ./$BB gzip -9 -n } > ../.initrd ) || x 'Failed to create run/.initrd' boot/$BB mount -o remount,noexec run if [ -n "$INITRD_PATH" ]; then echo '. Saving initrd to '"$INITRD_PATH$PART_SUFFIX" boot/$BB cp run/.initrd "$INITRD_PATH$PART_SUFFIX" boot/$BB chmod 0600 "$INITRD_PATH$PART_SUFFIX" fi fi KEXEC_ARGS="$KEXEC_ARGS --initrd=run/.initrd" echo ". $KEXEC_EXE -l $k $KEXEC_ARGS" eval $KEXEC_EXE -l $k $KEXEC_ARGS || x 'Cannot invoke '"$KEXEC_EXE"' -l' echo ". $KEXE_EXE -e" exec $KEXEC_EXE -e || x 'Cannot invoke '"$KEXEC_EXE"' -e' } # }}} xfilecheck() { # $1=$BB; assumes we are in /mnt (_next_ root) {{{ [ -z "$FILE_CHECK" ] && return echo '. Checking file equalities' umountpart= if xhavemnt "$1" "$FILE_CHECK_PARTITION"; then :; else umountpart=y $1 mount $FILE_CHECK_MOUNT_ARGS "$FILE_CHECK_PARTITION" || x 'Cannot mount '"$FILE_CHECK_PARTITION" fi err= $1 cmp -s "$1" "$FILE_CHECK_PARTITION"/$BB || err="$BB" $1 cmp -s boot/$CS "$FILE_CHECK_PARTITION"/$CS || err="$err $CS" $1 cmp -s boot/linux-init-lib.sh "$FILE_CHECK_PARTITION"/linux-init-lib.sh || err="$err init-lib.sh" $1 cmp -s boot/linux-init-s1.sh "$FILE_CHECK_PARTITION"/linux-init-s1.sh || err="$err init-s1.sh" if [ -f boot/early-ucode.cpio ] && [ -f "$FILE_CHECK_PARTITION"/early-ucode.cpio ]; then $1 cmp -s boot/early-ucode.cpio "$FILE_CHECK_PARTITION"/early-ucode.cpio || err="$err early-ucode.cpio" fi for f in $FILE_CHECK; do if [ -f "boot/$f" ] && [ -f "$FILE_CHECK_PARTITION/$f" ]; then $1 cmp -s "boot/$f" "$FILE_CHECK_PARTITION/$f" || err="$err $f" else err="$err (MISS: $f)" fi done [ -n "$umountpart" ] && $1 umount "$FILE_CHECK_PARTITION" if [ -z "$err" ]; then echo ' . Files successfully checked' else echo >&2 '! BOOT: $FILE_CHECK errors: '$err echo '! BOOT: $FILE_CHECK errors: '$err >> etc/motd $1 sleep 10 fi } # }}} xroot() { # {{{ # On initrd rootfs bb=./$BB cs=./$CS xbase_mounts "$bb" echo '. Setting free $PART_ROOT' printf "%s" "$ROOT_PASS" | $cs open $PART_ROOT p_root --key-file - || x 'Cannot cryptsetup open '"$PART_ROOT" $bb mount $ROOT_DECRYPT /dev/mapper/p_root mnt || x 2 bb=mnt/boot/$BB cs=mnt/boot/$CS xswap $bb $cs cd mnt || x 3 bb=boot/$BB cs=boot/$CS xfilecheck $bb echo '. Going switch_root sbin/init' $bb stty isig # -c dev/console exec $bb switch_root . sbin/init 2>&1 || x 'Cannot switch_root' } # }}} if [ "$1" = --stage1 ]; then cnf=$2 PART_SUFFIX=$3 PART_ROOT=$4 ROOT_DECRYPT=$5 ROOT_PASS=$6 [ -f "$cnf" ] || x 'Cannot find $INIT_S2: '"$cnf" . "$cnf" . /boot/linux-init-lib.sh xkexec "$cnf" elif [ $# -eq 2 ]; then . $($1 dirname "$0")/linux-init-lib.sh xsetup2 "$1" "$2" - else . /linux-init-lib.sh xroot fi exit 127 # s-sht-mode [-- Attachment #4: linux-init-lib.sh --] [-- Type: application/x-sh, Size: 3646 bytes --] #@ linux-init-lib.sh, helpers for simple init stage 1 and 2; will be sourced. # misc {{{ x() { echo >&2 'ERROR ('"$*"')! Going shell' [ -x ./$BB ] && exec ./$BB sh [ -x /$BB ] && exec /$BB sh [ -x /boot/$BB ] && exec /boot/$BB sh [ -x /bin/sh ] && exec /bin/sh exit 100 } xconfirm() { printf -- '. '"$*"' [Yy/else]? ' read answer case $answer in [Yy]*) return 0;; *) return 1;; esac } _mpx= _mp= xhavemnt() { # $1=$BB $2=file if [ "$2" = dev ]; then [ -e /dev/console ] && [ -e /dev/null ] && [ -e /dev/zero ] return $? elif [ "$2" = proc ]; then [ -e /proc/1 ] && [ -e /proc/self ] && [ -e /proc/uptime ] return $? elif [ "$2" = sys ]; then [ -e /sys/kernel ] && [ -e /sys/block ] && [ -e /sys/fs ] return $? fi if [ -z "$_mpx" ]; then _mpx=y $1 mountpoint -q / >/dev/null 2>&1 && _mp=y fi if [ -n "$_mp" ]; then $1 mountpoint -q "$2" >/dev/null 2>&1 return $? elif [ -f /proc/mounts ]; then $1 grep -q ' '"$2"' ' /proc/mounts >/dev/null 2>&1 return $? fi return 1 } _xrand= xrand() { _xrand=$(( (($RANDOM << 20) | ($RANDOM << 10) | ($RANDOM % 0x3FF)) ^ \ (($RANDOM << 20) | ($RANDOM << 10) | ($RANDOM % 0x3FF)))) } # }}} xbase_mounts() { # {{{ $1=$BB [uses $MDEV_RUN; calls stty -isig] echo '. Checking /{dev,sys,proc,run}' xhavemnt $1 dev || $1 mount -n -t devtmpfs -o mode=0755,exec,nosuid,noatime none dev xhavemnt $1 sys || $1 mount -n -t sysfs none sys xhavemnt $1 proc || $1 mount -n -t proc none proc xhavemnt $1 run || $1 mount -n -t tmpfs -o mode=0755,noexec,nosuid,nodev none run #[ -e /sys/firmware/efi ] && #mount -t efivarfs efivarfs /sys/firmware/efi/efivars -o nosuid,nodev,noexec $1 stty -isig if [ -n "$MDEV_RUN" ]; then echo '. About to run mdev -s, waiting for some more settlement' $1 sleep 8 $1 mdev -s $MDEV_RUN fi } # }}} xsetup_base() { # $1=$BB $2=$CS [$3=no mdev stuff if set] {{{ $1 cp -f "$1" ./$BB ./$BB chmod 0700 ./$BB ./$BB cp -f "$2" ./$CS ./$BB chmod 0700 ./$CS ./$BB cp -f "$0" . ./$BB cp -f $(./$BB dirname "$0")/linux-init-lib.sh ./linux-init-lib.sh if [ $# -ne 3 ]; then # Prepare for $MDEV_RUN, too! ./$BB mkdir etc bin dev mnt proc run sys $mdir ./$BB cp ./$BB bin/sh ./$BB chmod 0700 bin/sh echo ".* 0:0 660 @/$BB sh /etc/mdev.sh" > etc/mdev.conf ./$BB cat > etc/mdev.sh <<-_EOT #@ /etc/mdev.sh for ./mdev.conf run() { if [ "\$ACTION" = add ]; then case "\$SUBSYSTEM" in block) block;; *) ;; esac fi exit } block() { [ -n "\$MDEV" ] || return [ -d /dev/disk ] || /$BB mkdir -p /dev/disk/by-uuid uuid=\$(/$BB blkid "\$MDEV" \| /$BB sed -Ee "s/.*UUID=\"([^\"]+)\".*/\1/") [ -n "\$uuid" ] || return /$BB ln -s ../../"\$MDEV" /dev/disk/by-uuid/"\$uuid" } run _EOT ./$BB chmod 0700 etc/mdev.sh fi echo '! Plus kernel, init=SCRIPT, to be placed here' } xsetup() { xsetup_base "$@" exit } xsetup2() { xsetup_base "$@" # For $FILE_CHECK ./$BB cp -f $(./$BB dirname "$0")/linux-init-s1.sh ./linux-init-s1.sh exit } # }}} xswap() { # $1=$BB $2=$CS {{{ [ -z "$PART_SWAP" ] && return echo '. Encrypted swap' rand= if [ -n "$SWAP_INPUT" ]; then echo ' ... please type a line of randoms' $1 stty -echo >/dev/null 2>&1 read rand $1 stty echo >/dev/null 2>&1 else xrand rand=$_xrand fi xrand xr1=$_xrand xrand xr2=$_xrand xrand xr3=$_xrand xrand xr4=$_xrand xrand xr5=$_xrand echo "$xr1$rand$xr2$xr3$rand$xr4$xr5" | $2 open --type plain --cipher aes --key-size 256 --hash sha256 $PART_SWAP p_swap --key-file - && $1 mkswap /dev/mapper/p_swap } # }}} # s-sht-mode ^ permalink raw reply [flat|nested] 27+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-08 23:38 ` Steffen Nurpmeso 2023-09-09 22:43 ` Steffen Nurpmeso @ 2023-09-11 4:10 ` Theodore Ts'o 2023-09-11 22:05 ` Steffen Nurpmeso 1 sibling, 1 reply; 27+ messages in thread From: Theodore Ts'o @ 2023-09-11 4:10 UTC (permalink / raw) To: Michael Kjörling, tuhs On Sat, Sep 09, 2023 at 01:38:54AM +0200, Steffen Nurpmeso wrote: > You know, things change, and if you do not follow closely, you > stand in the rain. I am not a paid Linux engineer that follows > this rapidly moving target in the end. > For example the (no longer) new random developer chose to disable > feeding entropy via /dev/urandom, here (distribution) still is > > # Load random seed > /bin/cat /var/lib/urandom/seed > /dev/urandom > > for almost two decades (it is a rather young one), but the code > path was mutilated (i read the kernel source once he had rewritten > that to be blake2/some 32-byte block thing based), now one needs > to use some ioctl interface fwiw. Huh? That's not correct. You can still introduce entropy into the random pool by writing to /dev/urandom or /dev/random. It is true that there has been some changes to the design of /dev/random, but I assure you as the original /dev/random developer, was consulted and involved in the design discussions. Anyway, this is off-topic for TUHS, but if you have specific questions/complaints, feel free to address them to me and Jason. Your comments do make me suspect that there are some fundamental misunderstandings at work. Cheers, - Ted ^ permalink raw reply [flat|nested] 27+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-11 4:10 ` Theodore Ts'o @ 2023-09-11 22:05 ` Steffen Nurpmeso 0 siblings, 0 replies; 27+ messages in thread From: Steffen Nurpmeso @ 2023-09-11 22:05 UTC (permalink / raw) To: Theodore Ts'o; +Cc: Michael Kjörling, tuhs Theodore Ts'o wrote in <20230911041059.GH701295@mit.edu>: |On Sat, Sep 09, 2023 at 01:38:54AM +0200, Steffen Nurpmeso wrote: |> You know, things change, and if you do not follow closely, you |> stand in the rain. I am not a paid Linux engineer that follows |> this rapidly moving target in the end. |> For example the (no longer) new random developer chose to disable |> feeding entropy via /dev/urandom, here (distribution) still is |> |> # Load random seed |> /bin/cat /var/lib/urandom/seed > /dev/urandom |> |> for almost two decades (it is a rather young one), but the code |> path was mutilated (i read the kernel source once he had rewritten |> that to be blake2/some 32-byte block thing based), now one needs |> to use some ioctl interface fwiw. | |Huh? That's not correct. You can still introduce entropy into the |random pool by writing to /dev/urandom or /dev/random. It is true |that there has been some changes to the design of /dev/random, but I |assure you as the original /dev/random developer, was consulted and |involved in the design discussions. Ah, lesser and lesser freedom for man! That is the truth! Neither was (last i looked) that counted no more to unlock man from the chokehold of missing entropy, nor do newer kernels then decrement entropy when you read random. |Anyway, this is off-topic for TUHS, but if you have specific |questions/complaints, feel free to address them to me and Jason. Your |comments do make me suspect that there are some fundamental |misunderstandings at work. Surely for the latter. So much i give in on this song line. Greetings. --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) ^ permalink raw reply [flat|nested] 27+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-04 14:44 [TUHS] Unix install & "standalone" package Norman Wilson ` (2 preceding siblings ...) 2023-09-04 22:10 ` Steffen Nurpmeso @ 2023-09-05 1:07 ` Jonathan Gray 3 siblings, 0 replies; 27+ messages in thread From: Jonathan Gray @ 2023-09-05 1:07 UTC (permalink / raw) To: Norman Wilson; +Cc: tuhs On Mon, Sep 04, 2023 at 10:44:21AM -0400, Norman Wilson wrote: > I don't remember any special many-programs-in-one binary > like busybox in any Unix from the days when Unix was simple > enough for me to understand. That covers the entire lifetime > of the Research systems, but also System V and the BSDs and > their sundry offspring up into at least the 1990s. > > I'm pretty sure OpenBSD at least still has nothing like > busybox. The nearest thing was to make sure certain programs look at ls -i output on the install media/bsd.rd https://man.openbsd.org/crunchgen.8 Added to FreeBSD and NetBSD in 1994. ^ permalink raw reply [flat|nested] 27+ messages in thread
* [TUHS] Unix install & "standalone" package @ 2023-09-04 9:57 Paul Ruizendaal via TUHS 2023-09-04 14:53 ` [TUHS] " emanuel stiebler 2023-09-04 17:07 ` Warner Losh 0 siblings, 2 replies; 27+ messages in thread From: Paul Ruizendaal via TUHS @ 2023-09-04 9:57 UTC (permalink / raw) To: tuhs Recently, I was looking into the “Das U-Boot” boot loader package. Summarised with great simplification, u-boot bundles device drivers, file systems, commands and a Bourne-like shell into a standalone package. Normally it auto-runs a script that brings up a system, but when used in interactive mode it allows a great deal of poking around. It made me think of the “standalone” set of programs for installing early Unix. On 16-bit understandably each basic command has to be a separate standalone program, but after the shift to 32-bit bundling more functionality in a single binary would have become possible. How did the Unix “standalone” package evolve in the 80’s, both in the research and BSD lineages? Is there any retrospective paper about that? Or is it a case of “Use the source, Luke”? ^ permalink raw reply [flat|nested] 27+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-04 9:57 [TUHS] " Paul Ruizendaal via TUHS @ 2023-09-04 14:53 ` emanuel stiebler 2023-09-04 17:07 ` Warner Losh 1 sibling, 0 replies; 27+ messages in thread From: emanuel stiebler @ 2023-09-04 14:53 UTC (permalink / raw) To: Paul Ruizendaal, tuhs On 2023-09-04 05:57, Paul Ruizendaal via TUHS wrote: > > Recently, I was looking into the “Das U-Boot” boot loader package. Summarised with great simplification, u-boot bundles device drivers, file systems, commands and a Bourne-like shell into a standalone package. Normally it auto-runs a script that brings up a system, but when used in interactive mode it allows a great deal of poking around. U-Boot has his roots in embedded systems, where you have a nice environment to debug your system. You have already everything there to download files with s-record from serial, boot from tftp, etc. Actually also setting up the memory controllers, timing, setting environment variables ... This became so comfortable, that sometimes the U-Boot code is larger than the target system. So, after development is done, the system jumps into the real application or code ... ^ permalink raw reply [flat|nested] 27+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-04 9:57 [TUHS] " Paul Ruizendaal via TUHS 2023-09-04 14:53 ` [TUHS] " emanuel stiebler @ 2023-09-04 17:07 ` Warner Losh 2023-09-04 18:21 ` Dan Cross 2023-09-05 11:15 ` Paul Ruizendaal via TUHS 1 sibling, 2 replies; 27+ messages in thread From: Warner Losh @ 2023-09-04 17:07 UTC (permalink / raw) To: Paul Ruizendaal; +Cc: tuhs [-- Attachment #1: Type: text/plain, Size: 7235 bytes --] On Mon, Sep 4, 2023 at 3:58 AM Paul Ruizendaal via TUHS <tuhs@tuhs.org> wrote: > > Recently, I was looking into the “Das U-Boot” boot loader package. > Summarised with great simplification, u-boot bundles device drivers, file > systems, commands and a Bourne-like shell into a standalone package. > Normally it auto-runs a script that brings up a system, but when used in > interactive mode it allows a great deal of poking around. > > It made me think of the “standalone” set of programs for installing early > Unix. On 16-bit understandably each basic command has to be a separate > standalone program, but after the shift to 32-bit bundling more > functionality in a single binary would have become possible. > > How did the Unix “standalone” package evolve in the 80’s, both in the > research and BSD lineages? Is there any retrospective paper about that? Or > is it a case of “Use the source, Luke”? > The stand package continued in research and BSD to be those programs needed to install and/or recover badly damaged systems. You could create a new file system, copy a file from the tape to a partition, etc. You couldn't do general scripting with this, by and large. Originally, they were tape programs. This made sense because of its original focus. In time, some systems could load the stand alone programs instead of the kernel, but they continued the original focus. This is, imho, due in large part due to the miniroot. The miniroot evolved into both a full-enough system to do the installation scripts in shell instead of C (Venix, at least, had their install program written in C). You'd copy the minroot to swap and then install the system. But a number of additional programs were placed into the miniroot so you could do some limited filesystem repair, file editing, etc. In addition, many vendor's ROMs grew in complexity. Solbourne's ROMs, for example, could do basic repair of UFS (clri level, not fsck level), and copy files from one place to another. I often recovered a Solbourne system I screwed up by attaching an external SCSI drive that had a known good kernel, init, etc. The 'stand' environment was a whole set of tools that could be used to build stand-alone programs that shared much code of their full unix brethren, despite not having a full kernel under them. Kernel services were provided by different libraries that did filesystem things, block driver things, network things, etc in a similar way to Unix, but with a much reduced footprint. initramfs, as has been mentioned elsewhere, is pretty much a Linux invention. It was designed to 'punt' on the choose where to load things from and have a very minimal interface between the boot loader and the system. In time, it grew to support more interfaces, more ways of loading, and better ways to mount something that you could then 'pivot' onto. Few other unix systems went this route, though many adopted some variation on the pivot_root functionality. Linux has moved beyond the pivot root after having booted the correct kernel into being able to take over the machine early in, say, UEFI startup with a minimal kernel and initramfs that just knows how to load the next kernel. They skipped the complex boot loader stage, and went straight to the 'run linux earlier' stage which is how things like LinuxBoot, coreboot and others have put the boot logic into bash scripts. The ability to 'kexec' a kernel and replace the current running kernel originated in the 'non-stop' world that wanted to reduce downtime. Now, it's used to reduce firmware complexity by eliminating large swaths of UEFI from the boot process, but also generalizes in the embedded space. FreeBSD, from around FreeBSD 2 (1995 or so), had /rescue which largely took over form the stand alone environment for the repair duties of things. FreeBSD also adopted a more complex boot loader that would load the kernel, modules, set tunables, etc prior to kicking off the kernel. Between /rescue having all the tools needed to repair bad updates, repair failing disks, get that one last backup before the drive is dead while you wait for the new drive to be delivered, etc, and /boot/loader being able to script loading the kernel while the BIOS was still around so the need for drivers in the loader was lessened. However, as the BIOS evolved into UEFI and FreeBSD pushed into the embedded space whose firmware provided a less rich environment to the boot loader, so it was able to load things off fewer and fewer devices, it became clear that it would need a pivot root feature to allow it to boot all the way into FreeBSD, load some drivers from an included ram disk, and then use that mount a new root and then 'reroot' to that by killing everything and running init from that new root. FreeBSD also moved from Forth to Lua in its scripting language for the boot loader, giving 'pre boot' environment support better features. I also added the ability to use FreeBSD boot loader as a Linux binary to load FreeBSD and its metadata from a LinuxBoot environment. Finally, FreeBSD has 'spun out' and generalized the /rescue feature to allow creation of any 'BeastyBox' environment, similar to what you get in a busy box, or clone, environment. This environment, though, is meant in large part on both Linux and FreeBSD to be in constrained environments where a full install is prohibitive (even those that never pivot to something more, like ap, routers and nas boxes). NetBSD retains many of the old BSD stand-alone programs that started on the vax. I've not studied things beyond noticing this. OpenBSD is similar. Their boot chain is a bit simpler than FreeBSD's, though there's noises about porting FreeBSD's boot there. There's a port of /boot/loader to illumos too, but I don't know if it is the default, or just available. So I'll not chat about it more. So the original 'standalone' environment where you had one program running on a system has evolved into either a rich boot loader environment that lets one do a lot to decide what kernel to load, or towards having a minimal selection of unix programs faster and using /bin/sh or similar to do scripting. These reduced environments are often called standalone, though all they share just the name with the earlier 'stand' programs: they are full unix programs, but with reduced feature sets and 'linker magic' to package them in a way that's faster, smaller, etc (eg all in one binary). FreeBSD's boot loader is an outgrowth of the original standalone env, by way of a port of NetBSD's libsa. I suspect in the future, we'll see more and more of a trend for low-level init and then handing off to some built-in kernel (be it Linux, BSD-based (there's now kexec), or whatever) to reuse more of the vetted code rather than re-inventing Unix inside the boot loader (which is a valid criticism of FreeBSD's boot loader, though it's rich feature set is what you get for the complexity). Does that answer the prompt? Should I try to make this into more of a retrospective paper and actually do the research on the areas I was hand-wavy about? Warner [-- Attachment #2: Type: text/html, Size: 8399 bytes --] ^ permalink raw reply [flat|nested] 27+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-04 17:07 ` Warner Losh @ 2023-09-04 18:21 ` Dan Cross 2023-09-05 11:15 ` Paul Ruizendaal via TUHS 1 sibling, 0 replies; 27+ messages in thread From: Dan Cross @ 2023-09-04 18:21 UTC (permalink / raw) To: Warner Losh; +Cc: Paul Ruizendaal, tuhs On Mon, Sep 4, 2023 at 1:08 PM Warner Losh <imp@bsdimp.com> wrote: > [snip] > There's a port of /boot/loader to illumos too, but I don't know > if it is the default, or just available. So I'll not chat about it more. I can confirm that it is the default on i86pc, though not universally. At Oxide, for example, we boot directly into a very small loader held in flash that has a compressed cpio archive containing the kernel and a few necessary kernel modules compiled into it (as a byte blob); we uncompress that blob into physical memory, locate and load the kernel like a normal ELF binary, and jump to the kernel's ELF entry point, passing a few basic arguments (notably, the location and length of the cpio archive in physical memory). Entry to the kernel is thus in 64-bit mode with paging enabled. > So the original 'standalone' environment where you had one program running on a system has evolved > into either a rich boot loader environment that lets one do a lot to decide what kernel to load, or towards > having a minimal selection of unix programs faster and using /bin/sh or similar to do scripting. These > reduced environments are often called standalone, though all they share just the name with the earlier > 'stand' programs: they are full unix programs, but with reduced feature sets and 'linker magic' to package > them in a way that's faster, smaller, etc (eg all in one binary). FreeBSD's boot loader is an outgrowth > of the original standalone env, by way of a port of NetBSD's libsa. > > I suspect in the future, we'll see more and more of a trend for low-level init and then handing off to some > built-in kernel (be it Linux, BSD-based (there's now kexec), or whatever) to reuse more of the vetted code > rather than re-inventing Unix inside the boot loader (which is a valid criticism of FreeBSD's boot loader, > though it's rich feature set is what you get for the complexity). > > Does that answer the prompt? Should I try to make this into more of a retrospective paper and actually > do the research on the areas I was hand-wavy about? That would be interesting. I can still remember booting IBM 6150 RTs into a miniroot environment and using that to create and initialize filesystems when installing AOS (4.3BSD for the RT) back in the day. To my mind, the standalone programs were always oriented towards solving the related problems of bootstrap initialization onto a fresh machine, and disaster recovery when things were really, really messed up. As I recall, the RT miniroot could either load from tape, or one could `dd` it into swap and boot from that. In either case, I seem to recall it was copied into memory and run as a RAM disk. The idea of busy-box like "a bunch of utilities compiled into the same binary" was to save space, particularly since this would be copied into RAM; even with demand paging, redundant copies of bits from `libc` in each binary were a waste for what was intended to be a minimal environment anyway. - Dan C. ^ permalink raw reply [flat|nested] 27+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-04 17:07 ` Warner Losh 2023-09-04 18:21 ` Dan Cross @ 2023-09-05 11:15 ` Paul Ruizendaal via TUHS 2023-09-05 14:15 ` Clem Cole 2023-09-05 17:03 ` Warner Losh 1 sibling, 2 replies; 27+ messages in thread From: Paul Ruizendaal via TUHS @ 2023-09-05 11:15 UTC (permalink / raw) To: Warner Losh; +Cc: tuhs > Does that answer the prompt? Should I try to make this into more of a retrospective paper and actually > do the research on the areas I was hand-wavy about? That certainly answered the prompt, much appreciated the walkthrough. Currently just trying to get a view of the field and to collect recollections on how it was done back in the day. Part of it is finding a conceptual framework that can make sense of it all. One pitfall I would like to avoid in my own thinking is conflating “installing” and “booting”, even though the two seem related (one loads bits into permanent storage, the other into volatile storage; both are built around incrementally adding capability). When variable hardware comes into play, the two mix even more. Also related is the topic of recovery. The starting point seems to be a setup where a small set of standalone programs is used to load or repair the bits, as was done for 16-bit unix. A next conceptual step seems to be where first a very basic system is installed that is then used for further installation or for repair. This step seems to have come early, if this 32V install page is reflective of how it was done back in 1980: https://gunkies.org/wiki/Installing_32V_on_SIMH The idea to use disk swap space for this also seems to have come early (and I suppose the concept lives on in “rescue partitions”). Another conceptual step might be where early installer phases run a different, smaller kernel (or even OS) than the one being installed. There seems to be much potential for a “wheel of reincarnation” here where as an installer grows large, a pre-installer is created to load the installer and then the pre-installer grows large, etc. For booting, this wheel seems to have turned about 5 times in current Linux. Installing that from scratch on a fully blank SBC (without prepping a removable disk on another computer) also appears to have 4 or 5 revolutions. That is one revolution every 5 years for the 25 years from the late seventies to the early 2000’s. Then there is the question of where in the "installer stack" to stop. My current interest excludes (precursors to) package managers, containers, etc. In short: much to ponder, thanks to all for sharing recollections. > On 4 Sep 2023, at 19:07, Warner Losh <imp@bsdimp.com> wrote: > > > > On Mon, Sep 4, 2023 at 3:58 AM Paul Ruizendaal via TUHS <tuhs@tuhs.org> wrote: > > Recently, I was looking into the “Das U-Boot” boot loader package. Summarised with great simplification, u-boot bundles device drivers, file systems, commands and a Bourne-like shell into a standalone package. Normally it auto-runs a script that brings up a system, but when used in interactive mode it allows a great deal of poking around. > > It made me think of the “standalone” set of programs for installing early Unix. On 16-bit understandably each basic command has to be a separate standalone program, but after the shift to 32-bit bundling more functionality in a single binary would have become possible. > > How did the Unix “standalone” package evolve in the 80’s, both in the research and BSD lineages? Is there any retrospective paper about that? Or is it a case of “Use the source, Luke”? > > The stand package continued in research and BSD to be those programs needed to install and/or recover > badly damaged systems. You could create a new file system, copy a file from the tape to a partition, etc. > You couldn't do general scripting with this, by and large. > > Originally, they were tape programs. This made sense because of its original focus. In time, some systems > could load the stand alone programs instead of the kernel, but they continued the original focus. > > This is, imho, due in large part due to the miniroot. The miniroot evolved into both a full-enough system > to do the installation scripts in shell instead of C (Venix, at least, had their install program written in C). > You'd copy the minroot to swap and then install the system. But a number of additional programs were > placed into the miniroot so you could do some limited filesystem repair, file editing, etc. > > In addition, many vendor's ROMs grew in complexity. Solbourne's ROMs, for example, could do basic > repair of UFS (clri level, not fsck level), and copy files from one place to another. I often recovered a > Solbourne system I screwed up by attaching an external SCSI drive that had a known good kernel, > init, etc. > > The 'stand' environment was a whole set of tools that could be used to build stand-alone programs that > shared much code of their full unix brethren, despite not having a full kernel under them. Kernel services > were provided by different libraries that did filesystem things, block driver things, network things, etc > in a similar way to Unix, but with a much reduced footprint. > > initramfs, as has been mentioned elsewhere, is pretty much a Linux invention. It was designed to > 'punt' on the choose where to load things from and have a very minimal interface between the boot > loader and the system. In time, it grew to support more interfaces, more ways of loading, and better > ways to mount something that you could then 'pivot' onto. Few other unix systems went this route, though > many adopted some variation on the pivot_root functionality. Linux has moved beyond the pivot root after > having booted the correct kernel into being able to take over the machine early in, say, UEFI startup with > a minimal kernel and initramfs that just knows how to load the next kernel. They skipped the complex boot > loader stage, and went straight to the 'run linux earlier' stage which is how things like LinuxBoot, coreboot > and others have put the boot logic into bash scripts. The ability to 'kexec' a kernel and replace the current > running kernel originated in the 'non-stop' world that wanted to reduce downtime. Now, it's used to reduce > firmware complexity by eliminating large swaths of UEFI from the boot process, but also generalizes in > the embedded space. > > FreeBSD, from around FreeBSD 2 (1995 or so), had /rescue which largely took over form the stand alone environment > for the repair duties of things. FreeBSD also adopted a more complex boot loader that would load the > kernel, modules, set tunables, etc prior to kicking off the kernel. Between /rescue having all the tools needed > to repair bad updates, repair failing disks, get that one last backup before the drive is dead while you wait > for the new drive to be delivered, etc, and /boot/loader being able to script loading the kernel while the BIOS > was still around so the need for drivers in the loader was lessened. However, as the BIOS evolved into UEFI > and FreeBSD pushed into the embedded space whose firmware provided a less rich environment to the boot > loader, so it was able to load things off fewer and fewer devices, it became clear that it would need a pivot > root feature to allow it to boot all the way into FreeBSD, load some drivers from an included ram disk, and then > use that mount a new root and then 'reroot' to that by killing everything and running init from that new root. > FreeBSD also moved from Forth to Lua in its scripting language for the boot loader, giving 'pre boot' > environment support better features. I also added the ability to use FreeBSD boot loader as a Linux binary > to load FreeBSD and its metadata from a LinuxBoot environment. Finally, FreeBSD has 'spun out' and > generalized the /rescue feature to allow creation of any 'BeastyBox' environment, similar to what you get > in a busy box, or clone, environment. This environment, though, is meant in large part on both Linux and > FreeBSD to be in constrained environments where a full install is prohibitive (even those that never pivot > to something more, like ap, routers and nas boxes). > > NetBSD retains many of the old BSD stand-alone programs that started on the vax. I've not studied things > beyond noticing this. OpenBSD is similar. Their boot chain is a bit simpler than FreeBSD's, though there's > noises about porting FreeBSD's boot there. There's a port of /boot/loader to illumos too, but I don't know > if it is the default, or just available. So I'll not chat about it more. > > So the original 'standalone' environment where you had one program running on a system has evolved > into either a rich boot loader environment that lets one do a lot to decide what kernel to load, or towards > having a minimal selection of unix programs faster and using /bin/sh or similar to do scripting. These > reduced environments are often called standalone, though all they share just the name with the earlier > 'stand' programs: they are full unix programs, but with reduced feature sets and 'linker magic' to package > them in a way that's faster, smaller, etc (eg all in one binary). FreeBSD's boot loader is an outgrowth > of the original standalone env, by way of a port of NetBSD's libsa. > > I suspect in the future, we'll see more and more of a trend for low-level init and then handing off to some > built-in kernel (be it Linux, BSD-based (there's now kexec), or whatever) to reuse more of the vetted code > rather than re-inventing Unix inside the boot loader (which is a valid criticism of FreeBSD's boot loader, > though it's rich feature set is what you get for the complexity). > > Does that answer the prompt? Should I try to make this into more of a retrospective paper and actually > do the research on the areas I was hand-wavy about? > > Warner ^ permalink raw reply [flat|nested] 27+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-05 11:15 ` Paul Ruizendaal via TUHS @ 2023-09-05 14:15 ` Clem Cole 2023-09-05 17:03 ` Warner Losh 1 sibling, 0 replies; 27+ messages in thread From: Clem Cole @ 2023-09-05 14:15 UTC (permalink / raw) To: Paul Ruizendaal; +Cc: tuhs [-- Attachment #1: Type: text/plain, Size: 4922 bytes --] below.. On Tue, Sep 5, 2023 at 7:16 AM Paul Ruizendaal via TUHS <tuhs@tuhs.org> wrote: > One pitfall I would like to avoid in my own thinking is conflating > “installing” and “booting”, > That is an excellent point. I think that the real question comes back to how 'cold' the 'system' is. Booting is the process of making the processor 'live' including the loading of the 'initial memory contents' (IBM used to call it IPL for Initial Program Load), while 'installation' is setting up a 'system' which 'lives' in the peripherals as long-term storage. When you 'Power-On' a CPU, you often need to IPL a system even if the peripherals have been set up [back in the day of core memory, you might not even have that as the OS was not lost when power was removed]. > The starting point seems to be a setup where a small set of standalone > programs is used to load or repair the bits, as was done for 16-bit unix. > You are solving the 'chicken and egg' issue. Truth is Ken's famous "Reflections on Trusting Trust" comes to bear here. The fact is you are 'setting up' (or restoring) a system for bits that were generated on another. The critical point is that you are setting the initial 'systems' bits into the peripherals for that specific system. You are relying on making copies of those bits from another system. The question is how to get them into the new media. All the standalone system is doing is offering you a minimum way of making a copy of that other system without having access to its hardware directly. > A next conceptual step seems to be where first a very basic system is > installed that is then used for further installation or for repair. This > step seems to have come early, if this 32V install page is reflective of > how it was done back in 1980: > https://gunkies.org/wiki/Installing_32V_on_SIMH The idea to use disk > swap space for this also seems to have come early (and I suppose the > concept lives on in “rescue partitions”). > Newer versions of the system setup scheme offer more and more features, as the options of how you set up might be different from the origin system become greater and greater. Modern OS implementations of the Unix technologies now run on a much more comprehensive range of devices, and frankly, 'IPL' much less might be set up from so many different types of sources, it's not surprising that the IPL schemes and the system setup scheme are a lot more sophisticated. But remember, when you examine the past scheme, you must also consider the constraints of the timeframe (particularly the economics of certain choices). It's not that you could not have built something like some of today's schemes — it was expensive with respect to what was available and, frankly, not wholly necessary. > > Another conceptual step might be where early installer phases run a > different, smaller kernel (or even OS) than the one being installed. There > seems to be much potential for a “wheel of reincarnation” here where as an > installer grows large, a pre-installer is created to load the installer and > then the pre-installer grows large, etc. For booting, this wheel seems to > have turned about 5 times in current Linux. Installing that from scratch on > a fully blank SBC (without prepping a removable disk on another computer) > also appears to have 4 or 5 revolutions. That is one revolution every 5 > years for the 25 years from the late seventies to the early 2000’s. > The circle started long before that. For instance, the boot/IPL got more flexible (like Sam's autoconfiguration work developed for VAX). Why? Because the HW configurations had begun to branch and get bushy. Even with the PDP-11's peripheral set in V7, it was getting more complex. But those kernel configurations were statically linked. Sun added dynamic linking, making both IPL, much less setup/installation more complex. But frankly, to do that on an earlier machine took a lot of work. BSD2.9 did backport much of the VAX work but not all of it. It was often too difficult, and the 'gain' for the effort was low. So my modern UNIX implementations, be it macOS, *BSD, or the Linux family, the schemes have matured and been recreated over and over. For instance, you can look at what Apple's done with its system install scheme. It hides the setup/tear down as of the root and then makes a read-only root system under the covers. This is both a blessing and a curse. Sure, it helps make things a lot more secure for them and basic users, but that choice breaks traditional admin scripts [which drives me nuts think - /etc/periodic] because the new Apple system implementors don't understand why the core system works the way it does and thus those of us that have scripts have worked since V7 in the late 1970s, now don't. Clem ᐧ [-- Attachment #2: Type: text/html, Size: 9204 bytes --] ^ permalink raw reply [flat|nested] 27+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-05 11:15 ` Paul Ruizendaal via TUHS 2023-09-05 14:15 ` Clem Cole @ 2023-09-05 17:03 ` Warner Losh 1 sibling, 0 replies; 27+ messages in thread From: Warner Losh @ 2023-09-05 17:03 UTC (permalink / raw) To: Paul Ruizendaal; +Cc: tuhs [-- Attachment #1: Type: text/plain, Size: 13663 bytes --] On Tue, Sep 5, 2023 at 5:15 AM Paul Ruizendaal <pnr@planet.nl> wrote: > > Does that answer the prompt? Should I try to make this into more of a > retrospective paper and actually > > do the research on the areas I was hand-wavy about? > > That certainly answered the prompt, much appreciated the walkthrough. > Currently just trying to get a view of the field and to collect > recollections on how it was done back in the day. > > Part of it is finding a conceptual framework that can make sense of it all. > Yes. V7 and other early unix had a 'stand alone' framework. So that's one piece of the puzzle. It could be used for anything (I've seen private diagnostic stand alone programs at Solbourne that would do more extensive tests than the boot roms did for RMA and manufacturing line acceptance). But as they come down to us from AT&T, they are just the bits needed to install a system from scratch (or in some cases fix a system by booting the distribution tapes). > One pitfall I would like to avoid in my own thinking is conflating > “installing” and “booting”, even though the two seem related (one loads > bits into permanent storage, the other into volatile storage; both are > built around incrementally adding capability). When variable hardware comes > into play, the two mix even more. Also related is the topic of recovery. > There's "booting" and then there's "bootstrapping". booting happens all the time, but installation happens rarely. The problem is that the install process for V7 and earlier systems was to load a series of programs into memory from tape that did each step of the process to stay within the 64k limits imposed by the platform. That process was done by a series of standalone programs that could, in some cases, act as system repair for certain cases of 'afu' system. > The starting point seems to be a setup where a small set of standalone > programs is used to load or repair the bits, as was done for 16-bit unix. > Yea, that's where stand came from originally: An environment that could run a limited program w/o booting the kernel... > A next conceptual step seems to be where first a very basic system is > installed that is then used for further installation or for repair. This > step seems to have come early, if this 32V install page is reflective of > how it was done back in 1980: > https://gunkies.org/wiki/Installing_32V_on_SIMH The idea to use disk > swap space for this also seems to have come early (and I suppose the > concept lives on in “rescue partitions”). > Yes. It was quite common for the next step to be when demand paging became a thing. Once demand paging became a thing and you didn't need a dedicated swap partition, you could 'borrow' the installed system's swap partition (or what would become the installed system's swap partition) to have a richer set of tools. This would use otherwise wasted space for the install, then you'd overwrite it with whatever workload you put on your system. > Another conceptual step might be where early installer phases run a > different, smaller kernel (or even OS) than the one being installed. There > seems to be much potential for a “wheel of reincarnation” here where as an > installer grows large, a pre-installer is created to load the installer and > then the pre-installer grows large, etc. For booting, this wheel seems to > have turned about 5 times in current Linux. Installing that from scratch on > a fully blank SBC (without prepping a removable disk on another computer) > also appears to have 4 or 5 revolutions. That is one revolution every 5 > years for the 25 years from the late seventies to the early 2000’s. > Indeed, the whole linuxboot saga where you boot a minimal linux kernel that has a shell script (or similar) that decides where to load the complete kernel from is also interesting (doubly so to me because it means I can sneak in at that point and load FreeBSD onto the system). > Then there is the question of where in the "installer stack" to stop. My > current interest excludes (precursors to) package managers, containers, etc. > Yes. For me, the 'preboot environment' is a better mental framework to look at it with. Or 'pre-kernel' or 'non-kernel' environments. Though that paradigm runs out of steam in the 90s when people started booting kernels with bundled ram disks (in various flavors) to do the installation to allow boot medium to be used to load additional data (eg the system). The read/write nature of RAM made this vector useful, even when CDROMs appeared on the scene. It was easier to run off a ram disk than off the CDROM because you don't have to have symlinks to a (small) RAM disks for the bits of the system that needed to be read/write during the install process... The 'standalone' stuff, as typified by the src/stand directory, was all pre-kernel or non-kernel use-cases. Sometimes it was used to build a bit of code that had to be small, but was also used to build code to load the kernel (even dating back to the original boot loader for V7, though I'm relying on my memory which may be confusing that with the 2BSD bootstrap)... The stuff that was 'standalone-but-on-a-unix-kernel' came later. It might also be amusing to note: I once contemplated porting V7 unix to replace FreeBSD's stand environment to replace the various ad-hock hacks that had grown up. But I found that those hacks were smaller in the end, and needed less code space to support more things (excluding crypto and compression) than the V7 did once you started looking at modifying it to support multiple filesystem types, adding a network stack etc. While V7 is nice and clean, it's also immature in terms of features needed for a modern loader... Warner > In short: much to ponder, thanks to all for sharing recollections. > > > > On 4 Sep 2023, at 19:07, Warner Losh <imp@bsdimp.com> wrote: > > > > > > > > On Mon, Sep 4, 2023 at 3:58 AM Paul Ruizendaal via TUHS <tuhs@tuhs.org> > wrote: > > > > Recently, I was looking into the “Das U-Boot” boot loader package. > Summarised with great simplification, u-boot bundles device drivers, file > systems, commands and a Bourne-like shell into a standalone package. > Normally it auto-runs a script that brings up a system, but when used in > interactive mode it allows a great deal of poking around. > > > > It made me think of the “standalone” set of programs for installing > early Unix. On 16-bit understandably each basic command has to be a > separate standalone program, but after the shift to 32-bit bundling more > functionality in a single binary would have become possible. > > > > How did the Unix “standalone” package evolve in the 80’s, both in the > research and BSD lineages? Is there any retrospective paper about that? Or > is it a case of “Use the source, Luke”? > > > > The stand package continued in research and BSD to be those programs > needed to install and/or recover > > badly damaged systems. You could create a new file system, copy a file > from the tape to a partition, etc. > > You couldn't do general scripting with this, by and large. > > > > Originally, they were tape programs. This made sense because of its > original focus. In time, some systems > > could load the stand alone programs instead of the kernel, but they > continued the original focus. > > > > This is, imho, due in large part due to the miniroot. The miniroot > evolved into both a full-enough system > > to do the installation scripts in shell instead of C (Venix, at least, > had their install program written in C). > > You'd copy the minroot to swap and then install the system. But a number > of additional programs were > > placed into the miniroot so you could do some limited filesystem repair, > file editing, etc. > > > > In addition, many vendor's ROMs grew in complexity. Solbourne's ROMs, > for example, could do basic > > repair of UFS (clri level, not fsck level), and copy files from one > place to another. I often recovered a > > Solbourne system I screwed up by attaching an external SCSI drive that > had a known good kernel, > > init, etc. > > > > The 'stand' environment was a whole set of tools that could be used to > build stand-alone programs that > > shared much code of their full unix brethren, despite not having a full > kernel under them. Kernel services > > were provided by different libraries that did filesystem things, block > driver things, network things, etc > > in a similar way to Unix, but with a much reduced footprint. > > > > initramfs, as has been mentioned elsewhere, is pretty much a Linux > invention. It was designed to > > 'punt' on the choose where to load things from and have a very minimal > interface between the boot > > loader and the system. In time, it grew to support more interfaces, more > ways of loading, and better > > ways to mount something that you could then 'pivot' onto. Few other unix > systems went this route, though > > many adopted some variation on the pivot_root functionality. Linux has > moved beyond the pivot root after > > having booted the correct kernel into being able to take over the > machine early in, say, UEFI startup with > > a minimal kernel and initramfs that just knows how to load the next > kernel. They skipped the complex boot > > loader stage, and went straight to the 'run linux earlier' stage which > is how things like LinuxBoot, coreboot > > and others have put the boot logic into bash scripts. The ability to > 'kexec' a kernel and replace the current > > running kernel originated in the 'non-stop' world that wanted to reduce > downtime. Now, it's used to reduce > > firmware complexity by eliminating large swaths of UEFI from the boot > process, but also generalizes in > > the embedded space. > > > > FreeBSD, from around FreeBSD 2 (1995 or so), had /rescue which largely > took over form the stand alone environment > > for the repair duties of things. FreeBSD also adopted a more complex > boot loader that would load the > > kernel, modules, set tunables, etc prior to kicking off the kernel. > Between /rescue having all the tools needed > > to repair bad updates, repair failing disks, get that one last backup > before the drive is dead while you wait > > for the new drive to be delivered, etc, and /boot/loader being able to > script loading the kernel while the BIOS > > was still around so the need for drivers in the loader was lessened. > However, as the BIOS evolved into UEFI > > and FreeBSD pushed into the embedded space whose firmware provided a > less rich environment to the boot > > loader, so it was able to load things off fewer and fewer devices, it > became clear that it would need a pivot > > root feature to allow it to boot all the way into FreeBSD, load some > drivers from an included ram disk, and then > > use that mount a new root and then 'reroot' to that by killing > everything and running init from that new root. > > FreeBSD also moved from Forth to Lua in its scripting language for the > boot loader, giving 'pre boot' > > environment support better features. I also added the ability to use > FreeBSD boot loader as a Linux binary > > to load FreeBSD and its metadata from a LinuxBoot environment. Finally, > FreeBSD has 'spun out' and > > generalized the /rescue feature to allow creation of any 'BeastyBox' > environment, similar to what you get > > in a busy box, or clone, environment. This environment, though, is meant > in large part on both Linux and > > FreeBSD to be in constrained environments where a full install is > prohibitive (even those that never pivot > > to something more, like ap, routers and nas boxes). > > > > NetBSD retains many of the old BSD stand-alone programs that started on > the vax. I've not studied things > > beyond noticing this. OpenBSD is similar. Their boot chain is a bit > simpler than FreeBSD's, though there's > > noises about porting FreeBSD's boot there. There's a port of > /boot/loader to illumos too, but I don't know > > if it is the default, or just available. So I'll not chat about it more. > > > > So the original 'standalone' environment where you had one program > running on a system has evolved > > into either a rich boot loader environment that lets one do a lot to > decide what kernel to load, or towards > > having a minimal selection of unix programs faster and using /bin/sh or > similar to do scripting. These > > reduced environments are often called standalone, though all they share > just the name with the earlier > > 'stand' programs: they are full unix programs, but with reduced feature > sets and 'linker magic' to package > > them in a way that's faster, smaller, etc (eg all in one binary). > FreeBSD's boot loader is an outgrowth > > of the original standalone env, by way of a port of NetBSD's libsa. > > > > I suspect in the future, we'll see more and more of a trend for > low-level init and then handing off to some > > built-in kernel (be it Linux, BSD-based (there's now kexec), or > whatever) to reuse more of the vetted code > > rather than re-inventing Unix inside the boot loader (which is a valid > criticism of FreeBSD's boot loader, > > though it's rich feature set is what you get for the complexity). > > > > Does that answer the prompt? Should I try to make this into more of a > retrospective paper and actually > > do the research on the areas I was hand-wavy about? > > > > Warner > > [-- Attachment #2: Type: text/html, Size: 15791 bytes --] ^ permalink raw reply [flat|nested] 27+ messages in thread
end of thread, other threads:[~2023-09-11 22:06 UTC | newest] Thread overview: 27+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2023-09-04 14:44 [TUHS] Unix install & "standalone" package Norman Wilson 2023-09-04 14:55 ` [TUHS] " Vincenzo Nicosia 2023-09-04 17:20 ` Warner Losh 2023-09-04 19:05 ` Clem Cole 2023-09-05 17:03 ` Paul Winalski 2023-09-05 18:02 ` Clem Cole 2023-09-04 19:59 ` Theodore Ts'o 2023-09-04 23:51 ` Warner Losh 2023-09-04 17:18 ` Warner Losh 2023-09-04 22:10 ` Steffen Nurpmeso 2023-09-05 15:53 ` Steffen Nurpmeso 2023-09-06 17:50 ` Warner Losh 2023-09-07 0:11 ` Steffen Nurpmeso 2023-09-07 16:05 ` Warner Losh 2023-09-08 14:58 ` Theodore Ts'o 2023-09-08 13:56 ` Michael Kjörling 2023-09-08 23:38 ` Steffen Nurpmeso 2023-09-09 22:43 ` Steffen Nurpmeso 2023-09-11 4:10 ` Theodore Ts'o 2023-09-11 22:05 ` Steffen Nurpmeso 2023-09-05 1:07 ` Jonathan Gray -- strict thread matches above, loose matches on Subject: below -- 2023-09-04 9:57 [TUHS] " Paul Ruizendaal via TUHS 2023-09-04 14:53 ` [TUHS] " emanuel stiebler 2023-09-04 17:07 ` Warner Losh 2023-09-04 18:21 ` Dan Cross 2023-09-05 11:15 ` Paul Ruizendaal via TUHS 2023-09-05 14:15 ` Clem Cole 2023-09-05 17:03 ` Warner Losh
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).