* [TUHS] Unix install & "standalone" package @ 2023-09-04 14:44 Norman Wilson 2023-09-04 14:55 ` [TUHS] " Vincenzo Nicosia ` (3 more replies) 0 siblings, 4 replies; 22+ messages in thread From: Norman Wilson @ 2023-09-04 14:44 UTC (permalink / raw) To: tuhs I don't remember any special many-programs-in-one binary like busybox in any Unix from the days when Unix was simple enough for me to understand. That covers the entire lifetime of the Research systems, but also System V and the BSDs and their sundry offspring up into at least the 1990s. I'm pretty sure OpenBSD at least still has nothing like busybox. The nearest thing was to make sure certain programs were linked statically (or existed in alternate statically- linked versions) so they would work before shared libraries were available. (It seems to be common wisdom that `sbin' means `system bin' these days, but I remember once, long ago, being told it stood for `static bin'.) Perhaps the question to ask is why such a magic program is needed at all. Is it just because programs like the shell have become so large and unwieldy that they won't fit in a small environment suitable for loading into an initramfs? Older UNIXes, even on the VAX, didn't use an initramfs. the boot code had just enough understanding of devices and file systems to load the kernel and to point it at the real root file system. The VAX hardware designers had a clever scheme on many (but, strangely, not all) VAX variants by which the hardware had several little boot ROMs, each containing a bare-bones read-only device driver for a particular device along with a few instructions to read the first sector into memory and start it, with pointers to the boot-rom driver and device ID in specified registers. That was enough to support a device-independent Unix boot block that could read unix (or another file name typed on the console) from the root of the file system at the start of the disk. I know it was because I wrote such a boot block, though I don't know whether anyone else did. (Other systems, I think, just used it to load /boot, a larger and more-capable program.) Maybe it was just that the boot environment was simpler in older systems, without the need to load kernel modules or support multiple locations and means of access for the root? Norman Wilson Toronto ON ^ permalink raw reply [flat|nested] 22+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-04 14:44 [TUHS] Unix install & "standalone" package Norman Wilson @ 2023-09-04 14:55 ` Vincenzo Nicosia 2023-09-04 17:20 ` Warner Losh 2023-09-04 17:18 ` Warner Losh ` (2 subsequent siblings) 3 siblings, 1 reply; 22+ messages in thread From: Vincenzo Nicosia @ 2023-09-04 14:55 UTC (permalink / raw) To: tuhs rather, back then you didn't need the same kernel to run on a wide variety of hardware, with all the possible different combinations of peripherals, requiring all sorts of different drivers. I think that's the only real reason why initramfs came to existence: allowing a selection of kernel modules to be loaded at init time, depending on the hardware at disposal on that machine. Then things went south, and more recent initramfs have everything and the kitchen sink. But that's another story. HND Enzo Nicosia -- ^ permalink raw reply [flat|nested] 22+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-04 14:55 ` [TUHS] " Vincenzo Nicosia @ 2023-09-04 17:20 ` Warner Losh 2023-09-04 19:05 ` Clem Cole 2023-09-04 19:59 ` Theodore Ts'o 0 siblings, 2 replies; 22+ messages in thread From: Warner Losh @ 2023-09-04 17:20 UTC (permalink / raw) To: Vincenzo Nicosia; +Cc: tuhs [-- Attachment #1: Type: text/plain, Size: 1152 bytes --] On Mon, Sep 4, 2023 at 8:56 AM Vincenzo Nicosia <katolaz@freaknet.org> wrote: > rather, back then you didn't need the same kernel to run on a wide > variety of hardware, with all the possible different combinations of > peripherals, requiring all sorts of different drivers. > > I think that's the only real reason why initramfs came to existence: > allowing a selection of kernel modules to be loaded at init time, > depending on the hardware at disposal on that machine. Then things went > south, and more recent initramfs have everything and the kitchen sink. > But that's another story. Yea, it was an effort to move mounting of root out of the kernel. The earliest scripts just mounted the right disk and moved on, and didn't load any new drivers: they just had the logic to pick the desired root. But at the same time, there were a lot of people that were running on 4MB and 8MB systems that noticed they could put all the router software in the initramfs and never pivot to something else and they could have quite the product with that. And those were the first few bricks that paved the road to hell :) Warner [-- Attachment #2: Type: text/html, Size: 1584 bytes --] ^ permalink raw reply [flat|nested] 22+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-04 17:20 ` Warner Losh @ 2023-09-04 19:05 ` Clem Cole 2023-09-05 17:03 ` Paul Winalski 2023-09-04 19:59 ` Theodore Ts'o 1 sibling, 1 reply; 22+ messages in thread From: Clem Cole @ 2023-09-04 19:05 UTC (permalink / raw) To: Warner Losh; +Cc: tuhs [-- Attachment #1.1: Type: text/plain, Size: 4871 bytes --] I can not add too much to what happened later, but I think modern users don't get it why the things like the standalone system were needed back in the day. There were a few simple differences thanks to Moore's law, modern folks really may not grok: 1. The number of peripherals that needed to be supported was small [each firm made only a few for each target system] 2. The capacity/capabilities of those peripherals are minimal compared to today's world 3. Cheap ROMs, as we know it, were coming on the scene [the EPROM is only invented in 1971] One-time programmable and Mask ROMs were around but they were expensive and small. I hope Warren's emailer lets this pass through. Here is a picture of the original root 'ROM' for the PDP-11 M792-xx board, which gives you 32 words (64 bytes) of memory. It used individual diodes in or out arranged an array to represent individual bits with a diode present being a '1' and '0' if missing. As the Gunkies web page suggests, the board came in two versions - customer programmable which was fully populated with diodes, and the user removed unneeded diodes to program it. The other variants were programmed with the designed boot functions - designated M792-Yx (where 'x' is a capital letter, starting with 'A') [image: Read_Only_Memory_DEC_M792_Diode_Matrix_ROM_Aceware_Attribution.jpg] Also, remember that a $3-5k/drive RK05 is only 2.5Mbytes [4872 512 byte 'sectors' *a.k.a*. disk blocks -- 2 hd/disk * 15 sec/cyl * 203 cyl/disk], which is why we had separated /bin and /usr/bin and /lib and /usr/lib. Plus, as Norman points our dynamic linking only comes later, so all of these are static bound programs (*i.e.,* Sun create /bin and /sbin to separate those programs). The whole idea was what was in root was enough to get the system running in /bin, /lib and /etc and everything else was one the mounted file system, which often was another RK05. Many of our V6 and V7 systems ran with 3 RK05 /, /usr /home, and we might have an extra RK05 for /mnt. Tape was the standard thing. DEC tape was cheaper than 9-track, although in my history, we all went for 9-track because it was more portable to other systems, but often used after-market 9-track transports that emulated the DEC ones. Even with the VAX, the front-end runs on an LSI-11 with floppies, so things like microcode were loaded into the VAX and stored in ROM. By then, the RP and RM series had become more normal, and eventually, the RK07 and RL02 replaced the RK05. But we are not talking about substantial capacity in the storage systems. Only ten years later, in the early 1980s, a Fujisu 'Eagle' is just 470Mbytes and costs 12-15K per drive, plus another $6-8K for the controller for your Vax [also often after-market from SI or Emulex]. Sun and Apollo build 'diskless' systems because a 100MB ST-506 style disk was considered too expensive, and was worried about the entry-level price of their systems (I famously made a typo describing how those systems performed BTW with a dyslexic change of the s to c on a whole company email at Masscomp in the mid-1980s - to Sun's credit, it was the best marketing ploy ever - people bought diskless and discovered they sucked and then all added a disk - which cost more $s than the original Masscomp system which was had one built-in). But I digress... Those new workstations did have real ROM chips, unlike the original PDP-11s, but they were small, maybe 8Kbytes of NS2764, so the ROMs were tight and only had enough in them to recognize a couple of peripherals. So to answer more of your question, you build the tools you need that make sense at the time. /stand was a simple solution and was small. For V7 and the original BSD 3 and 4 released, the user needed to read a distribution media, traditionally a 9-track tape, and set up a local disk. The number of peripherals was small. One other thought ... in the case of the PDP-11 and Vaxen, the disks were often dismountable, like the different RK, RL, RP, and RM series drives. We often had more than one system. So once the system was up and running, it was not usual to use a different system with better tools to help repair things if bad stuff occurred. That is also why we often used 9-track tape, just because that was even easier to move back and forth. With the Wokstrations, the disks were generally sealed. Clem A small PS on another note -- I have memories of taking an ST-506 disk with me to France to work with some engineers in Grenoble and explaining to the French authorities that they could not open it up to look inside. BTW: this pre-internet - imagine what we had to do if we flew to Canada, which was taxing SW. Even bringing tapes in and out of Toronto airport could be treacherous. ᐧ [-- Attachment #1.2: Type: text/html, Size: 5769 bytes --] [-- Attachment #2: Read_Only_Memory_DEC_M792_Diode_Matrix_ROM_Aceware_Attribution.jpg --] [-- Type: image/jpeg, Size: 154411 bytes --] ^ permalink raw reply [flat|nested] 22+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-04 19:05 ` Clem Cole @ 2023-09-05 17:03 ` Paul Winalski 2023-09-05 18:02 ` Clem Cole 0 siblings, 1 reply; 22+ messages in thread From: Paul Winalski @ 2023-09-05 17:03 UTC (permalink / raw) To: Clem Cole; +Cc: tuhs On 9/4/23, Clem Cole <clemc@ccc.com> wrote: > > Even with the VAX, the front-end runs on an LSI-11 with floppies, so things > like microcode were loaded into the VAX and stored in ROM. I think you meant either "stored in RAM" or "not stored in ROM". The LSI-11 front end for the VAX-11/780 (and other early VAXen?) was connected to the SBI (Synchronous Backplane Interconnect--the system bus) and performed three main functions: microcode load and system bootstrap initiation, interface for the system console (originally a LA-36 dot matrix terminal), and some system diagnostics. CPU microcode was stored on a 5" floppy disk connected to the LSI-11 and was loaded into RAM at system power-up. -Paul W. ^ permalink raw reply [flat|nested] 22+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-05 17:03 ` Paul Winalski @ 2023-09-05 18:02 ` Clem Cole 0 siblings, 0 replies; 22+ messages in thread From: Clem Cole @ 2023-09-05 18:02 UTC (permalink / raw) To: Paul Winalski; +Cc: tuhs [-- Attachment #1: Type: text/plain, Size: 398 bytes --] On Tue, Sep 5, 2023 at 1:03 PM Paul Winalski <paul.winalski@gmail.com> wrote: > On 9/4/23, Clem Cole <clemc@ccc.com> wrote: > > > > Even with the VAX, the front-end runs on an LSI-11 with floppies, so > things > > like microcode were loaded into the VAX and stored in ROM. > > I think you meant either "stored in RAM" or "not stored in ROM". Yep -- thanks - dyslexia-r-me. ᐧ [-- Attachment #2: Type: text/html, Size: 1330 bytes --] ^ permalink raw reply [flat|nested] 22+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-04 17:20 ` Warner Losh 2023-09-04 19:05 ` Clem Cole @ 2023-09-04 19:59 ` Theodore Ts'o 2023-09-04 23:51 ` Warner Losh 1 sibling, 1 reply; 22+ messages in thread From: Theodore Ts'o @ 2023-09-04 19:59 UTC (permalink / raw) To: Warner Losh; +Cc: tuhs On Mon, Sep 04, 2023 at 11:20:31AM -0600, Warner Losh wrote: > > Yea, it was an effort to move mounting of root out of the > kernel. The earliest scripts just mounted the right disk and moved > on, and didn't load any new drivers: they just had the logic to pick > the desired root. But at the same time, there were a lot of people > that were running on 4MB and 8MB systems that noticed they could put > all the router software in the initramfs and never pivot to > something else and they could have quite the product with that. And > those were the first few bricks that paved the road to hell :) The other reason why you might need something like an initial ramdisk is if you don't yet *have* a root file system on the hardware, and you are trying to install a system using TFTP boot. At that point, you have three choices: (a) Use a NFS root (b) Use a read-only network block device (for example, MIT Project Athena had a read-only Remote Virtual Disk which was a network block device that was used to provide a read-only system image used for installation and update, as well as a read-only /usr image). (c) Use a built-in ramdisk. (And here's not just Linus which has adopted this strategy; OpenBSD's installer does this as well.) The other dynamic at play was that ramdisks were needed when you were installing on a 386 PC with, say, 16 megs of memory, a 320 megabyte hard disk, and a *single* 1.44MB floppy --- and no ethernet, because this was for the home PC user, so the best that you had might be a 38kbps dialup link with PPP --- if you were lucky. So putting the kernel on the first floppy disk, and putting a ramdisk image on the second floppy disk, so you could then eject the floppy disk and insert subsequent floppy disks so you could install rest of the installation binaries (including the shell utilities, the C compiler, X windows systems, etc.) and then reboot onto the HDD with a fully set up system, was a pretty natural evolution. And since we had the ramdisk infrastructure, and most users didn't want to configure their own kernel as part of the installation process, that begat kernel modules and the initial ramdisk being used to store the kernel modules. In the very early days, this made Linux *far* more user friendly for non-system programmers to install, compared to say, FreeBSD during that era, which was still stuck in the BSD 4.x days of a generic kernel, followed by a kernel configuration step, etc. These days, given that some enterprise setups what their servers to be installed from a Fibre Channel Storage Area Network, or iSCSI, perhaps using Kerberos to authenticate to the iSCSI target, it makes absolutely a huge amount of sense to have an initial ramdisk as an option. That being said, for a long time, the initial ramdisk was an *option*. You didn't have to use it, if you were willing to have a custom kernel and you are using a device which has a fixed and stable boot-time enumeration. This tended to only be used by people who knew what they were doing, but for example, my file system test appliance VM system doesn't use an initial ramdisk at all. It uses qemu, and has a built-in kernel configuration and build system, and this works perfectly fine booting into the latest Debian stable distribution, with no initial ramdisk used at all: https://github.com/tytso/xfstests-bld/blob/master/Documentation/kvm-quickstart.md - Ted the root filesystem on the second floppy disk (the first floppy disk contained the kernel), which was then copied into a ramdisk, so you could Trying to fit the kernel and the root file system on a single 1.44MB floppy, was (barely) doable, but you certainly wouldn't have space for a C compiler so you build or link your own custom kernel. So one of the first things that we did was to put the kernel on a single floppy disk, and then but the root on a single floppy disk, and then copy the root to the HDD, and then reboot onto the HDD, and then take the installation procedure from there. But that was awkward and ^ permalink raw reply [flat|nested] 22+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-04 19:59 ` Theodore Ts'o @ 2023-09-04 23:51 ` Warner Losh 0 siblings, 0 replies; 22+ messages in thread From: Warner Losh @ 2023-09-04 23:51 UTC (permalink / raw) To: Theodore Ts'o; +Cc: tuhs [-- Attachment #1: Type: text/plain, Size: 5093 bytes --] On Mon, Sep 4, 2023 at 1:59 PM Theodore Ts'o <tytso@mit.edu> wrote: > On Mon, Sep 04, 2023 at 11:20:31AM -0600, Warner Losh wrote: > > > > Yea, it was an effort to move mounting of root out of the > > kernel. The earliest scripts just mounted the right disk and moved > > on, and didn't load any new drivers: they just had the logic to pick > > the desired root. But at the same time, there were a lot of people > > that were running on 4MB and 8MB systems that noticed they could put > > all the router software in the initramfs and never pivot to > > something else and they could have quite the product with that. And > > those were the first few bricks that paved the road to hell :) > > The other reason why you might need something like an initial ramdisk > is if you don't yet *have* a root file system on the hardware, and you > are trying to install a system using TFTP boot. At that point, you > have three choices: > > (a) Use a NFS root > > (b) Use a read-only network block device (for example, MIT Project > Athena had a read-only Remote Virtual Disk which was a network block > device that was used to provide a read-only system image used for > installation and update, as well as a read-only /usr image). > > (c) Use a built-in ramdisk. (And here's not just Linus which has > adopted this strategy; OpenBSD's installer does this as well.) > Yes. FreeBSD did this originally, and some installers still do that. > The other dynamic at play was that ramdisks were needed when you were > installing on a 386 PC with, say, 16 megs of memory, a 320 megabyte > hard disk, and a *single* 1.44MB floppy --- and no ethernet, because > this was for the home PC user, so the best that you had might be a > 38kbps dialup link with PPP --- if you were lucky. > > So putting the kernel on the first floppy disk, and putting a ramdisk > image on the second floppy disk, so you could then eject the floppy > disk and insert subsequent floppy disks so you could install rest of > the installation binaries (including the shell utilities, the C > compiler, X windows systems, etc.) and then reboot onto the HDD with a > fully set up system, was a pretty natural evolution. > Yes. FreeBSD 2.0 definitely did that, and I think FreeBSD 1.0 as well. > And since we had the ramdisk infrastructure, and most users didn't > want to configure their own kernel as part of the installation > process, that begat kernel modules and the initial ramdisk being used > to store the kernel modules. In the very early days, this made Linux > *far* more user friendly for non-system programmers to install, > compared to say, FreeBSD during that era, which was still stuck in the > BSD 4.x days of a generic kernel, followed by a kernel configuration > step, etc. > GENERIC worked fine in the early days: it was small enough that the savings from removing all the extra devices was small... but that's because the number of devices supported was also relatively small. By the time it became an issue FreeBSD did have dynamic loading of drivers, but retained the GENERIC kernel for install (it should have gone to a minimal kernel, but hasn't yet completed the migration: the savings wasn't big enough due to Moore's Law growing machine memories faster than the kernel grew: FreeBSD's kernel only doubled every 2 years...). But yea, it was a bit of a kludge. And FreeBSD grew the ability to configure the morass of ISA devices in the boot loader early on so you could at least boot the generic kernel, and have the loader remember your choices. > These days, given that some enterprise setups what their servers to be > installed from a Fibre Channel Storage Area Network, or iSCSI, perhaps > using Kerberos to authenticate to the iSCSI target, it makes > absolutely a huge amount of sense to have an initial ramdisk as an > option. That being said, for a long time, the initial ramdisk was an > *option*. You didn't have to use it, if you were willing to have a > custom kernel and you are using a device which has a fixed and stable > boot-time enumeration. > Yea. FreeBSD can use it to this day (and many interesting use cases use it), but generally the loader tells the kernel what root filesystem to use. These smarts in the boot loader have also kept the pressure of the vast majority of cases where initramfs makes sense in Linux land. > This tended to only be used by people who knew what they were doing, > but for example, my file system test appliance VM system doesn't use > an initial ramdisk at all. It uses qemu, and has a built-in kernel > configuration and build system, and this works perfectly fine booting > into the latest Debian stable distribution, with no initial ramdisk > used at all: > > > https://github.com/tytso/xfstests-bld/blob/master/Documentation/kvm-quickstart.md > Ah, cool. Every single distro I've run in the past has had an initramfs. I thought it had become no longer an option... I do similar things for my VM setup with FreeBSD :) Warner [-- Attachment #2: Type: text/html, Size: 6738 bytes --] ^ permalink raw reply [flat|nested] 22+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-04 14:44 [TUHS] Unix install & "standalone" package Norman Wilson 2023-09-04 14:55 ` [TUHS] " Vincenzo Nicosia @ 2023-09-04 17:18 ` Warner Losh 2023-09-04 22:10 ` Steffen Nurpmeso 2023-09-05 1:07 ` Jonathan Gray 3 siblings, 0 replies; 22+ messages in thread From: Warner Losh @ 2023-09-04 17:18 UTC (permalink / raw) To: Norman Wilson; +Cc: tuhs [-- Attachment #1: Type: text/plain, Size: 2061 bytes --] On Mon, Sep 4, 2023 at 8:44 AM Norman Wilson <norman@oclsc.org> wrote: > Maybe it was just that the boot environment was simpler > in older systems, without the need to load kernel modules > or support multiple locations and means of access for > the root? > Older systems had fewer choices. On the PDP-11 you had Q-Bus or Unibus. You booted off a small selection of disks that converged to MSCP(?), so there was THE boot environment and THE driver and THE filesystem. And the kernels were often tuned to be exactly what the machine needed. The VAX, Sun, hp, ibm, etc continued this early trend. Though the boot process was still fairly narrow, with a limited list of supported boot devices. In large part this was because the machine was exactly the same every time you booted. But with USB, PC Card, CardBus, ExpressCard, SCSI, Thunderbolt, and a host of other removable technologies with a dizzying array of cards, like PCI, ISA, etc, the game changed. Also, the number of environments that a FreeBSD GENERIC[*] kernel can boot in is huge due to combinatoric explosion due to three or four boot environments, vm alternate startup paths, several supported root file system, BIOS, UEFI, OpenFirmware, u-boot, coreboot, etc as well as several thousand supported boot controller devices, things got complex, despite there being several common interfaces that made things simpler. I'm running a summer of code project this year to help tame the combinatoric explosion to provide better test coverage because though attempts were made to make things the same, variations exist that can cause unexpected breakage in different environments. Warner [*] I use GENERIC here as a catch all, including the more recent MINIMAL kernels that try to include just the core functionality, and omit the vast majority of drivers. Linux has similar issues, but more, and also solves them in many interesting ways... But I'm the FreeBSD boot loader guy in large part (though there's others that work on it), not a Linux person. [-- Attachment #2: Type: text/html, Size: 2719 bytes --] ^ permalink raw reply [flat|nested] 22+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-04 14:44 [TUHS] Unix install & "standalone" package Norman Wilson 2023-09-04 14:55 ` [TUHS] " Vincenzo Nicosia 2023-09-04 17:18 ` Warner Losh @ 2023-09-04 22:10 ` Steffen Nurpmeso 2023-09-05 15:53 ` Steffen Nurpmeso 2023-09-05 1:07 ` Jonathan Gray 3 siblings, 1 reply; 22+ messages in thread From: Steffen Nurpmeso @ 2023-09-04 22:10 UTC (permalink / raw) To: norman; +Cc: tuhs Norman Wilson wrote in <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org>: |I don't remember any special many-programs-in-one binary |like busybox in any Unix from the days when Unix was simple |enough for me to understand. That covers the entire lifetime |of the Research systems, but also System V and the BSDs and |their sundry offspring up into at least the 1990s. ... |Perhaps the question to ask is why such a magic program is |needed at all. Is it just because programs like the shell |have become so large and unwieldy that they won't fit in |a small environment suitable for loading into an initramfs? AlpineLinux as used on my vserver has busybox by default and can cover most utitilities like that. The lead developer Copa once said something like "The idea is you install explicitly [if you want something better]". (It is a symlink farm that is selectively replaced by installing "real" packages iirc.) For my laptop it allows me easy boot management. To save you the chatter ("Chatten" is the name of my tribe .. most likely; could be Franken, Sueben .. and you know how it is): this approach is much easier and smaller than having lots of static binaries to copy around etc. I do not use secure boot, i have on EFI only a kernel, busybox and cryptsetup, and scripts (the laptop is named "kent"):: ... drwxr-xr-x 4 root root 4096 Jul 15 2021 EFI/ ... -rwxr-xr-x 1 root root 272 Feb 1 2022 kent.sh* -rwxr-xr-x 1 root root 313 Feb 1 2022 kent-direct.sh* drwxr-xr-x 1 root root 252 Oct 9 2022 ../ -rwxr-xr-x 1 root root 4596 Feb 4 2023 linux-init-s1.sh* -rwxr-xr-x 1 root root 3646 Feb 4 2023 linux-init-lib.sh* -rwxr-xr-x 1 root root 5480120 Feb 11 2023 cryptsetup.static* -rwxr-xr-x 1 root root 1978368 Aug 15 18:51 busybox.static* -rwxr-xr-x 1 root root 10112672 Aug 26 18:44 ideapad-stage1.efi* So kent.sh can be init(8) for the ideapad-stage1.efi Linux kernel started via EFI as setup via efibootmgr(8) Boot0001* kent HD(1,GPT,5d6d756b-5de2-4e5d-b043-8d4ae1bb6eb0,0x800,0x82000)/File(\ideapad-stage1.efi)root=/dev/nvme0n1p1 rootfstype=vfat init=/kent.sh #!/busybox.static sh #@ kent, step 1., via EFI. PART_ROOT=/dev/nvme0n1p8 ROOT_DECRYPT='-t btrfs -o defaults,subvol=/crux/kent/root' PART_ROOT1=/dev/nvme0n1p8 ROOT_DECRYPT1='-t btrfs -o defaults,subvol=/crux/kent/root.old' INIT_S2=/boot/kent-2.sh . /linux-init-s1.sh and that allows me to unlock the harddisk. We then boot via $INIT_S2 and kexec(8) a kernel from the encrypted harddisk, so no code from EFI partition keeps on running. (We byte-compare the data from EFI with equal /boot/ files after booting the real system.) This allows nice and easy properties: only three files to track (cryptsetup, busybox, kernel), almost same set of files in /boot/ and /media/efi aka EFI. And ideapad-stage1.efi is the same kernel that later runs, but later we have also additional dynamic modules available. Ie, every few weeks i copy /boot/ideapad-6_1.efi over to be the new -stage1.efi. --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) ^ permalink raw reply [flat|nested] 22+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-04 22:10 ` Steffen Nurpmeso @ 2023-09-05 15:53 ` Steffen Nurpmeso 2023-09-06 17:50 ` Warner Losh 2023-09-08 13:56 ` Michael Kjörling 0 siblings, 2 replies; 22+ messages in thread From: Steffen Nurpmeso @ 2023-09-05 15:53 UTC (permalink / raw) To: norman; +Cc: tuhs Steffen Nurpmeso wrote in <20230904221059.sF2G0%steffen@sdaoden.eu>: |Norman Wilson wrote in | <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org>: ... ||Perhaps the question to ask is why such a magic program is ||needed at all. Is it just because programs like the shell ||have become so large and unwieldy that they won't fit in ||a small environment suitable for loading into an initramfs? ... |For my laptop it allows me easy boot management. ... | -rwxr-xr-x 1 root root 4596 Feb 4 2023 linux-init-s1.sh* | -rwxr-xr-x 1 root root 3646 Feb 4 2023 linux-init-lib.sh* | -rwxr-xr-x 1 root root 5480120 Feb 11 2023 cryptsetup.static* | -rwxr-xr-x 1 root root 1978368 Aug 15 18:51 busybox.static* | -rwxr-xr-x 1 root root 10112672 Aug 26 18:44 ideapad-stage1.efi* Only to add that this is because of Linux and the way it is doing things. If i would use FreeBSD on bare metal, then i would have an EFI boot loader on EFI that knows (only) enough to ask for passphrase (correct me if i am wrong), and can then boot the kernel from FFS or ZFS. (You have to choose dedicated ZFS boot loader iirc, but despite that...) I know GRUB (and maybe other) Linux bootloaders can do all that, but they are huge, are badly maintained, or under-documented, let alone with local manuals, and i am too stupid to configure them (due to all that). refind is ok, however. But.. be aware of typos in the configuration.. But anyhow. With an EFI_STUB Linux kernel i can save me all that, with busybox i get a complete environment (i then even create an initrd in /boot/ on the fly so i do not have to type the password a second time, that can (optionally) be cached, and is, actually -rw------- 1 root root 4495987 May 29 16:29 .kent.initrd.0 Unfortunately cryptsetup is needed even though, i think, the kernel has anything needed; you just cannot access it. cryptsetup is only needed for "$cs open $PART_ROOT p_root --key-file -". Of course i am no real Linux expert but only a do-it-yourself guy. busybox allows me to manage this easily, to answer your question. --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) ^ permalink raw reply [flat|nested] 22+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-05 15:53 ` Steffen Nurpmeso @ 2023-09-06 17:50 ` Warner Losh 2023-09-07 0:11 ` Steffen Nurpmeso 2023-09-08 13:56 ` Michael Kjörling 1 sibling, 1 reply; 22+ messages in thread From: Warner Losh @ 2023-09-06 17:50 UTC (permalink / raw) To: Norman Wilson, tuhs [-- Attachment #1: Type: text/plain, Size: 3147 bytes --] On Tue, Sep 5, 2023 at 9:53 AM Steffen Nurpmeso <steffen@sdaoden.eu> wrote: > Steffen Nurpmeso wrote in > <20230904221059.sF2G0%steffen@sdaoden.eu>: > |Norman Wilson wrote in > | <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org>: > ... > ||Perhaps the question to ask is why such a magic program is > ||needed at all. Is it just because programs like the shell > ||have become so large and unwieldy that they won't fit in > ||a small environment suitable for loading into an initramfs? > ... > |For my laptop it allows me easy boot management. > ... > | -rwxr-xr-x 1 root root 4596 Feb 4 2023 linux-init-s1.sh* > | -rwxr-xr-x 1 root root 3646 Feb 4 2023 linux-init-lib.sh* > | -rwxr-xr-x 1 root root 5480120 Feb 11 2023 cryptsetup.static* > | -rwxr-xr-x 1 root root 1978368 Aug 15 18:51 busybox.static* > | -rwxr-xr-x 1 root root 10112672 Aug 26 18:44 ideapad-stage1.efi* > > Only to add that this is because of Linux and the way it is doing > things. If i would use FreeBSD on bare metal, then i would have > an EFI boot loader on EFI that knows (only) enough to ask for > passphrase (correct me if i am wrong), and can then boot the > kernel from FFS or ZFS. (You have to choose dedicated ZFS boot > loader iirc, but despite that...) > No, you don't have to choose the dedicated ZFS boot loader, at least not anymore. Also, you can use boot1.efi to load loader.efi from the root filesystem to load the kernel, or you could use loader.efi directly on the ESP to load the kernel. boot1 barely knows anything (and has only one choice of what to boot). loader.efi is the full deal, and can do rather a lot of sophisticated things. > I know GRUB (and maybe other) Linux bootloaders can do all that, > but they are huge, are badly maintained, or under-documented, let > alone with local manuals, and i am too stupid to configure them > (due to all that). refind is ok, however. But.. be aware of > typos in the configuration.. > > But anyhow. With an EFI_STUB Linux kernel i can save me all that, > with busybox i get a complete environment (i then even create an > initrd in /boot/ on the fly so i do not have to type the password > a second time, that can (optionally) be cached, and is, actually > > -rw------- 1 root root 4495987 May 29 16:29 .kent.initrd.0 > > Unfortunately cryptsetup is needed even though, i think, the > kernel has anything needed; you just cannot access it. cryptsetup > is only needed for "$cs open $PART_ROOT p_root --key-file -". > Of course i am no real Linux expert but only a do-it-yourself guy. > busybox allows me to manage this easily, to answer your question. > You could do that on FreeBSD with a loader.efi that has a ram disk built into it as well, including a 'beastie box' thing that's akin to busybox. It will boot in one step and no no further I/O to get a running system. Others have used this for secure boot and to boot a small ram disk that's later discarded as userland decides what root should be. But it's much less automated than in Linux... Warner [-- Attachment #2: Type: text/html, Size: 4136 bytes --] ^ permalink raw reply [flat|nested] 22+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-06 17:50 ` Warner Losh @ 2023-09-07 0:11 ` Steffen Nurpmeso 2023-09-07 16:05 ` Warner Losh 0 siblings, 1 reply; 22+ messages in thread From: Steffen Nurpmeso @ 2023-09-07 0:11 UTC (permalink / raw) To: Warner Losh; +Cc: tuhs Hello! Warner Losh wrote in <CANCZdfoy0UJyFdvJ_fO4FDks_dfEOhsJo+7p2RORdk6gQWf7qg@mail.gmail.com>: |On Tue, Sep 5, 2023 at 9:53 AM Steffen Nurpmeso <steffen@sdaoden.eu> wrote: |> Steffen Nurpmeso wrote in |> <20230904221059.sF2G0%steffen@sdaoden.eu>: |>|Norman Wilson wrote in |>| <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org>: |> ... |>||Perhaps the question to ask is why such a magic program is |>||needed at all. Is it just because programs like the shell |>||have become so large and unwieldy that they won't fit in |>||a small environment suitable for loading into an initramfs? |> ... |>|For my laptop it allows me easy boot management. ... |> Only to add that this is because of Linux and the way it is doing |> things. If i would use FreeBSD on bare metal, then i would have |> an EFI boot loader on EFI that knows (only) enough to ask for |> passphrase (correct me if i am wrong), and can then boot the |> kernel from FFS or ZFS. (You have to choose dedicated ZFS boot |> loader iirc, but despite that...) | |No, you don't have to choose the dedicated ZFS boot loader, at least not |anymore. Ah! It seems regarding FreeBSD i am stuck in BIOS world, then. (zfsboot and gptzfsboot.) |Also, you can use boot1.efi to load loader.efi from the root filesystem to The former deprecated i see. |load the kernel, or you could use loader.efi directly on the ESP to load |the kernel. boot1 barely knows anything (and has only one choice of |what to boot). loader.efi is the full deal, and can do rather a lot of |sophisticated things. Great. ... |> But anyhow. With an EFI_STUB Linux kernel i can save me all that, |> with busybox i get a complete environment (i then even create an |> initrd in /boot/ on the fly so i do not have to type the password |> a second time, that can (optionally) be cached, and is, actually ... |> Unfortunately cryptsetup is needed even though, i think, the |> kernel has anything needed; you just cannot access it. cryptsetup |> is only needed for "$cs open $PART_ROOT p_root --key-file -". |> Of course i am no real Linux expert but only a do-it-yourself guy. Well, and user space only, and never having read BIOS or UEFI standards / implementations. I have reread (almost) all of FreeBSDs documentation (loader, uefi etc) today, and it is always great to see such good documentation! Of course some things are beyond what a "normal" user would understand, but at least there is documentation available. That is really great. |> busybox allows me to manage this easily, to answer your question. | |You could do that on FreeBSD with a loader.efi that has a ram disk |built into it as well, including a 'beastie box' thing that's akin to |busybox. |It will boot in one step and no no further I/O to get a running system. |Others have used this for secure boot and to boot a small ram disk that's |later discarded as userland decides what root should be. But it's much less |automated than in Linux... Hm, but wait. Isn't it easier for FreeBSD? I must admit i never used FreeBSD/geli on bare metal, i started using fully encrypted hard disks in mid 2021, no sooner. So when i read [1] it seems FreeBSD's boot procedure is so nicely linked with geli that the boot loader asks itself for the password of the encrypted partition, simply by setting some variables in loader.conf? I mean, the /boot/ must be available? [1] https://man.freebsd.org/cgi/man.cgi?query=geli&apropos=0&sektion=0&manpath=FreeBSD+13.2-RELEASE+and+Ports&arch=default&format=html These simple scripts i use for Linux also do not support true suspend, only suspend-to-ram. Ie, i have no idea how i could "suspend" the encrypted device and then "resume" it again; The "warm" security is only (initiated via ACPI event from root account) auto-unload of SSH and PGP keys etc, unload of encfs fuse volumes, and then slock of all X displays etc. Ie, all programs which run live on the encrypted volume. When the laptop wakes up, X is running etc. One would need the possibility to wake up to "a password prompt". Well, likely it would work to mount EFI vfat before the suspend, create a console and auto-switch to it, and then sit there, ready for reading in the password once the system resumes. That is to say: i always believed in FreeBSD this works just as easy as the boot procedure? --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) ^ permalink raw reply [flat|nested] 22+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-07 0:11 ` Steffen Nurpmeso @ 2023-09-07 16:05 ` Warner Losh 2023-09-08 14:58 ` Theodore Ts'o 0 siblings, 1 reply; 22+ messages in thread From: Warner Losh @ 2023-09-07 16:05 UTC (permalink / raw) To: Warner Losh, Norman Wilson, tuhs [-- Attachment #1: Type: text/plain, Size: 5531 bytes --] On Wed, Sep 6, 2023 at 6:12 PM Steffen Nurpmeso <steffen@sdaoden.eu> wrote: > Hello! > > Warner Losh wrote in > <CANCZdfoy0UJyFdvJ_fO4FDks_dfEOhsJo+7p2RORdk6gQWf7qg@mail.gmail.com>: > |On Tue, Sep 5, 2023 at 9:53 AM Steffen Nurpmeso <steffen@sdaoden.eu> > wrote: > |> Steffen Nurpmeso wrote in > |> <20230904221059.sF2G0%steffen@sdaoden.eu>: > |>|Norman Wilson wrote in > |>| <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org>: > |> ... > |>||Perhaps the question to ask is why such a magic program is > |>||needed at all. Is it just because programs like the shell > |>||have become so large and unwieldy that they won't fit in > |>||a small environment suitable for loading into an initramfs? > |> ... > |>|For my laptop it allows me easy boot management. > ... > |> Only to add that this is because of Linux and the way it is doing > |> things. If i would use FreeBSD on bare metal, then i would have > |> an EFI boot loader on EFI that knows (only) enough to ask for > |> passphrase (correct me if i am wrong), and can then boot the > |> kernel from FFS or ZFS. (You have to choose dedicated ZFS boot > |> loader iirc, but despite that...) > | > |No, you don't have to choose the dedicated ZFS boot loader, at least not > |anymore. > > Ah! It seems regarding FreeBSD i am stuck in BIOS world, then. > (zfsboot and gptzfsboot.) > Yes, for MBR, things are necessarily more complicated because the ZFS boot loader is too big. The first few sectors live in the 8k space that the traditional boot loader lives, and the rest is DD'd to a special area of the ZFS uberblock reserved for boot blocks. For gptboot vs gptzfsboot, they were written to be either or and combining them presented challenges, not least was the partition size that gptboot lives in was created for many releases too small to hold gptzfsboot and making it tricky to upgrade... > |Also, you can use boot1.efi to load loader.efi from the root filesystem > to > > The former deprecated i see. > > |load the kernel, or you could use loader.efi directly on the ESP to load > |the kernel. boot1 barely knows anything (and has only one choice of > |what to boot). loader.efi is the full deal, and can do rather a lot of > |sophisticated things. > > Great. > > ... > |> But anyhow. With an EFI_STUB Linux kernel i can save me all that, > |> with busybox i get a complete environment (i then even create an > |> initrd in /boot/ on the fly so i do not have to type the password > |> a second time, that can (optionally) be cached, and is, actually > ... > |> Unfortunately cryptsetup is needed even though, i think, the > |> kernel has anything needed; you just cannot access it. cryptsetup > |> is only needed for "$cs open $PART_ROOT p_root --key-file -". > |> Of course i am no real Linux expert but only a do-it-yourself guy. > > Well, and user space only, and never having read BIOS or UEFI > standards / implementations. > I have reread (almost) all of FreeBSDs documentation (loader, uefi > etc) today, and it is always great to see such good documentation! > Of course some things are beyond what a "normal" user would > understand, but at least there is documentation available. > That is really great. > > |> busybox allows me to manage this easily, to answer your question. > | > |You could do that on FreeBSD with a loader.efi that has a ram disk > |built into it as well, including a 'beastie box' thing that's akin to > |busybox. > |It will boot in one step and no no further I/O to get a running system. > |Others have used this for secure boot and to boot a small ram disk that's > |later discarded as userland decides what root should be. But it's much > less > |automated than in Linux... > > Hm, but wait. Isn't it easier for FreeBSD? I must admit i never > used FreeBSD/geli on bare metal, i started using fully encrypted > hard disks in mid 2021, no sooner. So when i read [1] it seems > FreeBSD's boot procedure is so nicely linked with geli that the > boot loader asks itself for the password of the encrypted > partition, simply by setting some variables in loader.conf? > I mean, the /boot/ must be available? > > [1] > https://man.freebsd.org/cgi/man.cgi?query=geli&apropos=0&sektion=0&manpath=FreeBSD+13.2-RELEASE+and+Ports&arch=default&format=html > > These simple scripts i use for Linux also do not support true > suspend, only suspend-to-ram. Ie, i have no idea how i could > "suspend" the encrypted device and then "resume" it again; > The "warm" security is only (initiated via ACPI event from root > account) auto-unload of SSH and PGP keys etc, unload of encfs fuse > volumes, and then slock of all X displays etc. Ie, all programs > which run live on the encrypted volume. When the laptop wakes up, > X is running etc. > One would need the possibility to wake up to "a password prompt". > Well, likely it would work to mount EFI vfat before the suspend, > create a console and auto-switch to it, and then sit there, ready > for reading in the password once the system resumes. That is to > say: i always believed in FreeBSD this works just as easy as the > boot procedure? > Yea. I find it easier :). But there's some better automation available in Linux for the 'complicated' situations. I was also trying to be generous to Linux because I know the passion of its supporters in the face of even mild criticism. Warner [-- Attachment #2: Type: text/html, Size: 7144 bytes --] ^ permalink raw reply [flat|nested] 22+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-07 16:05 ` Warner Losh @ 2023-09-08 14:58 ` Theodore Ts'o 0 siblings, 0 replies; 22+ messages in thread From: Theodore Ts'o @ 2023-09-08 14:58 UTC (permalink / raw) To: Warner Losh; +Cc: tuhs On Thu, Sep 07, 2023 at 10:05:37AM -0600, Warner Losh wrote: > > Yea. I find it easier :). But there's some better automation available in > Linux for the 'complicated' situations. Yeah, this is why most of the Linux distributions always use an initramfs, at least by default, because it handles all (or at least most) of these complicated sitautions automatically. So this leads to people assuming that Linux "requires" using a initramfs, when it would be more accurate to say that most Linux distributions makes it super easy to use an initramfs, and much more difficult (because the documentation is a bit scattered, and many may be somewhat out of date) to do things without an Linux distribution-engineered initramfs. And the other clear difference between Linux and FreeBSD is that things like initramfs tends to be distribution-specific, so there is competition between distributions to see who can make a simpler/easier installer and boot sequnce, and so market forces tends to strongly encourage developers (who after all, tend to be experts and who don't need the installer and boot up automation), to improve things much more aggressively. (And since there are product managers involved, often features and time to market often trump issues like technical debt.) The downside with this approach, though, is that distributions tend to reinvent the wheel multiple times, and the initramfs infrastructures are different. So learning how things work in say, Debian, isn't going to help you understand the fine details of how Fedora or Red Hat Enterprise Linux works except at a very basic, high level. And the same is most definitely true if you compare the installers between Debian, Ubuntu, Fedora, etc. FreeBSD has the advantage that it has a much more centralized development model, especially as it relates to kernel/libc integration, the installer, and the boot loader path. But while there is a certain amount of competitive forces of FreeBSD, NetBSD, OpenBSD, et. al., improving and simplifying the boot loader, from an outsider looking in, it would appear that those forces aren't quite as strong, and the level of convergence beween the *BSD seems to be at least as strong, if not stronger, in some areas, than between Linux distributions. Cheers, - Ted ^ permalink raw reply [flat|nested] 22+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-05 15:53 ` Steffen Nurpmeso 2023-09-06 17:50 ` Warner Losh @ 2023-09-08 13:56 ` Michael Kjörling 2023-09-08 23:38 ` Steffen Nurpmeso 1 sibling, 1 reply; 22+ messages in thread From: Michael Kjörling @ 2023-09-08 13:56 UTC (permalink / raw) To: tuhs On 5 Sep 2023 17:53 +0200, from steffen@sdaoden.eu (Steffen Nurpmeso): > Unfortunately cryptsetup is needed even though, i think, the > kernel has anything needed; you just cannot access it. cryptsetup > is only needed for "$cs open $PART_ROOT p_root --key-file -". > Of course i am no real Linux expert but only a do-it-yourself guy. If your need is restricted to a highly specific use case and you are trying to keep it as small as possible, then it should be possible to write a custom wrapper around whatever libcryptsetup functionality you need and avoid the extra code that you get with cryptsetup proper. -- Michael Kjörling 🔗 https://michael.kjorling.se “Remember when, on the Internet, nobody cared that you were a dog?” ^ permalink raw reply [flat|nested] 22+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-08 13:56 ` Michael Kjörling @ 2023-09-08 23:38 ` Steffen Nurpmeso 2023-09-09 22:43 ` Steffen Nurpmeso 2023-09-11 4:10 ` Theodore Ts'o 0 siblings, 2 replies; 22+ messages in thread From: Steffen Nurpmeso @ 2023-09-08 23:38 UTC (permalink / raw) To: Michael Kjörling; +Cc: tuhs Michael Kjörling wrote in <f948d06c-14f4-40bd-8e32-4db1c5b1dd21@home.arpa>: |On 5 Sep 2023 17:53 +0200, from steffen@sdaoden.eu (Steffen Nurpmeso): |> Unfortunately cryptsetup is needed even though, i think, the |> kernel has anything needed; you just cannot access it. cryptsetup |> is only needed for "$cs open $PART_ROOT p_root --key-file -". |> Of course i am no real Linux expert but only a do-it-yourself guy. | |If your need is restricted to a highly specific use case and you are |trying to keep it as small as possible, then it should be possible to |write a custom wrapper around whatever libcryptsetup functionality you |need and avoid the extra code that you get with cryptsetup proper. It is nicely documented, and my Linux distribution ships the static library anyhow. But i am a lazy sort regarding such, i just take the thing of my distribution and copy it over (they do build it statically also by default). You know, things change, and if you do not follow closely, you stand in the rain. I am not a paid Linux engineer that follows this rapidly moving target in the end. For example the (no longer) new random developer chose to disable feeding entropy via /dev/urandom, here (distribution) still is # Load random seed /bin/cat /var/lib/urandom/seed > /dev/urandom for almost two decades (it is a rather young one), but the code path was mutilated (i read the kernel source once he had rewritten that to be blake2/some 32-byte block thing based), now one needs to use some ioctl interface fwiw. Or once here cryptsetup was updated to use OpenSSL 3.0 suddenly ripemd160 was no longer available on EFI (aka purely static, without the filesystem avaialable), even though its release notes explicitly mentioned the problem as solved, and OpenSSL 3's libcrypto.a _had_ ripemd160... I had to switch to sha512 .. then to sha256 once cryptsetup started warning args had to be explicit in the future. Mind you, i in fact use it twice, also for encrypted swap, i only wrongly searched for $cs $2 open --type plain --cipher aes --key-size 256 \ --hash sha256 $PART_SWAP p_swap --key-file - && i said on IRC cryptsetup does EVP_DigestInit_ex(h->md, h->hash_id, NULL), i presume that does load additional things. That surely is it, i did not track it further. So no, to answer you, i have no highly specific use case at all. This is only an encrypted volume with my own boot-style that requires no boot loader but Linux itself. Maybe i should really look deeply in how cryptsetup then attaches a LUKS2 volume to the kernel, maybe it actually _would_ be possible to do this simply in some other way. But truly writing a program? I feel much saver in the horde, with so many people, specialists even, working on Linux, LUKS2, cryptsetup, OpenSSL, .. these are all moving targets. (I mean, i am lucky if i _can_ do a bit of programming on at least the MUA i maintain; so much to do! And roff hopefully somewhere on the horizon, somewhen; today it was zero minutes.) --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) ^ permalink raw reply [flat|nested] 22+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-08 23:38 ` Steffen Nurpmeso @ 2023-09-09 22:43 ` Steffen Nurpmeso 2023-09-11 4:10 ` Theodore Ts'o 1 sibling, 0 replies; 22+ messages in thread From: Steffen Nurpmeso @ 2023-09-09 22:43 UTC (permalink / raw) To: Michael Kjörling; +Cc: tuhs [-- Attachment #1: Type: text/plain, Size: 7275 bytes --] Steffen Nurpmeso wrote in <20230908233854.Xni_j%steffen@sdaoden.eu>: |Michael Kjörling wrote in | <f948d06c-14f4-40bd-8e32-4db1c5b1dd21@home.arpa>: ||On 5 Sep 2023 17:53 +0200, from steffen@sdaoden.eu (Steffen Nurpmeso): ... ||> Of course i am no real Linux expert but only a do-it-yourself guy. || ||If your need is restricted to a highly specific use case and you are ||trying to keep it as small as possible, then it should be possible to ... Actually maybe someone may find it funny, even though it is neither historic nor very sophisticated. Let me post it. In the end it takes some time to get there. I would expect almost all of you will not be interested in the lengthy rest of this mail. Ciao, and a nice Sunday i wish from Germany! All you need are statically linked busybox (and cryptsetup with encryption), and a kernel with EFI_STUB on the EFI partition, so it can be booted directly. The distribution kernels i have (ArchLinux, AlpineLinux) have it enabled; if your EFI is large enough you could simply copy it and all its masses of modules to EFI. Otherwise the kernel needs all the firmware and modules to boot and to access all filesystems where the real stuff is. My own kernel has a lot of things statically built-in, but it can load more via modules (stage2 uses "the same" kernel later, and does). Kernels on stage2 need kexec, i have CONFIG_KEXEC=y CONFIG_KEXEC_FILE=y CONFIG_ARCH_HAS_KEXEC_PURGATORY=y # CONFIG_KEXEC_SIG is not set CONFIG_KEXEC_CORE=y -s1.sh and -s2.sh are docu-commented at the top; they can also be used to create the necessary environment. If i recall correctly cd /boot sh linit-init-s2.sh PATH-TO-BUSYBOX PATH-TO-CRYPTSETUP mount EFI-PARTITION cd EFI-PARTITION sh linit-init-s1.sh PATH-TO-BUSYBOX PATH-TO-CRYPTSETUP This only creates files and directories etc, and beat me if it does any harm otherwise. I use a specific naming scheme, the script reacts in particular on -old and -new to select sensible defaults. #?0|kent:/boot# ll ideapad*.efi -rwxr-xr-x 1 root root 10112672 Aug 26 18:44 ideapad-stage1.efi* -rw-r--r-- 1 root root 10120512 Sep 2 19:52 ideapad-6_1.efi -rw-r--r-- 1 root root 10120512 Sep 9 18:18 ideapad-6_1-old.efi -rw-r--r-- 1 root root 10121376 Sep 9 18:18 ideapad-6_1-new.efi That very kernel (series) is in fact shared in between several computers of a similar series, kent is the one i write this on: #?1|kent:/boot# v kent.sh #!/busybox.static sh #@ kent, step 1., via EFI. PART_ROOT=/dev/nvme0n1p? ROOT_DECRYPT='-t btrfs -o defaults,subvol=/crux/kent/root' PART_ROOT1=/dev/nvme0n1p8 ROOT_DECRYPT1='-t btrfs -o defaults,subvol=/crux/kent/root.old' INIT_S2=/boot/kent-2.sh . /linux-init-s1.sh #?0|kent:/boot# v kent-2.sh #@ kent, stage 2 KERNEL_ID=ideapad ^ restrict kernel selection for kexec: ^ [ "${k##*$KERNEL_ID-*.efi}" != "$k" ] || continue PART_SWAP=/dev/nvme0n1p6 SWAP_INPUT=y ^ you can have additional interactive random for swap encryption ^ (it is a lie i use that in practice) INITRD_PATH=/boot/.kent.initrd KEXEC_ARGS="--append=\"rtw88_pci.disable_aspm=1 rc.hostname=kent\"" FILE_CHECK='kent-direct.sh kent.sh ideapad-stage1.efi' SWITCH_ROOT=media/initrd Stage 2 config file is sourced automatically, so no need to source linux-init-s2.sh yourself (stage 2 uses switch_root). I also have a direct boot, but have not used it for long: #?0|kent:~# v /boot/kent-direct.sh #!/busybox.static sh #@ kent-direct, sole step 1., via EFI. PART_ROOT=/dev/nvme0n1p8 ROOT_DECRYPT='-t btrfs -o defaults,subvol=/crux/kent/root' PART_ROOT1=/dev/nvme0n1p8 ROOT_DECRYPT1='-t btrfs -o defaults,subvol=/crux/kent/root.old' PART_SWAP=/dev/nvme0n1p6 PIVOT_OLDROOT=media/initrd . /linux-init-s1.sh This does not use kexec, but only pivot_root's to the decrypted filesystem, starting sbin/init there. The kernel from EFI continues to run. It is pretty cool how i now can drive several computers easily simply by creating a shell script with some variable assignments. In fact the filesystem is shared in between multiple computers (with only selective partitions being truly unique, but still backed up everywhere), and all those text configs are in /boot. Only the VFAT EFI partition is truly system-specific (ie, in that it only holds the stage1 config file for $HOSTNAME). I like that very much, and hope Linux continues to be so flexible. (Not that one day you need systemd-early to get into Linux to get to systemd-later, or something.) The EFI boot is driven via efibootmgr(8) and has config like #@ /root/hosts/self/efiboot d=/dev/nvme0n1p1 maxno=1 b1=0x01 L1=kent l1=ideapad-stage1 u1='root='${d}' rootfstype=vfat init=/kent.sh' driven by /root/bin/efiboot.sh (shortened): #!/bin/sh : ${HOSTNAME:=$(uname -n)} : ${DBG:=} if [ -f /root/hosts/${HOSTNAME}/efiboot ]; then . /root/hosts/${HOSTNAME}/efiboot else echo >&2 "MISS /root/hosts/${HOSTNAME}/efiboot" exit 1 fi if [ "$1" = create ]; then obo=$(efibootmgr | grep -E ^BootOrder: | sed -E 's/^BootOrder:[[:space:]]*//') if [ -z "$obo" ]; then echo >&2 'Cannot determine previous boot order' exit 1 fi nbo= i=1 while [ $i -le $maxno ]; do eval ${DBG} efibootmgr -c -b \$b$i -L \\\"\$L$i\\\" -d \$d$i -l \\\"\$l$i.efi\\\" -u \\\"\$u$i\\\" #>/dev/null [ -n "$nbo" ] && nbo=$nbo, eval nbo=\$nbo\$b$i i=$((i + 1)) done ${DBG} efibootmgr -o $nbo,$obo #> /dev/null elif [ "$1" = delete ]; then i=1 while [ $i -le $maxno ]; do eval ${DBG} efibootmgr -B -b \$b$i #>/dev/null i=$((i + 1)) done else echo >&2 'Synsopsis: efiboot.sh create|delete' exit 1 fi ${DBG} efibootmgr -v -u This is of course git managed, so i can distribute it easily. (Further even, the entire ideapad series configurations differ in only 193 lines, and most of those are the different SSH keys. The rest are different device names, ie network, display, volume etc, disk partitions, host names.) I am pretty lucky i found a way to manage my systems like this. I have several git branches in root.git, linux.kconfig, bin, iwd.network, and one for each computer. I can then "git merge" selectively together what is necessary. For example my web vserver only needs bin and its own configuration, whereas the laptops also need iwd.network. Etc etc. Like this every computer only knows the selective minimum. The full git repo is on an encrypted volume stored away. And it is not Puppet or Ansible or any of those programs, it is nothing but git and its branches. But now i really left manually driven cryptsetup by far. Ciao! --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) [-- Attachment #2: linux-init-s1.sh --] [-- Type: application/x-sh, Size: 4596 bytes --] #@ /linux-init-s1.sh, simple Linux init, stage 1. #@ To be sourced from a kernel command line init=SCRIPT like: #@ #@ #!/busybox.static sh #@ #@ For kent #@ #@ PART_ROOT=/dev/nvme0n1p8 #@ ROOT_DECRYPT='-t btrfs -o defaults,subvol=/crux/kent/root' #@ PART_SWAP=/dev/nvme0n1p6 #@ INIT_S2=boot/kent-2.sh #@ #@ . /linux-init-s1.sh #@ #@ Alternatively call AS ROOT and FROM WITHIN directory that boots like #@ sh linux-init-s1.sh PATH_TO_BUSYBOX PATH_TO_CRYPTSETUP #@ and is expected to be mounted as / on system start, for init purposes. # Non-empty and we run mdev -s very first (value passed to it via ARGV) : ${MDEV_RUN:=} # Unless ROOT_DECRYPT (only partition then) this can be mount spec, like # PART_ROOT='-t btrfs -o defaults,subvol=/crux/kent/root' [ $# -eq 0 ] && : ${PART_ROOT:?"Need a usable \$PART_ROOT"} # Non-empty and we decrypt ROOT; _it_ should be mount spec then : ${ROOT_DECRYPT:=} # It is possible to specify multiple PART_ROOT[/ROOT_DECRYPT] tuples by # giving PART_ROOT1, 2, 3... A dialog for choosing one to use then appears # Non-empty and we create encrypted swap from the given partition : ${PART_SWAP:=} # Non-empty and we ask for more random bytes for swap encryption : ${SWAP_INPUT:=} # Non-empty and we give up to the stage 2 -s2.sh that is expected to live # in [/dev/mapper/p_root]/boot. ONLY WITH $ROOT_DECRYPT! # The value is the config file of stage 2 as an _absolute_ path within the # newly mounted root. : ${INIT_S2:=} # Were we move old root to when doing pivot_root(8) in non-$INIT_S2 cases : ${PIVOT_OLDROOT:=media/efi} BB=busybox.static CS=cryptsetup.static # -- >8 -- 8< -- xroot() { # {{{ bb=./$BB cs=./$CS slept= # May want to sleep to pass by kernel async console output xbase_mounts "$bb" # (First for completeness sake [not needed]) # PART_SUFFIX=.0 if [ -n "$PART_ROOT1" ]; then # Sleep a bit and hope kernel async console output passed by then [ -z "$slept" ] && $bb sleep 3 slept=y while :; do echo echo '. Multiple $PART_ROOT[/$ROOT_DECRYPT] encountered:' echo " [0] PART_ROOT=$PART_ROOT" [ -n "$ROOT_DECRYPT" ] && echo " ROOT_DECRYPT=$ROOT_DECRYPT" ri=1 while :; do eval rx="\$PART_ROOT$ri" [ -z "$rx" ] && break echo " [$ri] ..ROOT$ri=$rx" eval rx="\$ROOT_DECRYPT$ri" [ -n "$rx" ] && echo " ..CRYPT$ri=$rx" ri=$((ri + 1)) done rmsg= rrt= if (echo y | read -t 1 chck) >/dev/null 2>&1; then rmsg=' (10 seconds until, or empty input for default)' rrt='-t 10' fi printf '. Please choose $PART_ROOT'"$rmsg"'? ' read $rrt PART_SUFFIX if [ $? -ne 0 ] || [ -z "$PART_SUFFIX" ] || [ "$PART_SUFFIX" = 0 ]; then PART_SUFFIX=.0 echo break elif [ "${PART_SUFFIX##*[^0-9]}" != "$PART_SUFFIX" ]; then echo >&2 '! Invalid input' continue fi eval ri=\$PART_ROOT$PART_SUFFIX if [ -z "$ri" ]; then echo >&2 '! No such PART_ROOT: '$PART_SUFFIX else PART_ROOT=$ri eval ROOT_DECRYPT=\$ROOT_DECRYPT$PART_SUFFIX PART_SUFFIX=.$PART_SUFFIX echo break fi done fi # if [ -n "$ROOT_DECRYPT" ]; then [ -z "$slept" ] && $bb sleep 3 slept=y while :; do printf '. Please set free $PART_ROOT: ' $bb stty -echo read ROOT_PASS $bb stty echo printf "%s" "$ROOT_PASS" | $cs open $PART_ROOT p_root --key-file - && break xconfirm 'Cannot cryptsetup open '"$PART_ROOT"', try again' || x 1 done $bb mount $ROOT_DECRYPT /dev/mapper/p_root mnt || x 2 if [ -n "$INIT_S2" ]; then echo '. Going $INIT_S2='"$INIT_S2" cd mnt || x 3 boot/$BB mount --bind ../dev dev || x 10 boot/$BB mount --bind ../sys sys || x 11 boot/$BB mount --bind ../proc proc || x 12 boot/$BB mount --bind ../run run || x 13 exec boot/$BB chroot . /boot/$BB sh \ /boot/linux-init-s2.sh --stage1 "$INIT_S2" \ "$PART_SUFFIX" "$PART_ROOT" \ "$ROOT_DECRYPT" "$ROOT_PASS" || x 'Cannot go $INIT_S2' fi else echo '. Mounting $PART_ROOT' $bb mount $PART_ROOT mnt || x 2 fi xswap $bb $cs echo '. About to pivot_root' cd mnt || x 3 bb=boot/$bb $bb pivot_root . $PIVOT_OLDROOT || x 4 $bb mount -o move $PIVOT_OLDROOT/dev dev || x 10 $bb mount -o move $PIVOT_OLDROOT/sys sys || x 11 $bb mount -o move $PIVOT_OLDROOT/proc proc || x 12 $bb mount -o move $PIVOT_OLDROOT/run run || x 13 echo '. Going sbin/init' $bb stty isig exec $bb chroot . sbin/init || x 'Cannot sbin/init' } # }}} if [ $# -ne 0 ]; then . $($1 dirname "$0")/linux-init-lib.sh xsetup "$1" "$2" else . $(./$BB dirname "$0")/linux-init-lib.sh xroot fi exit 127 # s-sht-mode [-- Attachment #3: linux-init-s2.sh --] [-- Type: application/x-sh, Size: 7186 bytes --] #@ /linux-init-s2.sh, simple Linux init, optional stage 2, called from stage 1. #@ Requires $BB, $CS, as below, and these funs, on EFI / as well as new /boot/. #@ #@ #@ For kent, stage 2 #@ #@ [KERNEL_ID=ideapad] - if set, only $KERNEL_ID-*.efi kernels are examined #@ PART_ROOT=/dev/nvme0n1p8 #@ ROOT_DECRYPT='-t btrfs -o defaults,subvol=/crux-3.6/kent/root' #@ PART_SWAP=/dev/nvme0n1p6 #@ INITRD_PATH=boot/.s2.initrd #@ SWITCH_ROOT=media/initrd #@ #@ Alternatively call AS ROOT and FROM WITHIN directory that boots like #@ sh linux-init-s2.sh PATH_TO_BUSYBOX PATH_TO_CRYPTSETUP #@ for init purposes. # Identical to stage 1. : ${MDEV_RUN:=} #: ${PART_ROOT:=} via stage 1 #: ${ROOT_DECRYPT:=} via stage 1 (ditto ROOT_PASS) : ${PART_SWAP:=} : ${SWAP_INPUT:=} # If non-empty, we try to load the given initrd (relative to root) instead of # newly creating it; also, we will try to save it there if we had to create it # Will get PART_SUFFIX of chosen PART_ROOT/ROOT_DECRYPT tuple! : ${INITRD_PATH:=} # Further arguments when kexec(8)ing the new kernel : ${KEXEC_ARGS:=} : ${KEXEC_EXE:=usr/sbin/kexec} # Running in the kexec(8) kernel, we may apply some file comparisons. # If set it is a list of space separated files which must exist in /boot as # well as $FILE_CHECK_PARTITION (most likely the same as $SWITCH_ROOT), that is # temporarily mounted if it is not yet. # The files $BB, $CS, linux-init-{lib,s1}.sh are implied : ${FILE_CHECK:=} : ${FILE_CHECK_PARTITION:=media/efi} : ${FILE_CHECK_MOUNT_ARGS:=-n -t vfat /dev/nvme0n1p1} # Where we move old root to when doing switch_root(8) : ${SWITCH_ROOT:=media/initrd} BB=busybox.static CS=cryptsetup.static # -- >8 -- 8< -- xkexec() { # $1=config file; assumes we are in / and boot/ "has it" {{{ conf=$1 # Find the possible kernels to boot set -- boot/*.efi ki=1 kdef= knew= for k do if [ -n "$KERNEL_ID" ]; then [ "${k##*$KERNEL_ID-*.efi}" != "$k" ] || continue fi if [ "${k##*-new.efi}" = "$k" ] && [ "${k##*-old.efi}" = "$k" ] && [ "${k##*-stage1.efi}" = "$k" ]; then #[ -z "$kdef" ] && kdef=$k kdef=$k fi if [ -z "$knew" ] && [ "${k##*-new.efi}" != "$k" ]; then knew=$k fi eval k$ki=$k ki=$((ki + 1)) done # Let user choose one trap 'x "Got signal"' INT QUIT TERM echo '+ trapping INT/QUIT/TERM to start a shell' k= k0=- while [ -z "$k" ]; do echo echo '. Available kernels:' echo ' [0] Edit $KEXEC_ARGS' kj=1 while [ $kj -lt $ki ]; do eval echo "\" [$kj] \$k$kj\"" kj=$((kj + 1)) done echo if [ -n "$knew" ] && xconfirm 'Use the new kernel ('"$knew"')?'; then k=$knew else kmsg= krt= if [ -n "$kdef" ] && (echo y | read -t 1 chck) >/dev/null 2>&1; then kmsg=' (10 seconds until, or empty input for '"$kdef"')' krt='-t 10' fi printf '. Please choose kernel'"$kmsg"'? ' read $krt kx if [ $? -ne 0 ] || [ -z "$kx" ]; then echo k=$kdef elif [ "${kx##*[^0-9]}" != "$kx" ]; then echo >&2 '! Invalid input' continue fi eval k=\$k$kx if [ -z "$k" ]; then echo >&2 '! No such kernel' elif [ "$k" = - ]; then printf "%s\n" "$KEXEC_ARGS" > run/.kexec echo '# Only first line counts. Care for quoting! :wq' run/.kexec boot/$BB vi run/.kexec k=$KEXEC_ARGS read KEXEC_ARGS < run/.kexec boot/$BB rm -f run/.kexec xconfirm "New KEXEC_ARGS=$KEXEC_ARGS" || KEXEC_ARGS=$k k= fi fi done # Create or reuse a temporary initrd that contains the password if [ -n "$INITRD_PATH" ] && [ -f "$INITRD_PATH$PART_SUFFIX" ] && boot/$BB cp "$INITRD_PATH$PART_SUFFIX" run/.initrd; then echo '. Reusing initrd '"$INITRD_PATH$PART_SUFFIX" else echo '. Creating initrd' boot/$BB mount -o remount,exec run ( set -e boot/$BB mkdir run/x cd run/x ( xsetup2 "../../boot/$BB" "../../boot/$CS" ) e=$? [ $e -ne 0 ] && exit $e ./$BB mknod dev/console c 5 1 # redundant echo "#!/$BB sh" > ./init ./$BB cat "$conf" >> ./init echo "PART_ROOT='$PART_ROOT'" >> ./init echo "ROOT_DECRYPT='$ROOT_DECRYPT'" >> ./init echo "ROOT_PASS='$ROOT_PASS'" >> ./init echo '. /linux-init-s2.sh' >> ./init ./$BB chmod 0755 ./init { # Microcode update must be uncompressed and first [ -f ../../boot/early-ucode.cpio ] && ./$BB cat ../../boot/early-ucode.cpio # Followed by (possibly compressed) normal initrd ./$BB find . | ./$BB cpio -H newc -o | ./$BB gzip -9 -n } > ../.initrd ) || x 'Failed to create run/.initrd' boot/$BB mount -o remount,noexec run if [ -n "$INITRD_PATH" ]; then echo '. Saving initrd to '"$INITRD_PATH$PART_SUFFIX" boot/$BB cp run/.initrd "$INITRD_PATH$PART_SUFFIX" boot/$BB chmod 0600 "$INITRD_PATH$PART_SUFFIX" fi fi KEXEC_ARGS="$KEXEC_ARGS --initrd=run/.initrd" echo ". $KEXEC_EXE -l $k $KEXEC_ARGS" eval $KEXEC_EXE -l $k $KEXEC_ARGS || x 'Cannot invoke '"$KEXEC_EXE"' -l' echo ". $KEXE_EXE -e" exec $KEXEC_EXE -e || x 'Cannot invoke '"$KEXEC_EXE"' -e' } # }}} xfilecheck() { # $1=$BB; assumes we are in /mnt (_next_ root) {{{ [ -z "$FILE_CHECK" ] && return echo '. Checking file equalities' umountpart= if xhavemnt "$1" "$FILE_CHECK_PARTITION"; then :; else umountpart=y $1 mount $FILE_CHECK_MOUNT_ARGS "$FILE_CHECK_PARTITION" || x 'Cannot mount '"$FILE_CHECK_PARTITION" fi err= $1 cmp -s "$1" "$FILE_CHECK_PARTITION"/$BB || err="$BB" $1 cmp -s boot/$CS "$FILE_CHECK_PARTITION"/$CS || err="$err $CS" $1 cmp -s boot/linux-init-lib.sh "$FILE_CHECK_PARTITION"/linux-init-lib.sh || err="$err init-lib.sh" $1 cmp -s boot/linux-init-s1.sh "$FILE_CHECK_PARTITION"/linux-init-s1.sh || err="$err init-s1.sh" if [ -f boot/early-ucode.cpio ] && [ -f "$FILE_CHECK_PARTITION"/early-ucode.cpio ]; then $1 cmp -s boot/early-ucode.cpio "$FILE_CHECK_PARTITION"/early-ucode.cpio || err="$err early-ucode.cpio" fi for f in $FILE_CHECK; do if [ -f "boot/$f" ] && [ -f "$FILE_CHECK_PARTITION/$f" ]; then $1 cmp -s "boot/$f" "$FILE_CHECK_PARTITION/$f" || err="$err $f" else err="$err (MISS: $f)" fi done [ -n "$umountpart" ] && $1 umount "$FILE_CHECK_PARTITION" if [ -z "$err" ]; then echo ' . Files successfully checked' else echo >&2 '! BOOT: $FILE_CHECK errors: '$err echo '! BOOT: $FILE_CHECK errors: '$err >> etc/motd $1 sleep 10 fi } # }}} xroot() { # {{{ # On initrd rootfs bb=./$BB cs=./$CS xbase_mounts "$bb" echo '. Setting free $PART_ROOT' printf "%s" "$ROOT_PASS" | $cs open $PART_ROOT p_root --key-file - || x 'Cannot cryptsetup open '"$PART_ROOT" $bb mount $ROOT_DECRYPT /dev/mapper/p_root mnt || x 2 bb=mnt/boot/$BB cs=mnt/boot/$CS xswap $bb $cs cd mnt || x 3 bb=boot/$BB cs=boot/$CS xfilecheck $bb echo '. Going switch_root sbin/init' $bb stty isig # -c dev/console exec $bb switch_root . sbin/init 2>&1 || x 'Cannot switch_root' } # }}} if [ "$1" = --stage1 ]; then cnf=$2 PART_SUFFIX=$3 PART_ROOT=$4 ROOT_DECRYPT=$5 ROOT_PASS=$6 [ -f "$cnf" ] || x 'Cannot find $INIT_S2: '"$cnf" . "$cnf" . /boot/linux-init-lib.sh xkexec "$cnf" elif [ $# -eq 2 ]; then . $($1 dirname "$0")/linux-init-lib.sh xsetup2 "$1" "$2" - else . /linux-init-lib.sh xroot fi exit 127 # s-sht-mode [-- Attachment #4: linux-init-lib.sh --] [-- Type: application/x-sh, Size: 3646 bytes --] #@ linux-init-lib.sh, helpers for simple init stage 1 and 2; will be sourced. # misc {{{ x() { echo >&2 'ERROR ('"$*"')! Going shell' [ -x ./$BB ] && exec ./$BB sh [ -x /$BB ] && exec /$BB sh [ -x /boot/$BB ] && exec /boot/$BB sh [ -x /bin/sh ] && exec /bin/sh exit 100 } xconfirm() { printf -- '. '"$*"' [Yy/else]? ' read answer case $answer in [Yy]*) return 0;; *) return 1;; esac } _mpx= _mp= xhavemnt() { # $1=$BB $2=file if [ "$2" = dev ]; then [ -e /dev/console ] && [ -e /dev/null ] && [ -e /dev/zero ] return $? elif [ "$2" = proc ]; then [ -e /proc/1 ] && [ -e /proc/self ] && [ -e /proc/uptime ] return $? elif [ "$2" = sys ]; then [ -e /sys/kernel ] && [ -e /sys/block ] && [ -e /sys/fs ] return $? fi if [ -z "$_mpx" ]; then _mpx=y $1 mountpoint -q / >/dev/null 2>&1 && _mp=y fi if [ -n "$_mp" ]; then $1 mountpoint -q "$2" >/dev/null 2>&1 return $? elif [ -f /proc/mounts ]; then $1 grep -q ' '"$2"' ' /proc/mounts >/dev/null 2>&1 return $? fi return 1 } _xrand= xrand() { _xrand=$(( (($RANDOM << 20) | ($RANDOM << 10) | ($RANDOM % 0x3FF)) ^ \ (($RANDOM << 20) | ($RANDOM << 10) | ($RANDOM % 0x3FF)))) } # }}} xbase_mounts() { # {{{ $1=$BB [uses $MDEV_RUN; calls stty -isig] echo '. Checking /{dev,sys,proc,run}' xhavemnt $1 dev || $1 mount -n -t devtmpfs -o mode=0755,exec,nosuid,noatime none dev xhavemnt $1 sys || $1 mount -n -t sysfs none sys xhavemnt $1 proc || $1 mount -n -t proc none proc xhavemnt $1 run || $1 mount -n -t tmpfs -o mode=0755,noexec,nosuid,nodev none run #[ -e /sys/firmware/efi ] && #mount -t efivarfs efivarfs /sys/firmware/efi/efivars -o nosuid,nodev,noexec $1 stty -isig if [ -n "$MDEV_RUN" ]; then echo '. About to run mdev -s, waiting for some more settlement' $1 sleep 8 $1 mdev -s $MDEV_RUN fi } # }}} xsetup_base() { # $1=$BB $2=$CS [$3=no mdev stuff if set] {{{ $1 cp -f "$1" ./$BB ./$BB chmod 0700 ./$BB ./$BB cp -f "$2" ./$CS ./$BB chmod 0700 ./$CS ./$BB cp -f "$0" . ./$BB cp -f $(./$BB dirname "$0")/linux-init-lib.sh ./linux-init-lib.sh if [ $# -ne 3 ]; then # Prepare for $MDEV_RUN, too! ./$BB mkdir etc bin dev mnt proc run sys $mdir ./$BB cp ./$BB bin/sh ./$BB chmod 0700 bin/sh echo ".* 0:0 660 @/$BB sh /etc/mdev.sh" > etc/mdev.conf ./$BB cat > etc/mdev.sh <<-_EOT #@ /etc/mdev.sh for ./mdev.conf run() { if [ "\$ACTION" = add ]; then case "\$SUBSYSTEM" in block) block;; *) ;; esac fi exit } block() { [ -n "\$MDEV" ] || return [ -d /dev/disk ] || /$BB mkdir -p /dev/disk/by-uuid uuid=\$(/$BB blkid "\$MDEV" \| /$BB sed -Ee "s/.*UUID=\"([^\"]+)\".*/\1/") [ -n "\$uuid" ] || return /$BB ln -s ../../"\$MDEV" /dev/disk/by-uuid/"\$uuid" } run _EOT ./$BB chmod 0700 etc/mdev.sh fi echo '! Plus kernel, init=SCRIPT, to be placed here' } xsetup() { xsetup_base "$@" exit } xsetup2() { xsetup_base "$@" # For $FILE_CHECK ./$BB cp -f $(./$BB dirname "$0")/linux-init-s1.sh ./linux-init-s1.sh exit } # }}} xswap() { # $1=$BB $2=$CS {{{ [ -z "$PART_SWAP" ] && return echo '. Encrypted swap' rand= if [ -n "$SWAP_INPUT" ]; then echo ' ... please type a line of randoms' $1 stty -echo >/dev/null 2>&1 read rand $1 stty echo >/dev/null 2>&1 else xrand rand=$_xrand fi xrand xr1=$_xrand xrand xr2=$_xrand xrand xr3=$_xrand xrand xr4=$_xrand xrand xr5=$_xrand echo "$xr1$rand$xr2$xr3$rand$xr4$xr5" | $2 open --type plain --cipher aes --key-size 256 --hash sha256 $PART_SWAP p_swap --key-file - && $1 mkswap /dev/mapper/p_swap } # }}} # s-sht-mode ^ permalink raw reply [flat|nested] 22+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-08 23:38 ` Steffen Nurpmeso 2023-09-09 22:43 ` Steffen Nurpmeso @ 2023-09-11 4:10 ` Theodore Ts'o 2023-09-11 22:05 ` Steffen Nurpmeso 1 sibling, 1 reply; 22+ messages in thread From: Theodore Ts'o @ 2023-09-11 4:10 UTC (permalink / raw) To: Michael Kjörling, tuhs On Sat, Sep 09, 2023 at 01:38:54AM +0200, Steffen Nurpmeso wrote: > You know, things change, and if you do not follow closely, you > stand in the rain. I am not a paid Linux engineer that follows > this rapidly moving target in the end. > For example the (no longer) new random developer chose to disable > feeding entropy via /dev/urandom, here (distribution) still is > > # Load random seed > /bin/cat /var/lib/urandom/seed > /dev/urandom > > for almost two decades (it is a rather young one), but the code > path was mutilated (i read the kernel source once he had rewritten > that to be blake2/some 32-byte block thing based), now one needs > to use some ioctl interface fwiw. Huh? That's not correct. You can still introduce entropy into the random pool by writing to /dev/urandom or /dev/random. It is true that there has been some changes to the design of /dev/random, but I assure you as the original /dev/random developer, was consulted and involved in the design discussions. Anyway, this is off-topic for TUHS, but if you have specific questions/complaints, feel free to address them to me and Jason. Your comments do make me suspect that there are some fundamental misunderstandings at work. Cheers, - Ted ^ permalink raw reply [flat|nested] 22+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-11 4:10 ` Theodore Ts'o @ 2023-09-11 22:05 ` Steffen Nurpmeso 0 siblings, 0 replies; 22+ messages in thread From: Steffen Nurpmeso @ 2023-09-11 22:05 UTC (permalink / raw) To: Theodore Ts'o; +Cc: Michael Kjörling, tuhs Theodore Ts'o wrote in <20230911041059.GH701295@mit.edu>: |On Sat, Sep 09, 2023 at 01:38:54AM +0200, Steffen Nurpmeso wrote: |> You know, things change, and if you do not follow closely, you |> stand in the rain. I am not a paid Linux engineer that follows |> this rapidly moving target in the end. |> For example the (no longer) new random developer chose to disable |> feeding entropy via /dev/urandom, here (distribution) still is |> |> # Load random seed |> /bin/cat /var/lib/urandom/seed > /dev/urandom |> |> for almost two decades (it is a rather young one), but the code |> path was mutilated (i read the kernel source once he had rewritten |> that to be blake2/some 32-byte block thing based), now one needs |> to use some ioctl interface fwiw. | |Huh? That's not correct. You can still introduce entropy into the |random pool by writing to /dev/urandom or /dev/random. It is true |that there has been some changes to the design of /dev/random, but I |assure you as the original /dev/random developer, was consulted and |involved in the design discussions. Ah, lesser and lesser freedom for man! That is the truth! Neither was (last i looked) that counted no more to unlock man from the chokehold of missing entropy, nor do newer kernels then decrement entropy when you read random. |Anyway, this is off-topic for TUHS, but if you have specific |questions/complaints, feel free to address them to me and Jason. Your |comments do make me suspect that there are some fundamental |misunderstandings at work. Surely for the latter. So much i give in on this song line. Greetings. --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) ^ permalink raw reply [flat|nested] 22+ messages in thread
* [TUHS] Re: Unix install & "standalone" package 2023-09-04 14:44 [TUHS] Unix install & "standalone" package Norman Wilson ` (2 preceding siblings ...) 2023-09-04 22:10 ` Steffen Nurpmeso @ 2023-09-05 1:07 ` Jonathan Gray 3 siblings, 0 replies; 22+ messages in thread From: Jonathan Gray @ 2023-09-05 1:07 UTC (permalink / raw) To: Norman Wilson; +Cc: tuhs On Mon, Sep 04, 2023 at 10:44:21AM -0400, Norman Wilson wrote: > I don't remember any special many-programs-in-one binary > like busybox in any Unix from the days when Unix was simple > enough for me to understand. That covers the entire lifetime > of the Research systems, but also System V and the BSDs and > their sundry offspring up into at least the 1990s. > > I'm pretty sure OpenBSD at least still has nothing like > busybox. The nearest thing was to make sure certain programs look at ls -i output on the install media/bsd.rd https://man.openbsd.org/crunchgen.8 Added to FreeBSD and NetBSD in 1994. ^ permalink raw reply [flat|nested] 22+ messages in thread
* [TUHS] Unix install & "standalone" package @ 2023-09-04 9:57 Paul Ruizendaal via TUHS 0 siblings, 0 replies; 22+ messages in thread From: Paul Ruizendaal via TUHS @ 2023-09-04 9:57 UTC (permalink / raw) To: tuhs Recently, I was looking into the “Das U-Boot” boot loader package. Summarised with great simplification, u-boot bundles device drivers, file systems, commands and a Bourne-like shell into a standalone package. Normally it auto-runs a script that brings up a system, but when used in interactive mode it allows a great deal of poking around. It made me think of the “standalone” set of programs for installing early Unix. On 16-bit understandably each basic command has to be a separate standalone program, but after the shift to 32-bit bundling more functionality in a single binary would have become possible. How did the Unix “standalone” package evolve in the 80’s, both in the research and BSD lineages? Is there any retrospective paper about that? Or is it a case of “Use the source, Luke”? ^ permalink raw reply [flat|nested] 22+ messages in thread
end of thread, other threads:[~2023-09-11 22:06 UTC | newest] Thread overview: 22+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2023-09-04 14:44 [TUHS] Unix install & "standalone" package Norman Wilson 2023-09-04 14:55 ` [TUHS] " Vincenzo Nicosia 2023-09-04 17:20 ` Warner Losh 2023-09-04 19:05 ` Clem Cole 2023-09-05 17:03 ` Paul Winalski 2023-09-05 18:02 ` Clem Cole 2023-09-04 19:59 ` Theodore Ts'o 2023-09-04 23:51 ` Warner Losh 2023-09-04 17:18 ` Warner Losh 2023-09-04 22:10 ` Steffen Nurpmeso 2023-09-05 15:53 ` Steffen Nurpmeso 2023-09-06 17:50 ` Warner Losh 2023-09-07 0:11 ` Steffen Nurpmeso 2023-09-07 16:05 ` Warner Losh 2023-09-08 14:58 ` Theodore Ts'o 2023-09-08 13:56 ` Michael Kjörling 2023-09-08 23:38 ` Steffen Nurpmeso 2023-09-09 22:43 ` Steffen Nurpmeso 2023-09-11 4:10 ` Theodore Ts'o 2023-09-11 22:05 ` Steffen Nurpmeso 2023-09-05 1:07 ` Jonathan Gray -- strict thread matches above, loose matches on Subject: below -- 2023-09-04 9:57 [TUHS] " Paul Ruizendaal via TUHS
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).