From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, MAILING_LIST_MULTI autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 366 invoked from network); 1 Jan 2023 20:30:13 -0000 Received: from minnie.tuhs.org (50.116.15.146) by inbox.vuxu.org with ESMTPUTF8; 1 Jan 2023 20:30:13 -0000 Received: from minnie.tuhs.org (localhost [IPv6:::1]) by minnie.tuhs.org (Postfix) with ESMTP id 1CDE642430; Mon, 2 Jan 2023 06:29:49 +1000 (AEST) Received: from ewsoutbound.kpnmail.nl (ewsoutbound.kpnmail.nl [195.121.94.170]) by minnie.tuhs.org (Postfix) with ESMTPS id 878584242F for ; Mon, 2 Jan 2023 06:29:33 +1000 (AEST) X-KPN-MessageId: ee5e8a98-8a12-11ed-8e68-005056ab378f Received: from smtp.kpnmail.nl (unknown [10.31.155.38]) by ewsoutbound.so.kpn.org (Halon) with ESMTPS id ee5e8a98-8a12-11ed-8e68-005056ab378f; Sun, 01 Jan 2023 21:29:03 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=planet.nl; s=planet01; h=to:message-id:date:from:subject:mime-version:content-type; bh=lMQsGL429eHz0F71bjDA/Q0h1oxfC9wWrCGW/7enp8I=; b=MTT8M/1Hur9shOMtqyJoN4dkp1367ggoGcKNYcKq7GrOB0p9yBDUH2ryc0ujXbiNiQk38R2FRm5T7 PCFEwt89rkOdr7KQ0YkGDY/BNxV3Oh1oXtZiNh+QC0PhR6DXEGNPVInnM9Qi7+C2eX7bB0OBI072TH W0qhFEdvYHth88vY= X-KPN-MID: 33|qb0E0X7Jg92nrjBPnfBwneF6Ej1TAfBh8tw3QgDVHkWs64UznPLbTTIVUkD1/ku WlA7os9LCWK4o+vC6WtgHKMUfb79PZvgxKrLXaMZXwoE= X-KPN-VerifiedSender: Yes X-CMASSUN: 33|yhhfbvKLdtpZEk7KyRdG3YNYbMT6XJKs5bbn1NvnGUMMuqSa7OUl/JX2SX1rpKw Fmzwl1mKn4WIgBKxcrVwsiA== X-Originating-IP: 77.172.38.96 Received: from smtpclient.apple (77-172-38-96.fixed.kpn.net [77.172.38.96]) by smtp.kpnmail.nl (Halon) with ESMTPSA id f71bc28d-8a12-11ed-97f1-005056abf0db; Sun, 01 Jan 2023 21:29:18 +0100 (CET) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) From: Paul Ruizendaal In-Reply-To: Date: Sun, 1 Jan 2023 21:29:17 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <335F89A9-30C2-41A2-8E84-C2D761746634@planet.nl> To: Theodore Ts'o X-Mailer: Apple Mail (2.3654.120.0.1.13) Message-ID-Hash: MMM4RGZZWJAILCHYX3NX65GPMMY4L6ES X-Message-ID-Hash: MMM4RGZZWJAILCHYX3NX65GPMMY4L6ES X-MailFrom: pnr@planet.nl X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-tuhs.tuhs.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: The Eunuchs Hysterical Society X-Mailman-Version: 3.3.6b1 Precedence: list Subject: [TUHS] Re: Porting the SysIII kernel: boot, config & device drivers List-Id: The Unix Heritage Society mailing list Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: > On 31 Dec 2022, at 23:38, Theodore Ts'o wrote: >=20 > If we don't agree on exactly what problem that we are trying to solve > with BSD-style autoconfig, vs UEFI vs ACPI vs the x86 BIOS vs CP/M's > BIOS, etc. Otherwise, the risk that we'll end up talking past one > another, or delivering a solution that we're happy with, but others > are not, is quite high IMHO. Wise words, I was indeed jumping to a conclusion without defining the = question. Douglas Adams comes to mind. On the positive side, it did = generate a lot of insightful posts that I hope to digest in the coming = days. > On 31 Dec 2022, at 22:10, Dan Cross wrote: >=20 > Mothy Roscoe gave a really interesting keynote at OSDI'21: > https://www.youtube.com/watch?v=3D36myc8wQhLo What an interesting keynote! At first he makes the case that OS research = is dead (not unlike Rob=E2=80=99s similar observation 20 years before). = However, he goes on to point out a specific challenge that he feels is = in dire need of research and innovation. In short his case is that a = modern SoC is nothing like a VAX, but Linux still treats all hardware = like a VAX. Having watched his keynote I can better place my generic discomfort with = the amount of code required outside of the classical OS on a SoC. =3D=3D=3D=3D Below some clarification on the topics that were on my mind: > That makes figuring out where the host OS image is and getting it = loaded > into memory a real pain in the booty; not to mention on modern systems > where you=E2=80=99ve got to do memory training as part of initializing = DRAM > controllers and the like. That was my immediate pain point in doing the D1 SoC port. = Unfortunately, the manufacturer only released the DRAM init code as = compiler =E2=80=98-S=E2=80=99 output and the 1,400 page datasheet does = not discuss its registers. Maybe this is a-typical, as I heard in the = above keynote that NXP provides 8,000 page datasheets with their = SoC=E2=80=99s. Luckily, similar code for ARM chips from this = manufacturer was available as C source and I could reverse engineer the = assembler file back to about 1,800 lines of C. See = https://gitlab.com/pnru/xv6-d1/-/blob/master/boot0/sdram.c=20 It does all the expected things (set voltage, switch on the main clock, = set up the PLL, calculate delays in terms of clocks, train for the line = delays, probe address multiplexing, etc.) by accessing a multitude of = registers that appear directly connected to the controller fabric. Yet, = at the same time it has only a single entry point (=E2=80=9Cinit_DRAM=E2=80= =9D) that takes 24 (poorly documented) words of 32 bits to define the = desired configuration (with options to specify =E2=80=9Cdefault=E2=80=9D = or =E2=80=9Cauto=E2=80=9D). Why does the main processor need to run this code? Why is there not a = small RV32E CPU inside this controller that runs the 1,800 lines and = exposes just the 24 words as registers to the main CPU? Are the square = mils really that expensive? Or is this the reason that many SoC=E2=80=99s = (but not the D1) have a small side CPU to do this for all devices? Next, to read from the SD Card, its controller needs to be set up. = Luckily here the manufacturer gives out actual C code and there is = documentation. However it is another 1,400 lines of code. About half the = code goes to initialising the controller hardware, the other half to = negotiating with the card to find out the data width and maximum speed. = Together with setting up the SoC clock distribution and the console UART = I have about 3,500 lines of setup code. And this is just the 1st stage = boot loader that has to run from a little bit of SRAM space on the SoC. = I have not looked yet at setting up the ethernet, wifi, display, usb, = etc., but undoubtedly this is between 1,000 and 2,000 lines each. This compares to about 7,000 lines for the SysIII kernel. There is also the question of what goes where. For the SD Card, does a = boot level module need to decide or operate the transfer mode? Should = this not be in the kernel? Maybe some of this should even be in a = daemon? It may seem to be a modern problem, but conceptually these very = same questions were at play when doing a compare and contrast between = the organisation of networking in 4.2BSD and 8th Edition. The former has = nearly everything in the kernel, the latter does a split over device = drivers, stream filters and daemons. This same question also comes up for all the other devices, in = particular usb. =3D=3D=3D=3D I=E2=80=99m not quite sure I agree with Rob=E2=80=99s observation on = boot ROM=E2=80=99s. While I get his point about organisational causes of = bloat, it is not just about the boot ROM. On a RISC-V or ARM SoC the = boot sequence is lengthy and the boot rom does little more than bring = the system to a safe state and load the first stage boot loader. The = full boot process is quite lengthy (e.g. = https://riscv.org/wp-content/uploads/2019/12/Summit_bootflow.pdf) and = several parts are SoC specific. One of the things on my mind was that SoC boards change quite quickly: = the D1 was last year=E2=80=99s hit with hobbyists, next year it will be = something else. Not nice if one has to redo 3,500-10,000 lines of code = for each board. Although I am not a fan of Forth, I can see that it is = useful when the controller IP blocks of a SoC (or the PCI cards of = discrete systems) expose some form of re-targetable program needed to = boot it up. The manufacturer BSP code is just not plug & play. Having watched Mothy Roscoe=E2=80=99s keynote, I realise that I was = trying to create a way to make a modern SoC look like a VAX. Maybe valid = for my archeology purposes, but not in the general case. On top it did = not solve the crappy BSP code issue. Maybe the correct solution for my = archeology is to just use the simpler FPGA SoC as a target. =3D=3D=3D=3D > Virtio is solving a very, very different problem: this is limited > paravirtualization of devices for virtual machines. Yes, this is where it started, but it seems to get traction as a = standardised embedded device interface, especially in automotive (Google = for "automotive virtio=E2=80=9D). Also in FPGA implementations it seems = to get more popular as a device interface e.g. the RVSoc system from the = Uni of Tokyo. This one has a very simple & partial HW implementation, = but it is enough to support an unmodified Linux. I don=E2=80=99t think I agree with every observation made on direct = virtio implementation (I certainly agree with some) -- but that is maybe = off topic for this list. I will observe however that many device = controllers of the early-mid-80=E2=80=99s contained some sort of = embedded processor, some even on the low cost end of the market -- think = of the WD-1002 hard disk controller variant used in the PC-AT. For = mini=E2=80=99s this was not unusual either, I already mentioned the M68K = on the Unibus DEULA board, the T11 on the Qbus RQDX2 board is another = example. Having smart controllers was cost competitive, even in the = 80=E2=80=99s. =3D=3D=3D=3D >> - MMU abstraction somewhat similar in idea to that in SYSV/68 [1] >=20 > It's not clear to me what this would buy you; the MMU tends to be > architecturally defined, Today, yes, but not in the early 80=E2=80=99s. For the M68K and the = Z8000 there were both segmented and paged MMU=E2=80=99s and often custom = designs were used (think Onyx C8000, SUN-1). However, my thinking here was driven by an entirely different thing. I = have referred before to the early history of Xenix, for example this = chart: http://seefigure1.com/images/xenix/xenix-timeline.jpg (the whole = page http://seefigure1.com/2014/04/15/xenixtime.html makes for an = interesting read). For a long time I have wondered why early Xenix did not make the jump to = a product that was split between a BIOS and a BDOS part, so that they = could sell BDOS-part updates to the total installed base. The BDOS part = could even be in some kind of p-code. Considering that they heavily = invested in their =E2=80=9Crevenue bomb=E2=80=9D C-compiler at the time, = this type of thinking was close to their hart (the Amsterdam Compiler = Kit was a similar idea). I am talking =E2=80=9981-=E2=80=9983 here, = thereafter it is clear that their economic interest was to focus on DOS. There are 3 possibilities: 1. It did not make technical sense 2. It did not make economic sense 3. It did make sense, but it simply did not happen So, yes, I was conflating a lot of different thoughts into a single = solution, without first thinking clearly about the question. > let's remember that Unix was _very_ success on a variety of > systems in that era without relying on a BIOS-type abstraction! For sure, that is undisputed. But could it have been even more = successful? Maybe the main reason for "no BIOS attempt" was purely = economic: for the companies having to port to each and every machine = created customer lock-in, and for the professionals it created an = industry of well paying porting & maintenance jobs. The customers were = willing to pay for it. Why kill the golden goose?