The Unix Heritage Society mailing list
 help / color / mirror / Atom feed
* [TUHS] Re: UNIX "Machine Layer" Standards
@ 2023-04-21 14:37 Noel Chiappa
  2023-04-21 17:58 ` John Cowan
  0 siblings, 1 reply; 9+ messages in thread
From: Noel Chiappa @ 2023-04-21 14:37 UTC (permalink / raw)
  To: tuhs; +Cc: jnc

    > From: Paul Ruizendaal

    > something like a boot rom only became the norm in the late
    > 70's. Before that, one keyed in two dozen words with a tiny program to
    > load the first boot stage.

A little wrong on that date. Even the PDP-11/20 (the first -11) had a boot
ROM:

  https://gunkies.org/wiki/BM792_ROM

which appreared in mid-1971 (about a year after the release of the /20). DEC
sold them pre-programmed, but one could 'program' one onself, if one wanted -
with a soldering iron! (Check out the image! I actually did that to one that
I was given, that had been eviscerated by someone.) From then on (follow the
category link), the rest used PROM chips.

  
    > From: Warner Losh

    > Oftentimes, the interrupt vector was in the lowest core addresses

It's worth remembering that in the early period, that restriction to low
addresses was built into the hardware (in an amusing way :-).

Take the DL11:

  https://gunkies.org/wiki/DL11_asynchronous_serial_line_interface

which was sort of mandatory as the 'console' serial interface on most early
-11's (until the DL11-W appeared; more on its big improvement in a second).
It set the interrupt vector with _jumpers_. (You want to change the interrupt
vector? Dig out your soldering iron! :-) There were only 6 jumpers - one each
for address bits 3 through 8. So the largest vector you could set was 0770.

The DL11-W was a big step forward - it replaced the jumpers with a DIP
switch! :-) Still only six bits, though. :-)

	Noel

^ permalink raw reply	[flat|nested] 9+ messages in thread

* [TUHS] Re: UNIX "Machine Layer" Standards
  2023-04-21 14:37 [TUHS] Re: UNIX "Machine Layer" Standards Noel Chiappa
@ 2023-04-21 17:58 ` John Cowan
  2023-04-21 20:36   ` Clem Cole
  0 siblings, 1 reply; 9+ messages in thread
From: John Cowan @ 2023-04-21 17:58 UTC (permalink / raw)
  To: Noel Chiappa; +Cc: tuhs

[-- Attachment #1: Type: text/plain, Size: 855 bytes --]

On Fri, Apr 21, 2023 at 10:37 AM Noel Chiappa <jnc@mercury.lcs.mit.edu>
wrote:


> A little wrong on that date. Even the PDP-11/20 (the first -11) had a boot
> ROM:
>

The PDP-8/E, which came out in the same year, had a boot ROM if you were
using DECtape as the system device for OS/8.  The bootstrap for the TD8E
DECtape controller was a full device driver 128 words long, way beyond
human toggle-in capability.  It was put up in addresses 77600-77777, which
meant that you could only have 28 kwords rather than the full 32 kwords of
RAM.

In principle I suppose you could have toggled in the RIM loader (17 words)
and read in the TD8E bootstrap using either Model 33 or high-speed paper
tape, but I doubt if anyone did that.  RIM format alternated between
addresses and values, with each taking up two characters on the paper tape.

[-- Attachment #2: Type: text/html, Size: 1591 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* [TUHS] Re: UNIX "Machine Layer" Standards
  2023-04-21 17:58 ` John Cowan
@ 2023-04-21 20:36   ` Clem Cole
  0 siblings, 0 replies; 9+ messages in thread
From: Clem Cole @ 2023-04-21 20:36 UTC (permalink / raw)
  To: John Cowan; +Cc: Noel Chiappa, tuhs

[-- Attachment #1: Type: text/plain, Size: 674 bytes --]

On Fri, Apr 21, 2023 at 1:58 PM John Cowan <cowan@ccil.org> wrote:

> In principle I suppose you could have toggled in the RIM loader (17 words)
> and read in the TD8E bootstrap using either Model 33 or high-speed paper
> tape, but I doubt if anyone did that.  RIM format alternated between
> addresses and values, with each taking up two characters on the paper tape.
>
I can't speak for an 8/E or OS/8, for that matter. As I recall, the machine
(single Digital serial # - original 8) that TSS/8 was originally developed
used the RIM loader to load the disk bootstrap. I remember a paper tape on
the console ASR-33 and then toggling the RIM loader., etc.
ᐧ

[-- Attachment #2: Type: text/html, Size: 1635 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* [TUHS] Re: UNIX "Machine Layer" Standards
  2023-04-20 18:57 ` segaloco via TUHS
@ 2023-04-20 20:18   ` Steve Nickolas
  0 siblings, 0 replies; 9+ messages in thread
From: Steve Nickolas @ 2023-04-20 20:18 UTC (permalink / raw)
  To: tuhs

On Thu, 20 Apr 2023, segaloco via TUHS wrote:

> Anywho, I'm kinda at a standstill on this project right now while I work 
> on a few other things, namely a disassembly of Dragon Quest for the 
> Famicom (embedded console games are a lot like operating systems) and 
> manual page diffing stuff, but I'm hoping to carve out some time this 
> summer, maybe even a little coding retreat from work for a few weeks, to 
> hammer out a true project plan on paper so I can start working on more 
> of this in earnest.  I'm not sure yet if I'll be headed down the "get 
> UNIX going" or "get my kernel going" route first when I get there, but 
> either way, I'm sure we'll have plenty to talk about in the coming year.

Funny you mention Dragon Warrior, that's been something that's breaking my 
brain from two different directions lately (since I'm trying to port it to 
the Apple //e and the Nabu, and I'm completely in the weeds for both). XD

I haven't worked much on my own Unix stuff lately because the Nabu stuff's 
taken a lot of my brainpower.

-uso.

^ permalink raw reply	[flat|nested] 9+ messages in thread

* [TUHS] Re: UNIX "Machine Layer" Standards
  2023-04-20 16:04 ` Warner Losh
@ 2023-04-20 19:04   ` segaloco via TUHS
  0 siblings, 0 replies; 9+ messages in thread
From: segaloco via TUHS @ 2023-04-20 19:04 UTC (permalink / raw)
  To: Warner Losh; +Cc: Paul Ruizendaal, tuhs

[-- Attachment #1: Type: text/plain, Size: 7615 bytes --]

The off-list reply (and reply to reply) wound up on list...my bad for not checking Cc list, but in any case, if any real developments come of things I'm tinkering on, I'll be sure to share them back. Thanks for the analysis Warner, BSD and Bell do present two interesting approaches to the abstraction matter.

What makes things difficult is there aren't a lot of good examples of "multi-target" Bell source trees whereas with BSDs you have single source code packages that have multiple "machine layer" instances present.

For instance, all the System V code I've seen floating around typically only has one variant of the kernel in the /usr/src folder, whereas you go into the machine layer on any BSD and you're bound to see the list of architectures and associated code being supported at the time. I suspect this was policy in the Bell UNIX group that the source provided with a given tape was for the architecture it was destined for, rather than /usr/src containing the entire port tree. Of course, this is just speculation, if someone knows the truth, I'm certainly curious how that sort of thing was organized/managed.

- Matt G.

P.S. sorry for the reply alls on the other fork of this thread, actually meant to not do that...

------- Original Message -------
On Thursday, April 20th, 2023 at 9:04 AM, Warner Losh <imp@bsdimp.com> wrote:

> On Thu, Apr 20, 2023 at 8:56 AM Paul Ruizendaal <pnr@planet.nl> wrote:
>
>>> Date: Mon, 10 Apr 2023 18:27:51 +0000
>>> From: segaloco
>>
>>> ... or was there no single guiding principle and each machine came up, at that level at least, in a relative vacuum, with only the machine interface to UNIX being the guiding principle?
>>
>> I stumbled into the same question last year, when doing my SysIII to RV64 port. I managed to turn that into a somewhat chaotic discussion, mixing old and new, and history with ideas. From that chaotic discussion I got the impression that it was indeed mostly ad hoc. In context, hardware was much easier to boot and drive back then -- it probably was not seen as complex enough to warrant much research into layering and abstraction.
>>
>> Also bear in mind that something like a boot rom only became the norm in the late 70’s. Before that, one keyed in two dozen words with a tiny program to load the first boot stage.
>>
>> That said, there is an implicit layering in v7 and beyond:
>>
>> - “low.s" does hardware setup, incl. such stuff as setting up interrupt tables. As this is closely tied to the hardware, it would have been a custom job in each case.
>
> V7 used l.s for this from research, though different names were used in different ports (though many retain l.s too). 32V used locore.s, a convention that all the BSDs I know of picked up and used, as well as many BSD-derived kernels that were later rewritten to support SMP.
>
> Oftentimes, the interrupt vector was in the lowest core addresses, and the first part of this file was just a giant table of places to jump for all the different architecturally defined exception and/or vectors (depending on the architecture). Often it contained glue from an interrupt to a ISR call as well, since there were many times where you'd share an exception, get the interrupt "level" or "vector" from some other bit of hardware and this code would often do the simple task of offsetting into a table and jumping.
>
> And it also had the "start" entry point for the whole kernel. And frequently silly aux routines like 'doadump' and the bcopy/fubyte(etc)/ and context switching code, which is also in assembler, often ended up there as well. Finally, it was a place to have various bits of storage that the kernel needed to bootstrap whatever VM was there.
>
>> - “mch.s” (later also mch.c) has the basic routines that are hardware dependent (switching stacks, changing priority levels and modes, etc.). It also has emulation for ‘missing’ instructions, such as floating point ops where this is not available in hardware. Same as above, I think. Maybe h/w related memory protection operations should live here as well, but the hardware was still quite divergent in this area in the 70’s and early 80’s.
>
> 32V called this machdep.c, which all the BSDs inherited. While machine dependent, it tended to be slightly more portable and was for stuff that could be written in C... Agreed on the very divergent part though.
>
>> - low-level device drivers live in the ‘dmr’ or (later) ‘io’ directory. Here there is some standardisation, as all device drivers must conform to the (char/block) device switch APIs. It seems to me that most of these drivers were written by taking one that was similar to what needed to be written and to start from there. Maybe this is still how it works in Linux today.
>
> To be fair, V7 was released in a time where there were no common protocol devices: There was no USB, bluetooth, SATA, SCSI, etc that had divergent drivers to talk to the hardware, but a common transport layer for the protocol. Even MSCP and TSCP, which were the first inklings of this, were a controller interface, not a transport one. The one SCSI driver from this era I've looked at implemented all the SCSI protocol itself. Thankfully the controller had a microcontroller for dealing with the physical signalling bits (unlike a different card for my DEC Rainbow which did it all in hardware by bit-banging I/O ports).
>
>> - To the extent that there is such a thing as 'high-level device drivers’ in early Unix, the structure is less clearly visible. The file system (and there was only one at the time of v7) is placed between the block device switch and the mount table so to speak. This was structured enough that splicing in other file systems seems to have been fairly easy in the early 80’s (the splicing in, not the writing of the file system itself, of course). Starting with 8th edition, the ‘file system switch’ created a clear API for multiple file systems. Arguably, the ‘tty’ subsystem is also a ‘high-level device driver’, but this one lives as custom code together with the serial port device drivers. Also in 8th Edition, ‘streams' were introduced. One could think of this as a structured approach to high-level device drivers for character mode devices, incl. the ’tty’ subsystem.
>
> Yes. It took a long time for there to even be common disk partition handling code. For a long time (and certainly in the V7 ports to PCish boxes) all that was in the driver, and usually cut and pasted from driver to driver. It was only later that better abstraction arose. Network stacks, and the like, were later inventions.
>
>> - I don’t think there was ever anything in early Unix that merged ’streams’ and the 'file system switch' into a single abstraction (but maybe 9P did?).
>
> I think you're right. They barely had a file system switch... And in the BSD side of the Unix world Network and File system were two different beasts.
>
>>> Where I'm trying to put this sort of knowledge into use is I'm starting to spec out a kernel bootstrap for the RPi Pico and Pine64 Ox64 boards (ARM32 and RISCV64 respectively) that is not only sufficient to start a V7-ish kernel on each, but that are ultimately based on the same design, varying literally only where the hardware strictly necessitates it, but similar enough that reading the two assembly files side by side yields essentially the exact same discrete operations.
>>
>> I have a similar interest, but to avoid the same chaos as I created before, I’ll respond to this with a pm.
>
> I'd be keen to understand this, but it's mostly a passing fancy...
>
> Warner

[-- Attachment #2: Type: text/html, Size: 9843 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* [TUHS] Re: UNIX "Machine Layer" Standards
  2023-04-20 15:57 Paul Ruizendaal
@ 2023-04-20 18:57 ` segaloco via TUHS
  2023-04-20 20:18   ` Steve Nickolas
  0 siblings, 1 reply; 9+ messages in thread
From: segaloco via TUHS @ 2023-04-20 18:57 UTC (permalink / raw)
  To: Paul Ruizendaal; +Cc: tuhs

The funny thing is what literally started all of this is I wanted to update the firmware on my VisionFive, followed their instructions to a T, and it bricked the firmware.

I found that the embedded recovery ROM still worked, so then tried to follow their processes to reflash the bootloader/SBI stuff.  That likewise failed, so I gave up on trusting any of their documentation/advice and just started hacking on the thing with binaries sent over XMODEM on the serial line.  Their XMODEM-CRC implementation is broken so I even had to write my own version of sx (they had a XMODEM send, likewise doesn't work stock...got it to work modified but then just wrote a simple stdio tool without all the cruft of their implementation.)

Fast forward to today and the most I've put together for that is this: https://gitlab.com/segaloco/riscv-bits (src is "kernel" stuff, util is that XMODEM-CRC transfer thing I mentioned)

I've got some process handling and VM stuff that hasn't gone up to the repo yet, but it's hanging around because it's incomplete anyway.  I then realized with what I'd learned in the process I was poised to start hacking on other SBCs at that level, so ordered the Pico, Ox64, and an Onion Omega2 (ARM32, RV64, and MIPS32 respectively) and have slowly been building up development workflows for each (I'm eschewing vendor libraries and SDKs given my experiences with the VisionFive...)

The boot matter itself is proving the most annoying part, since literally each one of those has a different mechanism to get an operational binary loaded up.  VisionFive is an (broken) XMODEM-CRC transfer, Pico is a UF2 file over USB, Ox64 is some custom Bouffalo Lab tool (although I think it's just a wrapper on top of some standard XMODEM/Kermit/etc. type protocol), and then the Omega2 I haven't even really explored raw boot yet, it's got OpenWRT on it so I've just been hacking on the existing OS thus far, getting more familiar with the board, my hunch is it's probably a UF2-ish thing since it's main UART is over USB like the Pico.

RISC-V presents some complications given the privilege model, what with the "master" mode vs the "supervisor" mode, as opposed to other chips where supervisor and master are essentially one in the same.  Where I'm torn is if I want to have OpenSBI in the mix just to go with what is ubiquitous in RISC-V land, or even try to handle that matter with my own code.  That wouldn't necessarily have an analog on all the other platforms.  Then there's hypervisor stuff, what with RISC-V having the extension and other chips having something akin to this, but it not being universal.  The simple answer is to just get everything, regardless of details into a "supervisor" state, ensure necessary interrupts are forwarded to this state if another, higher one exists, and enter into this supervisor state with MMU/VM turned off.  Then theoretically each machine can start from here by setting up memory, then device inits, then FS and other subsystem init, and finally process init.

Anywho, I'm kinda at a standstill on this project right now while I work on a few other things, namely a disassembly of Dragon Quest for the Famicom (embedded console games are a lot like operating systems) and manual page diffing stuff, but I'm hoping to carve out some time this summer, maybe even a little coding retreat from work for a few weeks, to hammer out a true project plan on paper so I can start working on more of this in earnest.  I'm not sure yet if I'll be headed down the "get UNIX going" or "get my kernel going" route first when I get there, but either way, I'm sure we'll have plenty to talk about in the coming year.

- Matt G.

------- Original Message -------
On Thursday, April 20th, 2023 at 8:57 AM, Paul Ruizendaal <pnr@planet.nl> wrote:


> Hi Matt,
> 
> I’ve responded on list about the early unix development process as I understand it, but I want to avoid discussing things that are not directly related to the history of Unix. Hence this PM as well.
> 
> > Where I'm trying to put this sort of knowledge into use is I'm starting to spec out a kernel bootstrap for the RPi Pico and Pine64 Ox64 boards (ARM32 and RISCV64 respectively) that is not only sufficient to start a V7-ish kernel on each, but that are ultimately based on the same design, varying literally only where the hardware strictly necessitates it, but similar enough that reading the two assembly files side by side yields essentially the exact same discrete operations.
> 
> 
> I have a similar interest, working with early Unix and modern RiscV hardware for a compare and contrast experience.
> 
> - My development targets are (i) an FPGA based RV32 SoC implementation, (ii) a Sipeed D1 RV64GC board and shortly (iii) a Pine64 Pinetab-V.
> 
> - My software targets are: (a) xv6-rv, (b) SysIII, (c) Linux, (d) experiments around SysIII
> 
> Linux is for me a secondary target, just for comparison and to see if ideas are “Linux capable”. I’m not overly interested in Arm at the moment.
> 
> 
> My ideas are still evolving, but currently more or less along the below lines:
> 
> - Boot rom loads SPL, this is custom in each case and set by the SoC's designers.
> 
> - SPL initialises DRAM system and loads next stage. Unfortunately, this too would seem to be quite system specific, but the BSP should provide the baseline for this. As BSP’s are often a mess, milage may vary.
> 
> - The next stage is a hybrid of BBL, OpenSBI and Virtio. The idea is to provide a standard abstraction layer that all of my software targets can work with. This idea is used for the FPGA target and allows booting a Linux kernel with just the generic Virtio device drivers (so far just disk and console).
> 
> - The last layer is the classical OS layer. If I get it right, each OS can run on all h/w targets without customisation.
> 
> At the moment I’m playing with USB, and how that might layer into the structure of V7, SysIII or 8th Edition -- and also the above.
> 
> 
> 
> 
> 
>

^ permalink raw reply	[flat|nested] 9+ messages in thread

* [TUHS] Re: UNIX "Machine Layer" Standards
  2023-04-20 14:56 Paul Ruizendaal
@ 2023-04-20 16:04 ` Warner Losh
  2023-04-20 19:04   ` segaloco via TUHS
  0 siblings, 1 reply; 9+ messages in thread
From: Warner Losh @ 2023-04-20 16:04 UTC (permalink / raw)
  To: Paul Ruizendaal; +Cc: tuhs

[-- Attachment #1: Type: text/plain, Size: 6438 bytes --]

On Thu, Apr 20, 2023 at 8:56 AM Paul Ruizendaal <pnr@planet.nl> wrote:

>
> > Date: Mon, 10 Apr 2023 18:27:51 +0000
> > From: segaloco
>
> > ... or was there no single guiding principle and each machine came up,
> at that level at least, in a relative vacuum, with only the machine
> interface to UNIX being the guiding principle?
>
> I stumbled into the same question last year, when doing my SysIII to RV64
> port. I managed to turn that into a somewhat chaotic discussion, mixing old
> and new, and history with ideas. From that chaotic discussion I got the
> impression that it was indeed mostly ad hoc. In context, hardware was much
> easier to boot and drive back then -- it probably was not seen as complex
> enough to warrant much research into layering and abstraction.
>
> Also bear in mind that something like a boot rom only became the norm in
> the late 70’s. Before that, one keyed in two dozen words with a tiny
> program to load the first boot stage.
>
> That said, there is an implicit layering in v7 and beyond:
>
> - “low.s" does hardware setup, incl. such stuff as setting up interrupt
> tables. As this is closely tied to the hardware, it would have been a
> custom job in each case.
>

V7 used l.s for this from research, though different names were used in
different ports (though many retain l.s too). 32V used locore.s, a
convention that all the BSDs I know of picked up and used, as well as many
BSD-derived kernels that were later rewritten to support SMP.

Oftentimes, the interrupt vector was in the lowest core addresses, and the
first part of this file was just a giant table of places to jump for all
the different architecturally defined exception and/or vectors (depending
on the architecture). Often it contained glue from an interrupt to a ISR
call as well, since there were many times where you'd share an exception,
get the interrupt "level" or "vector" from some other bit of hardware and
this code would often do the simple task of offsetting into a table and
jumping.

And it also had the "start" entry point for the whole kernel. And
frequently silly aux routines like 'doadump' and the bcopy/fubyte(etc)/ and
context switching code, which is also in assembler, often ended up there as
well. Finally, it was a place to have various bits of storage that the
kernel needed to bootstrap whatever VM was there.

- “mch.s” (later also mch.c) has the basic routines that are hardware
> dependent (switching stacks, changing priority levels and modes, etc.). It
> also has emulation for ‘missing’ instructions, such as floating point ops
> where this is not available in hardware. Same as above, I think. Maybe h/w
> related memory protection operations should live here as well, but the
> hardware was still quite divergent in this area in the 70’s and early 80’s.
>

32V called this machdep.c, which all the BSDs inherited. While machine
dependent, it tended to be slightly more portable and was for stuff that
could be written in C...  Agreed on the very divergent part though.


> - low-level device drivers live in the ‘dmr’ or (later) ‘io’ directory.
> Here there is some standardisation, as all device drivers must conform to
> the (char/block) device switch APIs. It seems to me that most of these
> drivers were written by taking one that was similar to what needed to be
> written and to start from there. Maybe this is still how it works in Linux
> today.
>

To be fair, V7 was released in a time where there were no common protocol
devices: There was no USB, bluetooth, SATA, SCSI, etc that had divergent
drivers to talk to the hardware, but a common transport layer for the
protocol. Even MSCP and TSCP, which were the first inklings of this, were a
controller interface, not a transport one. The one SCSI driver from this
era I've looked at implemented all the SCSI protocol itself. Thankfully the
controller had a microcontroller for dealing with the physical signalling
bits (unlike a different card for my DEC Rainbow which did it all in
hardware by bit-banging I/O ports).


> - To the extent that there is such a thing as 'high-level device drivers’
> in early Unix, the structure is less clearly visible. The file system (and
> there was only one at the time of v7) is placed between the block device
> switch and the mount table so to speak. This was structured enough that
> splicing in other file systems seems to have been fairly easy in the early
> 80’s (the splicing in, not the writing of the file system itself, of
> course). Starting with 8th edition, the ‘file system switch’ created a
> clear API for multiple file systems. Arguably, the ‘tty’ subsystem is also
> a ‘high-level device driver’, but this one lives as custom code together
> with the serial port device drivers. Also in 8th Edition, ‘streams' were
> introduced. One could think of this as a structured approach to high-level
> device drivers for character mode devices, incl. the ’tty’ subsystem.
>

Yes. It took a long time for there to even be common disk partition
handling code. For a long time (and certainly in the V7 ports to PCish
boxes) all that was in the driver, and usually cut and pasted from driver
to driver.  It was only later that better abstraction arose. Network
stacks, and the like, were later inventions.


> - I don’t think there was ever anything in early Unix that merged
> ’streams’ and the 'file system switch' into a single abstraction (but maybe
> 9P did?).
>

I think you're right. They barely had a file system switch... And in the
BSD side of the Unix world Network and File system were two different
beasts.

> Where I'm trying to put this sort of knowledge into use is I'm starting
> to spec out a kernel bootstrap for the RPi Pico and Pine64 Ox64 boards
> (ARM32 and RISCV64 respectively) that is not only sufficient to start a
> V7-ish kernel on each, but that are ultimately based on the same design,
> varying literally only where the hardware strictly necessitates it, but
> similar enough that reading the two assembly files side by side yields
> essentially the exact same discrete operations.
>
> I have a similar interest, but to avoid the same chaos as I created
> before, I’ll respond to this with a pm.
>

I'd be keen to understand this, but it's mostly a passing fancy...

Warner

[-- Attachment #2: Type: text/html, Size: 7638 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* [TUHS] Re: UNIX "Machine Layer" Standards
@ 2023-04-20 15:57 Paul Ruizendaal
  2023-04-20 18:57 ` segaloco via TUHS
  0 siblings, 1 reply; 9+ messages in thread
From: Paul Ruizendaal @ 2023-04-20 15:57 UTC (permalink / raw)
  To: tuhs

Hi Matt,

I’ve responded on list about the early unix development process as I understand it, but I want to avoid discussing things that are not directly related to the history of Unix. Hence this PM as well.
 
> Where I'm trying to put this sort of knowledge into use is I'm starting to spec out a kernel bootstrap for the RPi Pico and Pine64 Ox64 boards (ARM32 and RISCV64 respectively) that is not only sufficient to start a V7-ish kernel on each, but that are ultimately based on the same design, varying literally only where the hardware strictly necessitates it, but similar enough that reading the two assembly files side by side yields essentially the exact same discrete operations.

I have a similar interest, working with early Unix and modern RiscV hardware for a compare and contrast experience.

- My development targets are (i) an FPGA based RV32 SoC implementation, (ii) a Sipeed D1 RV64GC board and shortly (iii) a Pine64 Pinetab-V.

- My software targets are: (a) xv6-rv, (b) SysIII, (c) Linux, (d) experiments around SysIII

Linux is for me a secondary target, just for comparison and to see if ideas are “Linux capable”. I’m not overly interested in Arm at the moment.


My ideas are still evolving, but currently more or less along the below lines:

- Boot rom loads SPL, this is custom in each case and set by the SoC's designers.

- SPL initialises DRAM system and loads next stage. Unfortunately, this too would seem to be quite system specific, but the BSP should provide the baseline for this. As BSP’s are often a mess, milage may vary.

- The next stage is a hybrid of BBL, OpenSBI and Virtio. The idea is to provide a standard abstraction layer that all of my software targets can work with. This idea is used for the FPGA target and allows booting a Linux kernel with just the generic Virtio device drivers (so far just disk and console).

- The last layer is the classical OS layer. If I get it right, each OS can run on all h/w targets without customisation.

At the moment I’m playing with USB, and how that might layer into the structure of V7, SysIII or 8th Edition -- and also the above.









^ permalink raw reply	[flat|nested] 9+ messages in thread

* [TUHS] Re: UNIX "Machine Layer" Standards
@ 2023-04-20 14:56 Paul Ruizendaal
  2023-04-20 16:04 ` Warner Losh
  0 siblings, 1 reply; 9+ messages in thread
From: Paul Ruizendaal @ 2023-04-20 14:56 UTC (permalink / raw)
  To: tuhs


> Date: Mon, 10 Apr 2023 18:27:51 +0000
> From: segaloco

> ... or was there no single guiding principle and each machine came up, at that level at least, in a relative vacuum, with only the machine interface to UNIX being the guiding principle?

I stumbled into the same question last year, when doing my SysIII to RV64 port. I managed to turn that into a somewhat chaotic discussion, mixing old and new, and history with ideas. From that chaotic discussion I got the impression that it was indeed mostly ad hoc. In context, hardware was much easier to boot and drive back then -- it probably was not seen as complex enough to warrant much research into layering and abstraction.

Also bear in mind that something like a boot rom only became the norm in the late 70’s. Before that, one keyed in two dozen words with a tiny program to load the first boot stage.

That said, there is an implicit layering in v7 and beyond:

- “low.s" does hardware setup, incl. such stuff as setting up interrupt tables. As this is closely tied to the hardware, it would have been a custom job in each case.

- “mch.s” (later also mch.c) has the basic routines that are hardware dependent (switching stacks, changing priority levels and modes, etc.). It also has emulation for ‘missing’ instructions, such as floating point ops where this is not available in hardware. Same as above, I think. Maybe h/w related memory protection operations should live here as well, but the hardware was still quite divergent in this area in the 70’s and early 80’s.

- low-level device drivers live in the ‘dmr’ or (later) ‘io’ directory. Here there is some standardisation, as all device drivers must conform to the (char/block) device switch APIs. It seems to me that most of these drivers were written by taking one that was similar to what needed to be written and to start from there. Maybe this is still how it works in Linux today.

- To the extent that there is such a thing as 'high-level device drivers’ in early Unix, the structure is less clearly visible. The file system (and there was only one at the time of v7) is placed between the block device switch and the mount table so to speak. This was structured enough that splicing in other file systems seems to have been fairly easy in the early 80’s (the splicing in, not the writing of the file system itself, of course). Starting with 8th edition, the ‘file system switch’ created a clear API for multiple file systems. Arguably, the ‘tty’ subsystem is also a ‘high-level device driver’, but this one lives as custom code together with the serial port device drivers. Also in 8th Edition, ‘streams' were introduced. One could think of this as a structured approach to high-level device drivers for character mode devices, incl. the ’tty’ subsystem.

- I don’t think there was ever anything in early Unix that merged ’streams’ and the 'file system switch' into a single abstraction (but maybe 9P did?).


> Where I'm trying to put this sort of knowledge into use is I'm starting to spec out a kernel bootstrap for the RPi Pico and Pine64 Ox64 boards (ARM32 and RISCV64 respectively) that is not only sufficient to start a V7-ish kernel on each, but that are ultimately based on the same design, varying literally only where the hardware strictly necessitates it, but similar enough that reading the two assembly files side by side yields essentially the exact same discrete operations.

I have a similar interest, but to avoid the same chaos as I created before, I’ll respond to this with a pm.





^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2023-04-21 20:36 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-04-21 14:37 [TUHS] Re: UNIX "Machine Layer" Standards Noel Chiappa
2023-04-21 17:58 ` John Cowan
2023-04-21 20:36   ` Clem Cole
  -- strict thread matches above, loose matches on Subject: below --
2023-04-20 15:57 Paul Ruizendaal
2023-04-20 18:57 ` segaloco via TUHS
2023-04-20 20:18   ` Steve Nickolas
2023-04-20 14:56 Paul Ruizendaal
2023-04-20 16:04 ` Warner Losh
2023-04-20 19:04   ` segaloco via TUHS

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).