[TUHS] Re: A Census of /etc and /sys Prior to V4

[TUHS] Re: A Census of /etc and /sys Prior to V4

[Noel Chiappa]

    > From: Matt G.

    > there is a "core" file included, I wonder if kernel text is swept up in
    > that.

My _guess_ is perhaps not; the disks were really small (the UNIX people
started with an RF11, which the first DEC machine I used - a RSTS system -
also had; that was _really small - 512KB :-).

Probably it did whatever V1 did. I was not up for going to look, since I
wasn't familiar with the V1 code - but then I decided to break down and look
at it, and also create a minimal index to say what's in each module. (Here:

  https://gunkies.org/wiki/UNIX_First_Edition#Source_index

if anyone is interested. Made easier because the code is very well commented;
it's very easy to read.)

The code to take core dumps is in u1, at 'badsys:'. It dumps the user's
entire possible memory space (i.e. not just up to the 'break'), and then
(separately) the 'user' area. The system is not included. I doubt V2/V3 are
different.


    > ac and mq EAE registers are still in use in s2-bits binaries

Interesting. How did you work that out, BTW? Also, V1 seems to mandate use of
a KE11-A (use is made of it throughout the kernel).

     > but have been replaced by s1-bits.

Interesting; how did you work that out? V3's core (V):

  http://squoze.net/UNIX/v3man/man5/core

doesn't give the format, just says "The actual format of the information is
complicated because it depends on what hardware is present (EAE,
floating-point option)". Do you have C3's db(I) source? Oh, wait, TUHS has
what claims to be V2's db source:

  https://minnie.tuhs.org/cgi-bin/utree.pl?file=V2/cmd/db1.s

but it actually seems to be later; it's conditionalized for having the FPP.
So it must be for a machine running the -11/45 - which we seem to have
decided is V3?

The header for TUHS' V2 says: "The files in cmd/ are recreated from the text
fragments found on the file s1-bits.gz." Which agrees with your take:

    > All in all that pegs the s1-bits fragments as being closer in character
    > to V3

That's all for the moment...

	Noel

[TUHS] Re: A Census of /etc and /sys Prior to V4

[segaloco via TUHS]

> > ac and mq EAE registers are still in use in s2-bits binaries
> > but have been replaced by s1-bits.
> 
> Interesting. How did you work that out, BTW?

Regarding the EAE registers, in as19.s[1], there is this set of mappings:

012717;000000;01;177570 /csw
015176;000000;01;177300 /div
003270;000000;01;177302 /ac
051750;000000;01;177304 /mq
052224;000000;01;177306 /mul
073470;000000;01;177310 /sc
074620;000000;01;177311 /sr
054752;000000;01;177312 /nor
047000;000000;01;177314 /lsh
004500;000000;01;177316 /ash

Among others, here we have the mappings for ac, mq, mul, and div.

In the disassembled source of df(I) from s2-bits[2], at the second 9 label, ac, mq, div, and mul are used to perform a division by 10 (I think, my PDP-11-fu isn't *that* strong yet):

9:
	clr	ac
	mov	$10.,div
	mov	ac,-(sp)
	tst	mq
	beq	2f
	jsr	pc,9b

In the disassembler output (I used pdp11dasm[3]) these values were all the same addresses as the above table from as(I).

In the source df.s under the V2/cmd section[4], this is instead replaced by use of the dvd opcode:

9:
	clr	r2
	dvd	$10.,r2
	mov	r3,-(sp)
	mov	r2,r3
	beq	2f
	jsr	pc,9b

I've seen this sort of thing consistently as I comb through files in s2-bits, using stuff from s1-bits and V5 as references.

[1] - https://www.tuhs.org/cgi-bin/utree.pl?file=V2/cmd/as19.s
[2] - https://gitlab.com/segaloco/v2src/-/raw/master/cmd/df.s
[3] - https://github.com/caldwell/pdp11dasm
[4] - https://www.tuhs.org/cgi-bin/utree.pl?file=V2/cmd/df.s

- Matt G.

[TUHS] Re: A Census of /etc and /sys Prior to V4

[segaloco via TUHS]

There are the s2-bits binaries that I'm slowly disassembling.  I don't think there is a kernel image down in there though.  That said, there is a "core" file included, I wonder if kernel text is swept up in that.

In any case, if there are other binaries floating around, disassembly is one of my oldest techie hobbies, so happy to go mining for information.

Also, I've mentioned it a few times but in busier emails: The s1-bits source code fragments appear to be later than the s2-bits binaries, evidenced both by /etc/passwd subbed for /etc/uids in the former and likewise ac and mq EAE registers are still in use in s2-bits binaries but have been replaced by s1-bits. All in all that pegs the s1-bits fragments as being closer in character to V3 while the binaries are closer in character to V2.  As my V2 disassembly progresses I'll try and put together a more detailed list of differences between these two bits of code as well as the Bashkow init and sh sources, with V5 as a later reference.

- Matt G.

------- Original Message -------
On Friday, May 19th, 2023 at 7:36 AM, jnc@mercury.lcs.mit.edu <jnc@mercury.lcs.mit.edu> wrote:


> > From: Matt G.
> 
> 
> > Given the movement of UNIX to the 11/45 and then to C, does the Third
> 
> > Edition represent a version of UNIX for the 11/45 with protection but
> 
> > written in assembly, not C?
> 
> 
> I think so (evidence detailed below). The support may not have been quite
> identical to that in V4 (e.g. there was no support for pure texts in V3 -
> below), though.
> 
> > is there any other information such as documents, code, etc. concerning
> 
> > the 11/45 assembly version?
> 
> 
> This is the real problem, of course; all we have for V3 is some man pages.
> (And in relying on them, we have to hope that they were updated to match the
> then-current system - which is not guaranteed, but in general at this point
> in time, man pages do seem to match whats's in the code.)
> 
> > Was work completed on the 11/45 kernel changes in the context of this
> 
> > version and then simply "ported" to the C version or were there
> 
> > concepts that were cropping up in one or the other and varying amounts
> 
> > of transportation back and forth as 11/45 and C aspects were
> 
> > implemented?
> 
> 
> Without a lot more information, which is now almost certainly lost, we are
> unlikely to be able to tell. But let me start by laying out what we do know.
> 
> 
> To start with, it's important to realize that support for protection (and
> relocation - i.e. memory that looks, to user code, like it's at 0,
> is actually at, say, 060000 in physical terms) in PDP-11 UNIX pre-dates the
> -11/45. DEC had a rare, and now almost forgotten "Memory Protect & Relocate"
> option for the -11/20, the KS11:
> 
> https://gunkies.org/wiki/KS11_Memory_Protection_and_Relocation_option
> 
> What exactly it did, and how, is now uncertain (no documentation, or code
> that used it, appeats to have survived - all we have are a couple of vague
> recollections), but it is certain that that the UNIX group's -11/20 had it:
> 
> https://www.bell-labs.com/usr/dmr/www/odd.html
> 
> and Ken has said that he wrote the code to use it.
> 
> It's also important to remember that not all the machines running UNIX would
> have had their hardware updated simultaneously: e.g. the patent group's
> -11/20 would not have needed the KS11 as much, since it was runnng mature
> applications. So UNIX was probably conditionalized to run with and without
> the KS11. As late as V3, there were apparently still UNIX machines without
> relocation hardware: "The purpose of this command is to simplify the
> preparation of object programs for systems which have no relocation
> hardware.":
> 
> http://squoze.net/UNIX/v3man/man1/reloc
> 
> When the support for the KS11 appeared is uncertain. It's not in the extant
> V1 code; but V2 seems to have had it: "the current system, which has
> relocation and protection hardware":
> 
> http://squoze.net/UNIX/v2man/man5/core
> 
> V2 also seems to have started looking forward to the -11/45 - "a trap is
> simulated by the floating point simulator" (ditto); "if they correspond to
> 11/45 floating point instructions":
> 
> http://squoze.net/UNIX/v2man/man3/fptrap
> 
> It is possible that they already had the -11/45 at this point, but I would
> tend to doubt it: "immediate mode ((pc)+) is not supported, since the
> PDP-11/45 handbook is not clear on what to do about it." (If they had it, a
> simple experiment would have produced the answer.) And "Double precision
> results are probably less correct than the hardware will be" (note tense).
> (All from v2man/man3/fptrap.)
> 
> 
> V3 seems to have the -11/45: "it depends on what hardware is present (EAE,
> floating-point option)":
> 
> http://squoze.net/UNIX/v3man/man5/core
> 
> The "floating-point option" would only have been on the -11/45. (And again we
> see that V3 still ran on -11/20's; the -11/45 would not have had an EAE:
> 
> https://gunkies.org/wiki/KE11-A_Extended_Arithmetic_Element
> 
> since all the EAE operations - except normalization, but that's only needed
> for floating-point - were in the basic -11/45.)
> 
> Probably the protection and relocation provided to UNIX processes on the
> 11/45 was very similar to that provided with the KS11. Do note that thememory
> management was not exactly the same as V4's: "In the future the text segment
> will be write-protected and shared.":
> 
> http://squoze.net/UNIX/v3man/man5/a.out
> 
> However, it was keeping multiple processes in main memory at the same time:
> "only processes whose core images are on disk have visible names":
> 
> http://squoze.net/UNIX/v3man/man8/ps
> 
> 
> So we can actually tell a fair amount about the evolution through V2 and V3
> from the few scraps that are left to us. I do live in hope that a V2 or V3
> listing will turn up one day; the system changed a lot in that period, and
> many questions aren't answered definitively by the man pages.
> 
> (One big one is details of how the process' address space was laid out -
> ld(III) and exec(II) simply say nothing at all. I assume it started at 0 -
> but who knows? In V1, it must have started at a higher address - as on
> MINI-UNIX:
> 
> https://gunkies.org/wiki/MINI-UNIX#Implementation_details
> 
> which I am fairly familiar with - but again, neither V1's ld(III) or exec(II)
> mentions this detail. I suppose I could work it out from the V1 source, but
> I'm not that interested... :-))
> 
> It is possible that the evolution started with just protection (if the KS11
> could do that), and relocation was added later. It seems clear that the
> step from the KS11 to the -11/45 was probably not large.
> 
> 
> If anyone has a V2 or V3 listing, please sing up! That would be an
> incredibly valuable thing to add to the historical record.
> 
> Noel

[TUHS] Re: A Census of /etc and /sys Prior to V4

[Noel Chiappa]

    > From: Matt G.

    > Given the movement of UNIX to the 11/45 and then to C, does the Third
    > Edition represent a version of UNIX for the 11/45 with protection but
    > written in assembly, not C?

I think so (evidence detailed below). The support may not have been _quite_
identical to that in V4 (e.g. there was no support for pure texts in V3 -
below), though.

    > is there any other information such as documents, code, etc. concerning
    > the 11/45 assembly version?

This is the real problem, of course; all we have for V3 is some man pages.
(And in relying on them, we have to hope that they were updated to match the
then-current system - which is not guaranteed, but in general at this point
in time, man pages do seem to match whats's in the code.)

    > Was work completed on the 11/45 kernel changes in the context of this
    > version and then simply "ported" to the C version or were there
    > concepts that were cropping up in one or the other and varying amounts
    > of transportation back and forth as 11/45 and C aspects were
    > implemented?

Without a lot more information, which is now almost certainly lost, we are
unlikely to be able to tell. But let me start by laying out what we _do_ know.


To start with, it's important to realize that support for protection (and
relocation - i.e. memory that looks, to user code, like it's at 0,
is actually at, say, 060000 in physical terms) in PDP-11 UNIX _pre-dates_ the
-11/45. DEC had a rare, and now almost forgotten "Memory Protect & Relocate"
option for the -11/20, the KS11:

  https://gunkies.org/wiki/KS11_Memory_Protection_and_Relocation_option

What exactly it did, and how, is now uncertain (no documentation, or code
that used it, appeats to have survived - all we have are a couple of vague
recollections), but it is certain that that the UNIX group's -11/20 had it:

   https://www.bell-labs.com/usr/dmr/www/odd.html

and Ken has said that he wrote the code to use it.

It's also important to remember that not all the machines running UNIX would
have had their hardware updated simultaneously: e.g. the patent group's
-11/20 would not have needed the KS11 as much, since it was runnng mature
applications. So UNIX was probably conditionalized to run with and without
the KS11. As late as V3, there were apparently still UNIX machines without
relocation hardware: "The purpose of this command is to simplify the
preparation of object programs for systems which have no relocation
hardware.":

  http://squoze.net/UNIX/v3man/man1/reloc

When the support for the KS11 appeared is uncertain. It's not in the extant
V1 code; but V2 seems to have had it: "the current system, which has
relocation and protection hardware":

  http://squoze.net/UNIX/v2man/man5/core

V2 also seems to have started looking forward to the -11/45 - "a trap is
simulated by the floating point simulator" (ditto); "if they correspond to
11/45 floating point instructions":

  http://squoze.net/UNIX/v2man/man3/fptrap

It is possible that they already had the -11/45 at this point, but I would
tend to doubt it: "immediate mode ((pc)+) is not supported, since the
PDP-11/45 handbook is not clear on what to do about it." (If they had it, a
simple experiment would have produced the answer.) And "Double precision
results are probably less correct than the hardware will be" (note tense).
(All from v2man/man3/fptrap.)


V3 seems to have the -11/45: "it depends on what hardware is present (EAE,
floating-point option)":

  http://squoze.net/UNIX/v3man/man5/core

The "floating-point option" would only have been on the -11/45. (And again we
see that V3 still ran on -11/20's; the -11/45 would not have had an EAE:

  https://gunkies.org/wiki/KE11-A_Extended_Arithmetic_Element

since all the EAE operations - except normalization, but that's only needed
for floating-point - were in the basic -11/45.)

Probably the protection and relocation provided to UNIX processes on the
11/45 was very similar to that provided with the KS11. Do note that thememory
management was not exactly the same as V4's: "In the future the text segment
will be write-protected and shared.":

  http://squoze.net/UNIX/v3man/man5/a.out

However, it was keeping multiple processes in main memory at the same time:
"only processes whose core images are on disk have visible names":

  http://squoze.net/UNIX/v3man/man8/ps


So we can actually tell a fair amount about the evolution through V2 and V3
from the few scraps that are left to us. I do live in hope that a V2 or V3
listing will turn up one day; the system changed a lot in that period, and
many questions aren't answered definitively by the man pages.

(One big one is details of how the process' address space was laid out -
ld(III) and exec(II) simply say nothing at all. I assume it started at 0 -
but who knows? In V1, it must have started at a higher address - as on
MINI-UNIX:

 https://gunkies.org/wiki/MINI-UNIX#Implementation_details

which I am fairly familiar with - but again, neither V1's ld(III) or exec(II)
mentions this detail. I suppose I could work it out from the V1 source, but
I'm not _that_ interested... :-))

It is possible that the evolution started with just protection (if the KS11
could do that), and relocation was added later. It seems clear that the
step from the KS11 to the -11/45 was probably not large.


If anyone has a V2 or V3 listing, please sing up! That would be an
_incredibly_ valuable thing to add to the historical record.

	Noel

[TUHS] Re: A Census of /etc and /sys Prior to V4

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 4676 bytes --]

Matt,

This is wonderful work.  Thank you.

Clem

On Mon, May 15, 2023 at 12:28 AM segaloco via TUHS <tuhs@tuhs.org> wrote:

> I've just completed the Fourth Edition pass of commits in my manual
> history repository here: https://gitlab.com/segaloco/mandiff
>
> Something I've kept a particular eye on is what the landscape looked like
> on the filesystems over the early years of development.  Here are some of
> those observations with a few areas perhaps requiring further illumination:
>
> In the first two editions, there was a file, /etc/uids, which mapped
> simply a username to a uid.  The reason was presumably due to the plaintext
> passwords in /etc/passwd at the time.  The arrival of crypt(III) and
> related functionality rendered this moot by the time of V3.  Additional
> GECOS information is first spotted in /etc/ident in V2 but by V3 has also
> found home in /etc/passwd in the GECOS field today used often for a user's
> full name.  The s1-bits source codes refer to /etc/passwd where
> disassembled s2-bits binaries refer to /etc/uids still, dating both sets of
> code.
>
> References to /etc/motd first appear in the V2 manual from what I could
> find, so that may not have been around in V1.  Additionally, after V1 many
> files are moved from /etc to locations under /usr such as ascii and kbd
> moving to /usr/pub and roff's suftab moving to /usr/lib.  It seems in the
> First Edition, manual section VII mapped to /etc itself it seems, with etc
> and misc in the manual being synonymous.
>
> So all in all it seems, in terms of support files anyhow, /etc wound up
> smaller by the advent of the C system, at which point init beings using
> /etc/rc and the directory begins to expand again.
>
> Another directory of interest is /sys for a few reasons.  First, this
> directory serves different purposes depending on your kernel these days,
> with BSD systems storing system source code here whereas Linux provides a
> kernel interface filesystem.  I'm not sure what other contemporary systems
> may use this for, but from V3 and back, this was another RK disk mounted in
> addition to /usr.  This /sys directory appeared to contain the manuals,
> source code to system components including the commands, kernel,
> bootloader, and languages, and a copy of the kernel image referenced down
> in the source tree.
>
> In total I've identified the following directories: c, fort, lang, man,
> mdec, source, sys.  Most names should be obvious from later releases, with
> lang being a parent directory that contained bdir and mdir B and m6
> languages respectively.  My guess is that when RP support was made workable
> in V4, there was no longer a need to segregate data amongst RKs like this
> so /sys was merged into /usr, leading to the later structure we see in
> V4-V6.  Of note, this structure is implied in CB-UNIX still in the path
> names of the source code available on the archive.  The kernel is found at
> /tsys/sys/ much like the kernel in V1-V3 living at /sys/sys.
>
> One thing I haven't been able to glean in the process is precisely how the
> command and library source code was stored in these very early versions.
> The kernel in T.R. Bashkow's analysis is implied to be stored in files
> u[0-9x].s, and command source files at least exist somewhere as the command
> followed by .s.  As of V5, the command, syscall wrapper, and library source
> codes are split up amongst a number of directories with names such as s1,
> s2, s3, etc. under source.  By V7, this has taken on the cmd/lib/sys
> structure of later releases.
>
> Finally, just a general curiosity the version study involved has raised.
> Given the movement of UNIX to the 11/45 and then to C, does the Third
> Edition represent a version of UNIX for the 11/45 with protection but
> written in assembly, not C?  I've seen one handwritten document that makes
> mention of some of this, but is there any other information such as
> documents, code, etc. concerning the 11/45 assembly version?  Was work
> completed on the 11/45 kernel changes in the context of this version and
> then simply "ported" to the C version or were there concepts that were
> cropping up in one or the other and varying amounts of transportation back
> and forth as 11/45 and C aspects were implemented?
>
> As always, thanks for keeping up, hopefully I can get this repository up
> to V6 soon, then the real branching fun begins.  The V3 to V4 changes are
> hopefully the last time the commit diffs have major noise, what with the
> conversion from roff to nroff.  I suspect transitions to macro packages
> later won't be as bad.
>
> - Matt G.
>
-- 
Sent from a handheld expect more typos than usual

[-- Attachment #2: Type: text/html, Size: 5281 bytes --]