[TUHS] 2.11BSD on a Z180 (was: merry christmas)

[TUHS] 2.11BSD on a Z180 (was: merry christmas)

[Johnny Billquist]

On 2016-12-29 03:00, Nick Downing <downing.nick at gmail.com> wrote:

> I will let you know when I get
> it working :) It's not a current focus, but I will return to it someday.
> In the meantime, I'm putting it on bitbucket, so others will be able to
> pick it up if they wish. However, this also isn't my current focus, it's
> there, but it's not documented.
 >
> The IAR compiler on the Z180 supports a
> memory model similar to the old "medium" memory model that we used to
> use with Microsoft or Turbo C on DOS machines, that is, multiple code
> segments with a single data segment. Yes, the Z180 compiled C code is
> larger than the PDP-11 compiled C code, but luckily you can have
> multiple code segments, which you cannot (easily) have on the PDP-11.
 >
> Unfortunately code and data segments share the same 64 kbyte logical
> address space, so what I did was to partition the address space into 4
> kbytes (always mapped, used for interrupt handlers, bank switching
> routines, IAR compiler helper routines, etc), 56 kbytes (kernel or
> current process data and stack) and 4 kbytes (currently executing
> function). The currently executing function couldn't be more than 4
> kbytes and couldn't cross a physical 4 kbyte boundary due to the
> hardware mapping granularity, but this was acceptable in practice.
 >
> I got
> the Unix V7 clone working OK under this model and then added the
> networking, so although it was a bit of a dogs breakfast, it proves the
> concept works. My memory management left a fair bit to be desired (too
> much work to fix) however I think porting 2.11BSD would solve this
> problem since it works in the PDP-11 under split I/D, which has similar
> constraints except the 4 kbyte code constraint. My understanding is
> 2.11BSD is actually a cut down 4.3BSD running on the HAL from 2.xxBSD, I
> would like to audit each change from 4.3BSD to make sure I agree with
> it, so essentially my project would be porting 4.3BSD rather than
> 2.11BSD. But I'd take the networking stack and possibly a lot more code
> from 2.11BSD, since it is simplified, for instance the networking stack
> does not use SYN cookies. cheers, Nick

Having written quite some code on the Z180, as well as god knows how 
much code on the PDP-11, I'm going to agree with Peter Jeremy in that I 
do not believe 2.11BSD can be made to run on a Z180. (Well, of course, 
anything is possible, you could just write a 68000-emulator, for 
example, but natively... no.)

Unix V7 is miles from 2.11BSD. Unix V7 can run on very modest PDP-11 
models. 2.11BSD cannot be made to run on a PDP-11 without split I/D 
space, which effectively gives you 128K of address space to play with, 
in addition to the overlaying done with the MMU remappings.
The MMU remappings might be possible to emulate enough with the segment 
registers of the Z180 for the Unix needs, but the split I/D space just 
won't happen.

2.9BSD was the last version (I believe) which ran on non split-I/D machines.

	Johnny

 >
> On Wed, Dec 28, 2016 at 6:14 PM, Peter Jeremy <peter at rulingia.com> wrote:
>> On 2016-Dec-25 17:21:31 -0500, Steve Nickolas <usotsuki at buric.co> wrote:
>>> On Mon, 26 Dec 2016, Nick Downing wrote:
>>>> I became frustrated with the limitations of both UZI and NOS and decided to
>>>> port 2.11BSD to the cash register as the next step, my goal was (a) make it
>>>> cross compile from Linux to PDP-11, (b) check it can build an identical
>>>> release tape through cross compilation, (c) port it to Z80 using my
>>>> existing cross compiler.
>>> A Z180 is powerful enough to run 2.11BSD? o.o;
>> I suspect shoe-horning 2.11BSD onto a Z180 would be difficult - 2.11BSD
>> on a PDP-11 requires split I+D and has kernel and userland in separate
>> address spaces.  Even with that, keeping the non-overlay part of the
>> kernel in 64KB is difficult.  Equivalent Z180 code is going to be much
>> larger than PDP-11 code.
>>
>> I'd be happy to be proved wrong.
>>
>> --
>> Peter Jeremy


-- 
Johnny Billquist                  || "I'm on a bus
                                   ||  on a psychedelic trip
email: bqt at softjar.se             ||  Reading murder books
pdp is alive!                     ||  tryin' to stay hip" - B. Idol

[TUHS] 2.11BSD on a Z180 (was: merry christmas)

[Ron Natalie]

Yes.   Before the networking code in, the kernels on the non-split I/D machines got by with using a couple of the segments to make code overlays.    The mbufs required a overlay register of their own and once you got the minimal data segment as well as the top one for the stack, you just couldn't do it with 8 total.   With Split I/D you had 8 code and 8 data segments and that you could make work.

It was a boon for me.   I had started with Noel's MIT C Gateway but abandoned it for my own "Little OS" based version (Noel had been exiled to the Bahamas or some warm place and the MIT code was lacking in some necessary featuers for us).   I took all the non-split I/D machines around the lbs... everything from PDP-11/23's and 11/24's up to some 11/34's we had.   Eventually, UNIX got too big for even the 11/70's that were left, and those were turned into ( rather compute heavy) routers as well.

The 11/34's were an another amusing piece of recycling.   A contractor sold those to the government to be graphics remotes for our CDC 7600 mainframe.   Each one had a Vector General graphics station, an 11/34 with a couple of RK05's, a punched card reader, and a DQ11 and 56KB modem.   The contractor couldn't get the things to work with the mainframes, and Mike Muuss's standard answer to extra compute hardware was to put UNIX on it.   We did, and Mike wrote the early version of the BRL CAD for the Vector General (this was 1980, I remember him giving a talk on this at the Univ. of Delaware UUG).     We took the DQ/modems to make an early BRLNET (pre-IP) link.    Late one summer evening Mike and I wrote a version of ASTEROIDS for the system.   We finished about dawn, and when the real ballistic researchers came in that morning to play it they decided our physics was all wrong so by the time we returned later, it had been recoded.    Such was the lab in those days.

[TUHS] 2.11BSD on a Z180 (was: merry christmas)

[Nick Downing]

Yes, OK, I see that there are unanticipated problems with the 2BSD
Z180 port. Basically I had thought that the kernel ran like a
userspace program, I knew there might be overlays involved with the
code space, but I hadn't realized there are also overlays with the
data space (from what you say I guess the mbufs are accessed using a
kind of a "far" pointer). Anyway, I will check into it further.

Based on the previous day's conversations I got inspired to continue
the project, I got out the 2.11BSD toolchain which I had ported to a
modern machine as a cross compiler (actually Mac OSX although I was
using it like a poor man's Linux)... started again using Linux and a
different way of building that would require less change to the
Makefiles, fixed all compile warnings, fixed many bugs, it now
produces:
bin/ar
bin/as
bin/cc
bin/ld
bin/nm
lib/c0
lib/c1
lib/c2
lib/cpp
lib/crt0.o
lib/libc.a
lib/mcrt0.o
usr/bin/lorder
usr/bin/ranlib
usr/lib/libc_p.a

The executables above are x86-64 Linux ELF executables that work alike
to the corresponding PDP-11 2.11BSD a.out executables, except that
hard coded paths like /usr/include or /usr/lib have an extra prefix
added to them, at the moment it works like this:
  /home/nick/src/2bsd_cc.git/bin/cc contains a hard coded executable
path of /home/nick/2bsd_cc.git/lib.cpp
  /home/nick/src/2bsd_cc.git/lib/cpp contains a hard coded include
path of /home/nick/src/2bsd_cc.git/usr/include
and so on, these are set up at build time. Changes to the makefiles
are pretty minimal, in the libc source directory I had to change
things a bit to make it refer to the cross compiler rather than the
native compiler, by changing stuff like "cc" to "${CC}" and adding
stuff like:
  DESTDIR=../../..
  CC=${DESTDIR}/bin/cc
and so forth. The revised toolchain if compiled with -DPDP11 will be
more-or-less as original, so it should not break the self-hosted
compile system (up to possible minor breakage because I haven't tested
this mode yet), for instance if the original code looked like this:
  int some_function();
  ...
  some_function(a) int a; { some code }
then it would now look like this:
  int some_function PARAMS((int a));
  ...
  int some_function(a) int a; { some code }
where the PARAMS macro is set up appropriately depending on whether we
are compiling on the PDP-11 or the modern Linux machine.

One significant difference with the cross toolchain is that I have
translated the PDP-11 assembler /bin/as from assembler into C. So I
guess it will be slightly slower and use slightly more memory, but I
doubt the difference would be noticeable in practice. In future when I
merge my changes into 2.11BSD and fix any breakage, I'll probably just
delete the as0.s, as2.s and replace with as0.c, as2.c. Also I fixed
some memory allocation bugs and illegal pointer dereferences in the
other utilities that just happened to be benign on PDP-11.

Anyway, I'm pretty stoked because after successfully building the C
library, etc, with the cross toolchain, I compiled a hello world
program and copied it into the running simh system, it executed
without problems. When I compiled the same program on the self-hosted
compile system in the running simh system everything was the same
except the final executable, I will have to pay a bit of attention to
the lorder and tsort stuff when it builds the C library, to make sure
everything comes out in a predictable way for comparison.

I will test this more extensively in the coming weeks, e.g. by
building a kernel using the cross toolchain and checking it's the
same. I am nearly ready to put the cross toolchain on bitbucket, but I
just want to take a bit of care before doing "git commit" to make sure
it's not capturing any junk I don't want in there, so I will do it
later. I had better pay some attention to the family :) Happy new year
everyone.

cheers, Nick

On Fri, Dec 30, 2016 at 5:20 AM, Ron Natalie <ron at ronnatalie.com> wrote:
> Yes.   Before the networking code in, the kernels on the non-split I/D machines got by with using a couple of the segments to make code overlays.    The mbufs required a overlay register of their own and once you got the minimal data segment as well as the top one for the stack, you just couldn't do it with 8 total.   With Split I/D you had 8 code and 8 data segments and that you could make work.
>
> It was a boon for me.   I had started with Noel's MIT C Gateway but abandoned it for my own "Little OS" based version (Noel had been exiled to the Bahamas or some warm place and the MIT code was lacking in some necessary featuers for us).   I took all the non-split I/D machines around the lbs... everything from PDP-11/23's and 11/24's up to some 11/34's we had.   Eventually, UNIX got too big for even the 11/70's that were left, and those were turned into ( rather compute heavy) routers as well.
>
> The 11/34's were an another amusing piece of recycling.   A contractor sold those to the government to be graphics remotes for our CDC 7600 mainframe.   Each one had a Vector General graphics station, an 11/34 with a couple of RK05's, a punched card reader, and a DQ11 and 56KB modem.   The contractor couldn't get the things to work with the mainframes, and Mike Muuss's standard answer to extra compute hardware was to put UNIX on it.   We did, and Mike wrote the early version of the BRL CAD for the Vector General (this was 1980, I remember him giving a talk on this at the Univ. of Delaware UUG).     We took the DQ/modems to make an early BRLNET (pre-IP) link.    Late one summer evening Mike and I wrote a version of ASTEROIDS for the system.   We finished about dawn, and when the real ballistic researchers came in that morning to play it they decided our physics was all wrong so by the time we returned later, it had been recoded.    Such was the lab in those days.
>

[TUHS] 2.11BSD on a Z180 (was: merry christmas)

[Robert Swierczek]

Have you looked into apout?  It is a user level simulator for Unix
a.out binaries:
https://github.com/DoctorWkt/unix-jun72/tree/master/tools/apout

I have used it successfully for running some standalone binaries such
as (~/v1/fs/root)/bin/as.

[TUHS] 2.11BSD on a Z180 (was: merry christmas)

[Nick Downing]

Yes, I am aware of the concept, I had used a similar emulator to run CP/M
binaries on MS-DOS, specifically Microsoft Macro-80, Link-80 and Basic-80
compiler. Yes I was doing cash registers that far back :) :) It is really
convenient having access to your host filesystem (compared with attaching a
simh formatted emulated tape image, tarring to/from it, and converting
to/rom simh format and plain binary as I did yesterday).

I hadn't seen apout but it is a good idea, I was toying with doing the same
thing by cross compiling the 2.11BSD kernel to create something like User
Mode Linux (but then adding user mode PDP-11 CPU emulation since it does
not make sense to compile x86-64 2.11BSD userspace executables even though
it is theoretically possible). I don't know how stable is the ABI between
Unix V7 and BSD or between BSD versions. I suspect it is very similar but
there would be differences in things like struct stat which are bound to
cause breakage. So I think it has to run the correct kernel to get the
correct ABI, and in turn that kernel has to have null or pass thru drivers
to access the host facilities. I do hope to get this working someday,
especially since a full cross compile of 2.11BSD includes the f77 stuff
which cannot reasonably be compiled on a modern system given the compiler
is not written in C, I think it is PDP-11 assembly.

For now I am trying to reduce the amount of stuff I need to change, so my
idea is to modify simh to allow switching between Z80 and PDP-11
instruction sets (I could use the PSW so that it can process an interrupt
or syscall in PDP-11 mode and transparently return to Z80 mode when it
restores the PSW) and then just convert little pieces at a time, I could
compile a Z80 hello world program and then link it against the PDP-11 C
library for instance. So this puts off having to deal with MMU or driver
issues till later on. I am looking at changing /lib/c1 and /bin/as to add a
.z80 directive and the z80 instruction set.

cheers, Nick

On 31/12/2016 5:19 PM, "Robert Swierczek" <rmswierczek at gmail.com> wrote:

> Have you looked into apout?  It is a user level simulator for Unix
> a.out binaries:
> https://github.com/DoctorWkt/unix-jun72/tree/master/tools/apout
>
> I have used it successfully for running some standalone binaries such
> as (~/v1/fs/root)/bin/as.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20161231/8bb33261/attachment.html>

[TUHS] 2.11BSD on a Z180 (was: merry christmas)

[Warner Losh]

On Sat, Dec 31, 2016 at 3:27 AM, Nick Downing <downing.nick at gmail.com> wrote:
> I hadn't seen apout but it is a good idea, I was toying with doing the same
> thing by cross compiling the 2.11BSD kernel to create something like User
> Mode Linux (but then adding user mode PDP-11 CPU emulation since it does not
> make sense to compile x86-64 2.11BSD userspace executables even though it is
> theoretically possible). I don't know how stable is the ABI between Unix V7
> and BSD or between BSD versions. I suspect it is very similar but there
> would be differences in things like struct stat which are bound to cause
> breakage. So I think it has to run the correct kernel to get the correct
> ABI, and in turn that kernel has to have null or pass thru drivers to access
> the host facilities. I do hope to get this working someday, especially since
> a full cross compile of 2.11BSD includes the f77 stuff which cannot
> reasonably be compiled on a modern system given the compiler is not written
> in C, I think it is PDP-11 assembly.

qemu has a 'user mode' that lets one implement the 'kernel' inside
qemu so that you can execute mips binaries on an x86 with full
awareness of the host's filesystem, but with enough 'special case
hooks' that things like shared libraries use the mips .so rather than
the x86 .so. It emulates a bunch of systems, but none of them
pdp-<anything>. It has both BSD and Linux user-mode support, and
FreeBSD uses it to build armv7, armv8, mips and powerpc packages on
x86_64 hosts.

Warner