[TUHS] Early Internet work (Was: History of select(2))

[TUHS] Early Internet work (Was: History of select(2))

[Noel Chiappa]

    > From: Johnny Billquist

    >> everyone working on TCP/IP heard about Version 4 shortly after the
    >> June, 1978 meeting.

    > Over a year before any documents said anything about it.

Incorrect. It's documented in IEN-44, June 1978 (written shortly after the
meeting, in the same month).


    > I'm sure people were doing networking protocols and stuff earlier, but
    > it wasn't the TCP/IP we know and talk about today

People were working on Unix in 1977, but it's not the same Unix we know and
talk about today. Does that mean it's not Unix they were working on?


    >> there were working implementations (as in, they could exchange data with
    >> other implementations) of TCP/IPv4 by January 1979 - see IEN 77.
                                       ^^
    > But not TCP4 then.

I just specified that it was v4 (see above).

    > thus, not interoperable with an implementation today

No, with properly-chosen addresses (because of the changes in address
handling), they probably would be.

    Noel

[TUHS] Early Internet work (Was: History of select(2))

[Noel Chiappa]

    > From: Clem Cole

    > Steve Ward's guys writing Trix hacked together a compiler, assembler and
    > the like.

All of which I have the source for - just looked through it.

    > If memory serves me, tjt wrote the assembler

I have the NROFF source for the "A68 Assembler Reference", and it's by James
L. Gula and Thomas J. Teixeira. It says that "A68 is an edit of the MICAL
assembler also written by Mike [Patrick].".

    > Jack Test did much of the compiler and again IIRC that was based on PCC.

I dunno, I'm not familiar with PCC, so I can't say. It definitely looks very
different from the Ritchie C compiler.

	  Noel

[TUHS] Early Internet work (Was: History of select(2))

[Clem Cole]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 693 bytes --]

On Mon, Jan 30, 2017 at 11:15 AM, Noel Chiappa <jnc at mercury.lcs.mit.edu>
wrote:

>
>     > If memory serves me, tjt wrote the assembler
>
> I have the NROFF source for the "A68 Assembler Reference", and it's by
> James
> L. Gula and Thomas J. Teixeira.

​Indeed.   Thomas J. Teixeira - aka tjt - who would be my office mate at
Masscomp and Stellar.   We would work together 3 or 4 times.   He's good
guy and good friend.

Clem​

PS Glad to hear to you saved Trix.  I always thought it there were some
cool ideas in it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170130/a490ea79/attachment.html>

[TUHS] Early Internet work (Was: History of select(2))

[Clem Cole]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 517 bytes --]

On Mon, Jan 30, 2017 at 11:15 AM, Noel Chiappa <jnc at mercury.lcs.mit.edu>
wrote:

> I dunno, I'm not familiar with PCC, so I can't say. It definitely looks
> very
> different from the Ritchie C compiler.
>

To many beers ago, I'm not sure I would recognize the differences by
inspection any more.  But Steve reads the list.   He might recognize​
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170130/838e2887/attachment.html>

[TUHS] Early Internet work (Was: History of select(2))

[Noel Chiappa]

    > From: Paul Ruizendaal

    >> the headers say they date from 1974-75.

    > Wow, that's great! That means that you have the initial version.

The file write dates are May 1979, so that's the latest it can be. There is
one folder called 'DTI' which contains an email message from someone at DTI to
someone at SRI which is dated "10 Apr 1979" so that seems to indicate that
that's indeed when they are from.

(The message says that the folder contains the source for DTI's IMP-11A
driver, which is different from UIll's, although they both descend from the
same original version.)

    > Possibly it is V5 not V6

Nope, definitely V6 here.

    > All my leads for the 1975 version of this code base came up dry and I
    > feared it lost.

I could have sworn that I'd seen _listings_ of the code in a UIllinois
document about NCP Unix that I had found (and downloaded) on the Internet, but
I can't find them here now. I did look again and found:

  "A Network Unix System for the Arpanet", by Karl C. Kelley, Richard Balocca,
  and Jody Kravitz

but it doesn't contain any sources.

    > it may contain the first version of 'mbufs'

It might - the code is conditionalized for "UCBUFMOD" all over the place.


    > Yes, a 'history' file seems to have been common practice at BBN. The
    > kernel would have had many modifications:
    > - the 'ports' extension from Rand

Yes.

    > - the 'await' extension by Jack Haverty

Yup.

    > - an 1822-driver

Yes (also by Haverty) - although IMP11-A drivers are all over the place, there
are two different ones in the NCP Unix alone.

    > - possibly, an Autodin II network driver

Didn't see one.

    > - possibly, shared memory extensions

Yes, there are two module in 'ken', map_page.c and set_lcba.c (I was unable to
work out what 'LCBA' stood for) which seem to do something with mapping.

    > It might even have some NCP code in it

Yes, there's an 'ncpkernel' directory.

    > There seem to have been two versions of the BBN modified kernel. One was
    > done for systems without separate I/D with stuff heavily trimmed

Yes, there's a 'SMALL' preprocessor flag which conditionally removes some
stuff.

    > The other may have extended the V6 kernel to run in separate I and D
    > spaces

That capability was present in stock V6.

     Noel

[TUHS] Early Internet work (Was: History of select(2))

[Paul Ruizendaal]

On 30 Jan 2017, at 16:34 , Noel Chiappa wrote:
> 
>>> the headers say they date from 1974-75.
> 
>> Wow, that's great! That means that you have the initial version.
> 
> The file write dates are May 1979, so that's the latest it can be. There is
> one folder called 'DTI' which contains an email message from someone at DTI to
> someone at SRI which is dated "10 Apr 1979" so that seems to indicate that
> that's indeed when they are from.

Based on that extra info I think you have a later version of Network Unix, which
is still wonderful and exciting.

> I could have sworn that I'd seen _listings_ of the code in a UIllinois
> document about NCP Unix that I had found (and downloaded) on the Internet, but
> I can't find them here now. I did look again and found:
> 
>  "A Network Unix System for the Arpanet", by Karl C. Kelley, Richard Balocca,
>  and Jody Kravitz
> 
> but it doesn't contain any sources.

The initial 1975 implementation was - in the authors' recollection - only some
one to two thousand lines of extra kernel code and one thousand for the NCP
daemon. That would make for some 50 pages of printout. It is possible.

I know the Kelley document well and sections 5 and 6 contain a fairly
detailed code walkthrough. Perhaps this is what lingered in your memory.
I suspect this (Oct 1978) code walkthrough will match with the code on your
tape.

[TUHS] Early Internet work (Was: History of select(2))

[Noel Chiappa]

    > From: Paul Ruizendaal

    > Great! I'd love to take a look at all that.

OK, it'll all be appearing once we have a chance to get organized (it's all
mixed in with personal files).

    > That is very interesting. It may be related to the V6 with NCP from
    > UoI/DTI.

I think it _is_ the V6 from UoI/DTI. The source has Gary (?) Grossman's and
Steve Holmgren's name on it, and the headers say they date from 1074-75.

    > The printout does not have the kernel modifications with it, so it would
    > be great if your archive does include that.

The archive does include the complete kernel, but i) the changes aren't listed
in any way (I forsee a lot of 'diffs', unless you just take the entire
kernel), ii) there's a file called 'history' which contains a long list of
general changes/improvements of the kernel not really related to TCP/IP, by a
long list of people, dated from the middle of '78 to the middle of '79. So it
looks like he started with a considerably modified system.

The only client code I see is User Telnet. (The MIT code has User and
Server Telnet and FTP, as well as SMTP, but it uses a wholly different
TCP interface.)

    Noel

[TUHS] Early Internet work (Was: History of select(2))

[Nick Downing]

Thanks a lot for the valuable info about C compilers everyone. I was
aware of BDS C and also Hitech C and Small C. I briefly picked up cc65
(6502 port of Small C which has an ANSI frontend and a macroassembler
and other improvements) and started trying to port it back to Z80, but
I didn't get too far. I found quite a few things there that I didn't
like, so I put aside the project. I hadn't been aware that BDS C had
such a following, I thought of it as a closed source commercial
offering of no value to me. I'm happy to see it's now open source!!
Note that I actually own copies of Manx Aztec C and IAR C which are
both pretty good commercial C compilers for Z80 and Z180 respectively,
however being closed source they are of no value to my projects, which
I hope to eventually release as open source.

A funny story arises actually in regards to IAR C and its
dongle-protection. I was working for an overseas customer who bought
the IAR C package for me to use, I think they bought 3 copies, one for
my manager, one for my colleague and one for me. I went to visit the
customer for some weeks and we did a lot of work together. When I got
home I had to attend to various things that couldn't be put off any
longer, like family matters, cleaning the house and so on. For a few
weeks I strung my manager along saying "yes I have implemented this,
yes I have done that, blah blah"... knowing it would be easy to catch
up when I had a chance to sit down and work. When that time came, I
was embarrassed to discover that I had left my dongle over there at
the customer's premises... what was I going to say?? Haha.

So, Visual Studio to the rescue, it was relatively easy to trace into
the executable and find out where the dongle checks were occurring. I
don't think they had even stripped the executable, which made things a
bit easier, although the dongle stuff was obviously a 3rd party
library since it did have some protection against being traced and so
on. Anyway, it took me about half to one day to circumvent this, and I
was very happy that I had done so, because the dongle check took quite
a while (between .1s and .3s if my memory is reliable) and it was
doing this hundreds of times during a full build. I don't know why it
was so slow, possibly because it was polling a licence server even
though I didn't have a licence server. So my builds were much faster,
and of course I didn't have the hassle of dongle protection anymore.

But getting back to the point, I'm fairly keen to standardize on a
PCC-based compiler. There are a number of reasons for this.
(1) Since I have in mind targetting multiple platforms such as PDP-11,
VAX, Z180, 8086, 80286, possibly 68K, it seems a natural choice.
(2) I like the fact that lint uses a PCC front-end, if all my
compilers use a PCC front-end then all code is interpreted in the SAME
dialect.
(3) I have a heavy focus in my project on source-to-source
transformations, presently these are a bit ad-hoc (using scripts or C
programs based on getchar() / putchar() type stuff), and I want to
upgrade them to a PCC front-end, a transformation, and a C-target PCC
backend.
(4) I have the option of upgrading to the ANSI-compliant PCC compiler
and the latest backend stuff which has a register allocator, these
features are not things which I would want to spend time on, but I
feel that people who download my eventual release packages might.
(5) The ANSI-compliant PCC compiler has a selection of backends, such
as 8086 and 80286 backends I believe, so might save me time.

Having said all that, I have been very careful to preserve
compatibility with all reasonable dialects of C. To make this work, I
do not use any features that are only available in ANSI C (except if
there is a traditional C alternative, such as ## versus /**/, which I
can check for using a macro like __STDC__), but I make sure all code
is valid ANSI C or traditional C, using the __P(()) macro and such
like. So there would be no real reason I cannot use BDS C as-is in my
project. Note I'm going to standardize on the asxxxx assemblers, and
indeed one of my next projects will be to make the PCC VAX-targeted
compiler output asxxxx rather than its own assembler code. I can do
the same for BDS C. That way, it will generate (via the assembler) the
correct *.o and relocation entries, and I can use the 4.3BSD standard
ld. For the time being, I ported 4.3BSD "as" as a VAX cross-assembler,
just to establish a baseline, but I will be changing to asxxxx when I
can.

cheers, Nick







On Mon, Jan 30, 2017 at 1:50 PM, Noel Chiappa <jnc at mercury.lcs.mit.edu> wrote:
>     > From: Paul Ruizendaal
>
>     > Great! I'd love to take a look at all that.
>
> OK, it'll all be appearing once we have a chance to get organized (it's all
> mixed in with personal files).
>
>     > That is very interesting. It may be related to the V6 with NCP from
>     > UoI/DTI.
>
> I think it _is_ the V6 from UoI/DTI. The source has Gary (?) Grossman's and
> Steve Holmgren's name on it, and the headers say they date from 1074-75.
>
>     > The printout does not have the kernel modifications with it, so it would
>     > be great if your archive does include that.
>
> The archive does include the complete kernel, but i) the changes aren't listed
> in any way (I forsee a lot of 'diffs', unless you just take the entire
> kernel), ii) there's a file called 'history' which contains a long list of
> general changes/improvements of the kernel not really related to TCP/IP, by a
> long list of people, dated from the middle of '78 to the middle of '79. So it
> looks like he started with a considerably modified system.
>
> The only client code I see is User Telnet. (The MIT code has User and
> Server Telnet and FTP, as well as SMTP, but it uses a wholly different
> TCP interface.)
>
>     Noel

[TUHS] Early Internet work (Was: History of select(2))

[Ron Natalie]

> A  funny story arises actually in regards to IAR C and its dongle-protection. I was working for an overseas customer who bought the IAR C package for me to use

Sounds even worse the Plauger's Whitesmith Software "stamp" you were supposed to stick to your machine.

[TUHS] Early Internet work (Was: History of select(2))

[Dave Horsfall]

My gripe with BDS C was that it wasn't ANSI (HiTech was), such as no 
function prototypes, no static initialisations, etc.  As Henry Spencer 
once said, "To be called a C compiler it ought to at least compile C".

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."

[TUHS] Early Internet work (Was: History of select(2))

[Paul Ruizendaal]

On 30 Jan 2017, at 3:50 , Noel Chiappa wrote:

> I think it _is_ the V6 from UoI/DTI. The source has Gary (?) Grossman's and
> Steve Holmgren's name on it, and the headers say they date from 1074-75.

Wow, that's great! That means that you have the initial version. Possibly it
is V5 not V6 (according to Holmgren they started out on V5 and ported to V6).

All my leads for the 1975 version of this code base came up dry and I feared
it lost. I think this code base is an important milestone. For example, I
think it may contain the first version of 'mbufs'. It may also contain the
first (Unix) instance of a network "work queue", which also seems to have
been a common design element of early (Unix) TCP's.

I still have one pending lead on the 1978 version of this code base.

> The archive does include the complete kernel, but i) the changes aren't listed
> in any way (I forsee a lot of 'diffs', unless you just take the entire
> kernel), ii) there's a file called 'history' which contains a long list of
> general changes/improvements of the kernel not really related to TCP/IP, by a
> long list of people, dated from the middle of '78 to the middle of '79. So it
> looks like he started with a considerably modified system.

Yes, a 'history' file seems to have been common practice at BBN. The kernel
would have had many modifications:
- the 'ports' extension from Rand
- the 'await' extension by Jack Haverty
- an 1822-driver
- possibly, an Autodin II network driver
- possibly, shared memory extensions
It might even have some NCP code in it, and if so probably derived from the
above. Indeed, lot's of diff'ing ahead to figure out what changes belong
together.

There seem to have been two versions of the BBN modified kernel. One was done
for systems without separate I/D with stuff heavily trimmed (even kernel
messages were shortened to a few bytes to save space). The other may have
extended the V6 kernel to run in separate I and D spaces and was less
anemic.

Paul

[TUHS] Early Internet work (Was: History of select(2))

[Noel Chiappa]

    > From: Nick Downing

    > This is a wonderful find

Yes, I was _very_ happy to find those tapes in my basement; up till that, I
was almost sure all those bits were gone forever.

Thanks to Chuck Guzis, whose old data recovery service made this possible - he
actually read the tape.

    > is it possible for you to read the other tapes also?

Alas, they're all of the same system. So the most we're going to get is the
files that are missing on this one due to bad spots on the tape.

	   Noel

[TUHS] Early Internet work (Was: History of select(2))

[Noel Chiappa]

    > some Unix from BBN

This one is from 1979, it includes Mike Wingfield's TCP. The 'Trix UNIX' is a
port to the 68K, probably started with something V7ish (I see "setjmp.h" in
there). Bits of the Montgomery EMACS appear to date from 1981, but the main
source files seem to be from 1984. I also have the source to 'vsh' (Visual
Shell), whatever that is.

	Noel

[TUHS] Early Internet work (Was: History of select(2))

[Noel Chiappa]

    > From: Paul Ruizendaal <pnr at planet.nl>

    >> I have this distinct memory of Dave Clark mentioning the Liza Martin
    >> TCP/IP for Unix in one of the meeting report publihed as IENs

    > It may be mentioned in this report:
    > http://web.mit.edu/Saltzer/www/publications/rfc/csr-rfc-228.pdf

Yeah, I had run across that in my search for any remnants of the Martin
stuff.

    > Would you know if any of its source code survived?

As I had mentioned, I had found some old dump tapes, and had one of them read;
it had some bad spots, but we've just (this morning) succeeding in having a
look as to what's there, and I _think_ all of the source is OK (including the
kernel code, as well as applications like server Telnet and FTP). No SCCS or
anything like that, so it's a bit hit or miss doing history - the file write
dates were preserved, but of course a lot of them would have been edited over
time to fix bugs, add features, etc.

The tape appears to contains a _lot_ of other historic material, and it's
going to take a while to sort it all out; it includes a Version 6 with NCP
from NOSC/SRI, some Unix from BBN; a BCPL compiler; a 'bind' for .rel format
files (produced by MACRO-11 and probably BCPL) written in BCPL; programs to
convert from .rel to a.out and back; an early verion of Montgomery EMACS;
another Unix from 'TMI' (whoever that might be); another UNIX that's somehow
associated with TRIX; someone's early kernel overlay stuff; an early 68K C
compiler, and also an early 8080 C compiler - just a ton of stuff (that's just
a few items that grabbed my eye as I scrolled by).

Algol, alas, appears not to be there (we probably didn't add it, because of
space reasons). The copy of LISP on this tape seem to be damaged; I do have 3
other tapes, and between them, I hope we'll be able to retrieve it.

	Noel

[TUHS] Early Internet work (Was: History of select(2))

[Paul Ruizendaal]

On 29 Jan 2017, at 18:41 , Noel Chiappa wrote:

> 
>> From: Paul Ruizendaal <pnr at planet.nl>
> 
>>> I have this distinct memory of Dave Clark mentioning the Liza Martin
>>> TCP/IP for Unix in one of the meeting report publihed as IENs
> 
>> It may be mentioned in this report:
>> http://web.mit.edu/Saltzer/www/publications/rfc/csr-rfc-228.pdf
> 
> Yeah, I had run across that in my search for any remnants of the Martin
> stuff.
> 
>> Would you know if any of its source code survived?
> 
> As I had mentioned, I had found some old dump tapes, and had one of them read;
> it had some bad spots, but we've just (this morning) succeeding in having a
> look as to what's there, and I _think_ all of the source is OK (including the
> kernel code, as well as applications like server Telnet and FTP). No SCCS or
> anything like that, so it's a bit hit or miss doing history - the file write
> dates were preserved, but of course a lot of them would have been edited over
> time to fix bugs, add features, etc.

Great! I'd love to take a look at all that.

> The tape appears to contains a _lot_ of other historic material, and it's
> going to take a while to sort it all out; it includes a Version 6 with NCP
> from NOSC/SRI, [...]

That is very interesting. It may be related to the V6 with NCP from UoI/DTI.

>> [...] some Unix from BBN
> 
> This one is from 1979, it includes Mike Wingfield's TCP.

Super! In the last couple of months I had retyped the Wingfield TCP from a
printout, some 5,000 lines done and some 700 still to go. Ah well, at least
I have now read the source with attention for each and every line. The
printout does not have the kernel modifications with it, so it would be great
if your archive does include that.

Once you're ready for that, I'd love to get a copy of those 3 versions of Unix.

Paul

[TUHS] Early Internet work (Was: History of select(2))

[Nick Downing]

This is a wonderful find, is it possible for you to read the other tapes
also?

I would be particularly interested in the early 8080 compiler, I am
actively working on something like that at the moment. I have quite
extensively reverse engineered the famous Ritchie PDP-11 C compiler to
figure out how it works, it is actually pretty straightforward and I may
write a document about it someday (the code is a bit horrible due to the
many exceptional cases added to improve the output in particular
situations, all this has to be ignored in order to get at the underlying
algorithm which is elegant).

Steven Schultz or someone else also seems to have begun a PDP-11-targeted
port of the of 4.3BSD VAX-targeted PCC backend, I can't see myself
completing this but I was considering trying to adapt the Ritchie pass2 to
understand PCC intermediate code instead of Ritchie pass1 intermediate code
and using it more-or-less as-is as a PCC backend. There is no requirement
that a PCC backend use the PCC instruction table or macro format and in
this case it would probably be simpler if it did not. But one or other of
these backends has to be ported to Z180 (~= Z80 ~= 8080) and I'd be
thrilled to have a starting point.

I will also eventually pick up the 68K compiler too although I believe some
pretty good PCC based 68K C compilers will be extant due to late versions
of BSDs having been developed on 68K (I could be wrong about the BSDs and
68K but I am sure many unices ran on 68010+ and even a few on 68000 using
the famous second CPU chip to handle faults).

cheers, Nick

On 30/01/2017 4:42 AM, "Noel Chiappa" <jnc at mercury.lcs.mit.edu> wrote:

>     > From: Paul Ruizendaal <pnr at planet.nl>
>
>     >> I have this distinct memory of Dave Clark mentioning the Liza Martin
>     >> TCP/IP for Unix in one of the meeting report publihed as IENs
>
>     > It may be mentioned in this report:
>     > http://web.mit.edu/Saltzer/www/publications/rfc/csr-rfc-228.pdf
>
> Yeah, I had run across that in my search for any remnants of the Martin
> stuff.
>
>     > Would you know if any of its source code survived?
>
> As I had mentioned, I had found some old dump tapes, and had one of them
> read;
> it had some bad spots, but we've just (this morning) succeeding in having a
> look as to what's there, and I _think_ all of the source is OK (including
> the
> kernel code, as well as applications like server Telnet and FTP). No SCCS
> or
> anything like that, so it's a bit hit or miss doing history - the file
> write
> dates were preserved, but of course a lot of them would have been edited
> over
> time to fix bugs, add features, etc.
>
> The tape appears to contains a _lot_ of other historic material, and it's
> going to take a while to sort it all out; it includes a Version 6 with NCP
> from NOSC/SRI, some Unix from BBN; a BCPL compiler; a 'bind' for .rel
> format
> files (produced by MACRO-11 and probably BCPL) written in BCPL; programs to
> convert from .rel to a.out and back; an early verion of Montgomery EMACS;
> another Unix from 'TMI' (whoever that might be); another UNIX that's
> somehow
> associated with TRIX; someone's early kernel overlay stuff; an early 68K C
> compiler, and also an early 8080 C compiler - just a ton of stuff (that's
> just
> a few items that grabbed my eye as I scrolled by).
>
> Algol, alas, appears not to be there (we probably didn't add it, because of
> space reasons). The copy of LISP on this tape seem to be damaged; I do
> have 3
> other tapes, and between them, I hope we'll be able to retrieve it.
>
>         Noel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170130/3a79b214/attachment.html>

[TUHS] Early Internet work (Was: History of select(2))

[Clem Cole]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 4148 bytes --]

On Sun, Jan 29, 2017 at 8:34 PM, Nick Downing <downing.nick at gmail.com>
wrote:

> But one or other of these backends has to be ported to Z180 (~= Z80 ~=
> 8080) and I'd be thrilled to have a starting point.
>
​In 1978/79, there was a Z80 Z Compiler from either Teletype or Western
Electric, that  IIRC was based on the Ritchie compiler.    Somebody that
worked in the IH or Columbus labs might remember such as Phil Karn might be
a good source.

That said, is you want a goos starting point the best C compiler for the
8080 was the Leor Zolman's "Brain Damaged Software" C -- aka BDS C.   Which
he has put in the public Domain:   http://www.bdsoft.com/resources/bdsc.html

There is also the classic from Ron Cain, the "Small C Compiler" [
http://www.svipx.com/pcc/PCCminipages/zc9b6ec9e.html] which was updated and
expanded by Jim Hendrix is also in the public domain:
http://www.deturbulator.org/jim.asp  ​

>
> I will also eventually pick up the 68K compiler too although I believe
> some pretty good PCC based 68K C compilers will be extant due to late
> versions of BSDs having been developed on 68K
>
​Close, but you have some of the history a little bent.    I believe that
the first C compiler for the 68000 was started at motorola ​by the 68K
developers themselves (a little known fact -- what would become the 68K was
developed on an PDP-11/7- running UNIX).  As I understand from Les Crudele
(who lead the 68K team), the compiler was based on the Ritchie compiler,
but I do not think it as every taken very far. I do not believe a C tool
chain was ever made available to their customers/UNIX community.   We
should ask some of the Moto guys about that.

So, before the chip was released and became the 68K (when there were 10
beta sites and it was an eXperiment -- i.e.  an "X-Series" chip), we had a
couple them @ Tek Labs.  Which were the basis for the Magnolia system that
Roger Bates created.   I hacked up the Ritchie compiler shortly there after
(I was not aware of the Motorola work at the time), and created something
that worked sans FP.   The assembler & linker were written by Paul
Blattner.  My compiler used a 16 bit int.

Sometime after the Tek work, Steve Ward's guys writing Trix hacked together
a compiler, assembler and the like.   If memory serves me, tjt wrote the
assembler and hacked the linker and Jack Test did much of the compiler and
again IIRC that was based on PCC.   But, they did support FP and an "int"
was 32 bits.

I believe that this was the "seed" compiler for most of the UNIX
workstations.   It was scattered across the land as it were.   I switch
over to it at some point, although I don't remember if  I switched it to a
16-bit int.   I know we talked about it and I've forgotten how that all
played out.



> (I could be wrong about the BSDs and 68K but I am sure many unices ran on
> 68010+ and even a few on 68000 using the famous second CPU chip to handle
> faults).
>

​Right the Masscomp and Apollo systems were the two most successful to use
the Forest Basket "Fixer/Executor" model.   The original Masscomp C
Compiler was based on the MIT Compiler, as was Suns.   Apollo (being
Pr1mates, used Ratfor & Pascal as their systems languages at beginning) and
did not cut C in until later.      All three firms would start compiler
teams.   The Masscomp compiler team was direct decedent of the VMS Compiler
team with many of the same players.   Those stories are better told over
beers. ;-)​

BTW: for the first 2-3 years, the reason why Masscomp was faster than Sun,
and each were using the 10Mhz 68000 chips, was the Masscomp had a real
optimizing compiler, in the same key as the DEC and DG compilers (al biet
written in C from the ground up).   The team was lead by Peter Darnell and
had Marty Jack and a number of other infamous backend folks.  Masscomp was
getting 5-25% better performance than the MIT/PCC based one, so eventually
Sun & Apollo (also have a bunch of ex-pat DECies, made similar investments).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170129/49b39896/attachment.html>

[TUHS] Early Internet work (Was: History of select(2))

[Larry McVoy]

On Sun, Jan 29, 2017 at 09:19:48PM -0500, Clem Cole wrote:
> That said, is you want a goos starting point the best C compiler for the
> 8080 was the Leor Zolman's "Brain Damaged Software" C -- aka BDS C.   Which
> he has put in the public Domain:   http://www.bdsoft.com/resources/bdsc.html

Oh, how I know that compiler.  Or knew it.  When I was a student and they 
had 30-40 people logged into the VAX 11/780, I said "screw this, it's too
slow" and I got $2000 Okidata CPM Z80 machine (why that one?  Because it
had an integrated printer and the screen had colors, wasn't B&W). 

CPM wasn't that great, I was used to BSD Unix, so I wrote a bunch of clones
of the unix commands like ls, cat, more, cp, rm, etc.  For those I used
assembler because I was trying like crazy to get each one to fit in one
sector of the floppy disk; that was faster to load and left more room on
the disk for other stuff.

Anyhoo, that BDS compiler was awesome.  Not as awesome as turbo pascal but
it wasn't pascal, if you know what I mean.

I did most of my school projects on that machine, wrote my on dial / terminal
program (why?  Because I could :), all of that in BDS C.

Anyone remember his non-standard standard I/O?

[TUHS] Early Internet work (Was: History of select(2))

[Ron Natalie]

I remember BDS C.   We were early users of that.   Amusingly I became
affiliated with a company called BDS (no relation).    We were an early
player in the internet domains and I owned BDS.COM for a while.

[TUHS] Early Internet work (Was: History of select(2))

[Clem Cole]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 544 bytes --]

On Sun, Jan 29, 2017 at 9:30 PM, Larry McVoy <lm at mcvoy.com> wrote:

> I was used to BSD Unix,


​I remember, Leor had it running in his room on floppies from Z80 box of
some flavor at the first Boston USENIX (in Fairmont Copley Place)​.   We
took Dennis to see it and he was pretty impressed.  His comment was it
reminded him of UNIX when they originally wrote it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170129/7a701b14/attachment-0001.html>

[TUHS] Early Internet work (Was: History of select(2))

[Larry McVoy]

On Sun, Jan 29, 2017 at 09:19:48PM -0500, Clem Cole wrote:
> ???Right the Masscomp and Apollo systems were the two most successful to use
> the Forest Basket "Fixer/Executor" model.   

Huh, I didn't know that.  So he's the guy who came up with that design?

[TUHS] Early Internet work (Was: History of select(2))

[Clem Cole]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 836 bytes --]

On Sun, Jan 29, 2017 at 9:32 PM, Larry McVoy <lm at mcvoy.com> wrote:

> On Sun, Jan 29, 2017 at 09:19:48PM -0500, Clem Cole wrote:
> > ???Right the Masscomp and Apollo systems were the two most successful to
> use
> > the Forest Basket "Fixer/Executor" model.
>
> Huh, I didn't know that.  So he's the guy who came up with that design?

I used have a copy of the paper. I looked for it a few years ago and could
not find it. [ I lost some stuff in the great condo flood of 1990.   I fear
that was one of the papers that got soaked].

IIRC: He presented it at the Asilomar microprocessor conference in 1980.
Robert Chew might know, as he used to run the conference.​
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170129/f87bdb41/attachment.html>

[TUHS] Early Internet work (Was: History of select(2))

[Dave Horsfall]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 872 bytes --]

On Sun, 29 Jan 2017, Clem Cole wrote:

> That said, is you want a goos starting point the best C compiler for the 
> 8080 was the Leor Zolman's "Brain Damaged Software" C -- aka BDS C.   
> Which he has put in the public Domain: 
>   http://www.bdsoft.com/resources/bdsc.html

Talk about brain-damaged software...  BDS C is irretrievably broken, with 
all sorts of things missing.  I used to use Hi-Tech C for CP/M, but I 
don't know whether they're still in business.  It was a full ANSI C 
compiler, with function prototypes, initialisation assignments, static 
variables, etc.

As Henry Spencer once said, to be called a C compiler it ought to at least 
be able to compile C...

(And as I've said on many occasions, the true test of a compiler is 
whether it can compile itself.)

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."

[TUHS] Early Internet work (Was: History of select(2))

[Larry McVoy]

On Tue, Jan 31, 2017 at 12:13:48AM +1100, Dave Horsfall wrote:
> On Sun, 29 Jan 2017, Clem Cole wrote:
> 
> > That said, is you want a goos starting??point the best C compiler for the 
> > 8080 was the Leor Zolman's "Brain Damaged Software" C -- aka BDS C. ?? 
> > Which he has put in the public Domain: 
> > ????http://www.bdsoft.com/resources/bdsc.html
> 
> Talk about brain-damaged software...  BDS C is irretrievably broken, with 
> all sorts of things missing.  

I dunno, I wrote a lot of code with it.  It felt enough like C to me.
At the time, I believe it was the fastest CPM C compiler you could get.
That was a big part of why I used it.

[TUHS] Early Internet work (Was: History of select(2))

[Noel Chiappa]

    > From: Tony Finch

This is getting a bit far afield from Unix, so my apologies to the list for
that. But to avoid dumping it in the Internet-History list abruptly, let me
answer here _briefly_ (believe it or not, the below _is_ brief).

    > AIUI there were two major revisions to the IPv4 addressing architecture:

Not quite (see below). First, one needs to understand that there are two
different timelines for changes to addressing: in the hosts, and in the
routers (called 'gateways' originally). To start with, they were tied
together, but as of RFC-1122, they were formally separated: hosts no longer
fully understood the syntax/semantics of addresses, just (mostly) treated them
as opaque 32-bit quantities.

    > subnetting (RFC 917, October 1994 ... RFC 950, August 1985), and
    > classless routing (RFC 1519, September 1993)

Originally, network numbers were 8 bits, and the 'rest' (local) part was 24.
Mapping from IP addresses to physical network addresses was some with direct
mapping - ARP did not exist - the actual local address (e.g. IMP/Port) was
contained in the 'rest' field - each network had a document which specified
the mapping. (Which is part of the interoperability issue with old
implementations.)

As some point early on, it was realized that 8 bits of network number were not
enough, and the awful A/B/C kludge was added (it was dropped on the community,
not discussed before-hand). Subnetting was indeed the next change.  Then the
host/router split happened.

Classless routing (which effectively extended addesses, for path-computation
purposes, to 32+N bits - since you couldn't look at a 32-bit IP address and
immediately tell which was the 'network' part any more, you _had_ to have the
mask as well, to tell you how many bits of any given address were the network
number) was more of a process than a single change - the inter-AS routing
(BGP) had to change, but so did IGP's (OSPF, IS-IS), etc, etc.

    > originally called supernetting (RFC 1338, June 1992).

There was this effort called ROAD which produced RFC-1338 and 1519, and IIRC
there was an intermediate, involving blocks of network numbers (1338), and
that slowly evolved into arbitrary blocks (1519).

One should also note that the term "super-netting" comes from a proposal by
Carl-Hubert ("Roki") Rokitansky which did not, alas, make it to RFC. (His
purpose was different, but it used the same mechanism.) Alas, the authors of
1338/1519 failed to properly acknowledge his earlier work.

	  Noel

[TUHS] Early Internet work (Was: History of select(2))

[Johnny Billquist]

On 2017-01-16 03:00, jnc at mercury.lcs.mit.edu (Noel Chiappa) wrote:
>     > From: Johnny Billquist
>
>     > Like I pointed out, RFC760 lacks ICMP.
>
> So? TCP will work without ICMP.

True. However, IP and UDP will have issues.

>     > Which also makes one question how anyone would have known about IPv4 in
>     > 1978.
>
> Well, I can assure you that _I_ knew about it in 1978! (The decision on the v4
> packet formats was taken in the 5th floor conference room at 545 Tech Sq,
> about 10 doors down from my office!)
>
> But everyone working on TCP/IP heard about Version 4 shortly after the June,
> 1978 meeting.

Over a year before any documents said anything about it. This is where I 
have problems. :-)

>     > Also, first definition of TCP shows up in RFC 761
>
> If you're speaking of TCPv4 (one needs to be precise - there were also of
> course TCP's 1, 2, 2.5 and 3, going back to 1974), please see IEN-44. (Ignore
> IEN's -40 and -41; those were proposals for v4 that got left by the wayside.)

That is a very good point. I've been talking v4 all the time (both for 
IP and TCP). Like I said, I'm sure people were doing networking 
protocols and stuff earlier, but it wasn't the TCP/IP we know and talk 
about today, and you just reaffirmed this.

And yes, the TCP/IP we know today did not come out of a blue sky. Of 
course it is based on earlier work. (Just do you don't have to go on 
about that again.)

>     > So yes, I still have problems with claims that they had it all running
>     > in 1978.
>
> I never said we had it "all" running in 1978 - and I explicitly referenced
> areas (congestion, addressing/routing) we were still working on over 10 years
> later.
>
> But there were working implementations (as in, they could exchange data with
> other implementations) of TCP/IPv4 by January 1979 - see IEN 77.

But not TCP4 then. And thus, not interoperable with an implementation 
today, and interoperable in general being a rather floating and moving 
target, as you had several imvompatible TCP versions, using different 
protocol numbers, and several incompatible IP versions.

> (I'll never forget that weekend - we were in at ISI on Saturday, when it was
> normally closed, and IIRC we couldn't figure out how to turn the hallway
> lights on, so people were going from office to office in the gloom...)

Fun times, I bet.

	Johnny

-- 
Johnny Billquist                  || "I'm on a bus
                                   ||  on a psychedelic trip
email: bqt at softjar.se             ||  Reading murder books
pdp is alive!                     ||  tryin' to stay hip" - B. Idol

[TUHS] Early Internet work (Was: History of select(2))

[Noel Chiappa]

    > (Check the minutes in the IEN's, that's probably the best source of data
    > on progress of the various implementations.)

Another place to look is Internet Monthly Reports and TCP-IP Digests (oh, I
see you've seen those, I see a reference to one).

I have this distinct memory of Dave Clark mentioning the Liza Martin TCP/IP
for Unix in one of the meeting report publihed as IENs, but a quick look
didn't find it.

    Noel

[TUHS] Early Internet work (Was: History of select(2))

[Paul Ruizendaal]

It may be mentioned in this report:
http://web.mit.edu/Saltzer/www/publications/rfc/csr-rfc-228.pdf

If so, it would seem to be late 1981, early 1982.

Would you know if any of its source code survived?

Paul

On 16 Jan 2017, at 2:47 , Noel Chiappa wrote:

> 
>> (Check the minutes in the IEN's, that's probably the best source of data
>> on progress of the various implementations.)
> 
> Another place to look is Internet Monthly Reports and TCP-IP Digests (oh, I
> see you've seen those, I see a reference to one).
> 
> I have this distinct memory of Dave Clark mentioning the Liza Martin TCP/IP
> for Unix in one of the meeting report publihed as IENs, but a quick look
> didn't find it.
> 
>    Noel

[TUHS] Early Internet work (Was: History of select(2))

[Noel Chiappa]

    > From: Paul Ruizendaal

    > I guess by April 1981 (RFC777) we reach a point where things are
    > specified to a level where implementations would interoperate with
    > today's implementations.

Yes and no. Earlier boxes would interoperate, _if addresses on each end were
chosen properly_. Modern handling of addresses on hosts (for the 'is this
destination on my physical network' step of the packet-sending algorithm) did
not come in until RFC-1122 (October 1989); prior to that, lots of host code
probably tried to figure out if the destination was class A, B or C, etc, etc.

Also, until RFC-826 (ARP, November 1982) pretty much all the network
interfaces (and thus the code to turn the 'destination IP address' into an
attached physical network address, for the first hop) were things like ARPANet
that no longer exist, so you could't _actually_ fire up one of them unless you
do something like the 'ARPANet emulation' that the various PDP-10 simulators
use to allow old OS's running on them to talk to the current Internet.


    > only if one accepts IEN54/55 as 'TCP/IP'

What are they, if not TCP/IP?

Not the modern variant, of course, but then again, nothing before the early
90's is truly 'modern TCP/IP'.

    > IEN98 mentions a TCP3 stack done for Network Unix ...  in 1978 by DTI /
    > Gary Grossman.

I read this, BITD, but don't recall much about it. I was not impressed by the
coding style.

    > at the same time it also uses old-style assignments ('=+' instead of
    > '+='). Could this be "typesetter C"?

I don't know. IIRC, that compiler supported both styles. It had to have been a
later compiler than the one that came with V6, that didn't support longs.  But
I don't recall any bug with long support in the typetter C compiler we had at
MIT.

    > From the above I would support the moniker "first TCP/IP in C on Unix"

No. That clearly belongs to the DTI one. (The difference between V3 and V4,
while significant, aren't enough to make the DTI not 'TCP/IP in C for Unix'.)

If you want to say 'first V4TCP/IP in C for Unix', maybe; I'd have to look for
dates on the one done at MIT for V6, that may be earlier, but I don't think
so. (Check the minutes in the IEN's, that's probably the best source of data
on progress of the various implementations.)


    > One thing that I'm unclear about is why all this Arpanet work was not
    > filtering more into the versions of Unix done at Bell Labs.   

Here's my _guess_ - ask someone from Bell for a sure answer.

You're using 20/20 hindsight. At that point in time, it was not at all obvious
that TCP/IP was going to take over the world.

There were a couple of alternatives for moving data around that Bell used -
Datakit, and UUCP - and they worked pretty well, and there was no reason to
pick up on this TCP/IP thing.

I suspect that it wasn't until LAN's became popular than TCP/IP looked like a
good thing to have - it fits very well with the capabilities most LANs had (in
term of the service provided to things attached to them). Datakit was its own
thing, and for UUCP you'd have to provide a reliable stream, and TCP/IP 'just
did that'.

     Noel

[TUHS] Early Internet work (Was: History of select(2))

[Paul Ruizendaal]

On 16 Jan 2017, at 2:01 , Noel Chiappa wrote:

> 
>> From: Paul Ruizendaal
> 
>> I guess by April 1981 (RFC777) we reach a point where things are
>> specified to a level where implementations would interoperate with
>> today's implementations.
> 
> Yes and no. Earlier boxes would interoperate, _if addresses on each end were
> chosen properly_. Modern handling of addresses on hosts (for the 'is this
> destination on my physical network' step of the packet-sending algorithm) did
> not come in until RFC-1122 (October 1989); prior to that, lots of host code
> probably tried to figure out if the destination was class A, B or C, etc, etc.

This is true of the Gurwitz implementation. The Wingfield implementation still uses the older form, where the first 8 bits signify the network and the remaining 24 bits the host address on that network. In terms of routing my view would be to keep it simple: traffic is either local or destined for the single interface / gateway (see below). Interop hence is just looking at TCP.

> 
> Also, until RFC-826 (ARP, November 1982) pretty much all the network
> interfaces (and thus the code to turn the 'destination IP address' into an
> attached physical network address, for the first hop) were things like ARPANet
> that no longer exist, so you could't _actually_ fire up one of them unless you
> do something like the 'ARPANet emulation' that the various PDP-10 simulators
> use to allow old OS's running on them to talk to the current Internet.

Yes: all these old implementations have an IMP interface driver at their lowest level. What I'm doing for testing is replacing that by a SLIP driver so that I can hook up to today's network and see if it works.

Paul

[TUHS] Early Internet work (Was: History of select(2))

[Tony Finch]

Noel Chiappa <jnc at mercury.lcs.mit.edu> wrote:
>
> Modern handling of addresses on hosts (for the 'is this destination on
> my physical network' step of the packet-sending algorithm) did not come
> in until RFC-1122 (October 1989); prior to that, lots of host code
> probably tried to figure out if the destination was class A, B or C,
> etc, etc.

AIUI there were two major revisions to the IPv4 addressing architecture:
subnetting (RFC 917, October 1994 ... RFC 950, August 1985), and classless
routing (RFC 1519, September 1993) which was originally called
supernetting (RFC 1338, June 1992). RFC 1122 consolidated all the
implementation requirements in one place, and said RFC 950 subnetting was
mandatory.

Tony.
-- 
f.anthony.n.finch  <dot at dotat.at>  http://dotat.at/  -  I xn--zr8h punycode
Bailey, Fair Isle, Faeroes: Southwest 5 or 6, occasionally 7 later in Bailey,
becoming variable 4 at times. Moderate or rough in Fair Isle, otherwise rough
or very rough. Rain or drizzle, fog patches. Moderate or good, occasionally
very poor.

[TUHS] Early Internet work (Was: History of select(2))

[Noel Chiappa]

    > From: Johnny Billquist

    > Like I pointed out, RFC760 lacks ICMP.

So? TCP will work without ICMP.

    > Which also makes one question how anyone would have known about IPv4 in
    > 1978.

Well, I can assure you that _I_ knew about it in 1978! (The decision on the v4
packet formats was taken in the 5th floor conference room at 545 Tech Sq,
about 10 doors down from my office!)

But everyone working on TCP/IP heard about Version 4 shortly after the June,
1978 meeting.

    > Also, first definition of TCP shows up in RFC 761

If you're speaking of TCPv4 (one needs to be precise - there were also of
course TCP's 1, 2, 2.5 and 3, going back to 1974), please see IEN-44. (Ignore
IEN's -40 and -41; those were proposals for v4 that got left by the wayside.)

    > So yes, I still have problems with claims that they had it all running
    > in 1978.

I never said we had it "all" running in 1978 - and I explicitly referenced
areas (congestion, addressing/routing) we were still working on over 10 years
later.

But there were working implementations (as in, they could exchange data with
other implementations) of TCP/IPv4 by January 1979 - see IEN 77.

(I'll never forget that weekend - we were in at ISI on Saturday, when it was
normally closed, and IIRC we couldn't figure out how to turn the hallway
lights on, so people were going from office to office in the gloom...)

    Noel

[TUHS] Early Internet work (Was: History of select(2))

[Noel Chiappa]

    > From: Paul Ruizendaal

    >> On 12 Jan 2017, at 4:54 , Clem Cole wrote:

    >> The specifications for what would become IP and TCP were kicking around
    >> ... in the late 1970s.

The whole works actually started considerably earlier than that; the roots go
back to 1972, with the formation of the International Packet Network Working
Group - although that group went defunct before TCP/IP itself was developed
under DARPA's lead.

I don't recall the early history well, in detail - there's a long draft
article by Ronda Hauben which goes into it in detail, and there's also "INWG
and the Conception of the Internet: An Eyewitness Account" by Alexander
McKenzie which covers it too.

By 1977 the DARPA-led effort had produced several working prototype
implementations, and TCP/IP (originally there was only TCP, without a separate
data packet carriage layer) were up to version 3.

    > My understanding is that all RFC's and IEN's were available to all legit
    > users of the Arpanet.

Yes and no. The earliest distribution mechanism (for the initial NCP/ARPANet
work) was hardcopy (you can't distribute things over the 'net before you have
it working :-), and in fact until a recent effort to put them all online, not
all RFC's were available in machine-readable form. (I think some IEN's still
aren't.) So for many of them, if you wanted a copy, you had to have someone at
ISI make a photocopy (although I think they stocked them early on) and
physically mail it to you!

    > Apparently nobody had the idea to put all RFC's in a directory and give
    > FTP access to it.

I honestly don't recall when that happened; it does seem obvious in
retrospect! Most of us were creating document in online text systems, and it
would have been trivial to make them available in machine-readable form. Old
habits die hard, I guess... :-)

    > I think I should put a question out about this, over on the internet
    > history mailing list.

Yes, good idea.


    > As an aside: IMHO, conceptually the difference between NCP and TCP
    > wasn't all that big.

Depends. Yes, the service provided to the _clients_ was very similar (which
can be seen in how similar the NCP and TCP versions of thing like TELNET, FTP,
etc were), but internally, they are very different.

    > In my current understanding the big difference that was NCP assumes
    > in-order, reliable delivery of packets ... and that TCP allows for
    > unreliable links.

Yes, that's pretty accurate (but it does mean that there are _a lot_ of
differences internally - re-transmissions, etc). One other important
difference is that there's no flow control in the underlying network
(something that took years to understand and deal with properly).

    > yes, these concepts were kicking around for over a decade in academia
    > before BSD.

TCP/IP was the product of a large, well-organized, DARPA-funded and -led
effort which involved industry, academic and government players (the first
two, for the most part, DARPA-funded). So I wouldn't really call it an
'academic' project.

	Noel

[TUHS] History of select(2)

[Warren Toomey]

Also, I came across this history of select(2) a while back:

https://idea.popcount.org/2016-11-01-a-brief-history-of-select2/

Cheers, Warren

[TUHS] History of select(2)

[Paul Ruizendaal]

On 9 Jan 2017, at 3:35 , Warren Toomey wrote:

> Also, I came across this history of select(2) a while back:
> 
> https://idea.popcount.org/2016-11-01-a-brief-history-of-select2/
> 
> Cheers, Warren

That is an interesting blog post, but I think it is a bit short on the history of things before 4.2BSD. Below my current understanding of what came before select().

In March 1975 the first networked Unix was created at the University of Illinois, initially based on 5th edition, but soon ported to 6th edition. It is described in RFC681 and a paper by Greg Chesson. Note that UoI was the very first Unix licensee. Its primary authors were Steve Holmgren, Steve Bunch and Gary Grossman. Greg Chesson was also involved. Grossman had already done two earlier Arpanet implementations (the ANTS and ANTS II systems) on bare metal and had a deep understanding of what a good implementation needed.

Their implementation was compact (about a thousand lines added to the kernel, and another thousand in the connection daemon) and - I'm my opinion at least - conceptually well integrated into the existing file API. It became the leading Unix Arpanet implementation with wide use from 1975 to 1981. Two things stand out: (i) no accept(); and (ii) no select(). The original authors are still with us, with the exception of Greg, and I asked for their input as well.

(i) no accept()

Listening sockets worked a bit different from today. If one opened a listening socket it would not return a descriptor but block instead; when a connection was made it would return with the listening socket now bound to the new connection. Server applications would open a listening socket and do a double fork for the client connection (i.e. getting process 1 as its parent); the main process would loop around and open a new listening socket (this can all be verified in surviving application sources). According to Steve Holmgren this was not perceived as a big problem at the time. Network speeds were still so low that the brief gap in listening did not matter much, and the double fork was just a few lines of code.

This changed when the CSRG team moved from a long-haul, Arpanet, 56Kb/s context to a local, Ethernet, 3Mb/s context and Sam Leffler came up with the concept of accept(). In 4.1a BSD and 2.9BSD the queue of pending connections was fixed (possibly 1, I have to check). In 4.1c BSD listen() was introduced; before then whether a socket was active or listening was a flag to opening the socket. The second parameter to listen() specified the maximum number of pending connections [as an aside, note that I'm using 'socket' in the BSD sense; the term socket changed meaning several times between 1973 and 1983].

(ii) no select()

This was the real pain (Holmgren reconfirmed that). This is what Dennis must have referred to in his retrospect paper. Various solutions were thought of, but in Network Unix the model remained using separate processes for simultaneous reading and writing. Progress in this area came from two other places involved in Unix and Arpanet: Rand and BBN.

In 1977 Rand was taking on this problem (see http://www.dtic.mil/dtic/tr/fulltext/u2/a044200.pdf and http://www.dtic.mil/dtic/tr/fulltext/u2/a044201.pdf). They considered a  solution with a new system call 'empty()' that would tell if there was any data available on a file descriptor, a crude form of non-blocking I/O if you like. As this would consume precious CPU cycles it proved inadequate. Instead they came up with "ports". A port was a (possibly named) pipe with multiple writers and a single reader, and it was created with a 'port()' system call. The reader would see each write preceded by a header block identifying the reader. The implementation (see second PDF) was simple, apparently only taking 200 words of kernel code. Rand ports are a simplistic version of the 'mpx' facility done by Greg Chesson at Bell Labs (in 1978?). I am not sure whether this was independent invention or that Greg was aware of Rand ports. Unfortunately we cannot ask him anymore.

Later in 1977, over at BBN, Jack Haverty was doing an experimental TCP/IP stack for Unix (this was TCP 2.5, not TCP 4). He had a working stack written in PDP11 assembler for a different OS and was making this run on Unix. He was using Rand ports to connect clients to the network stack, but still lacked the required primitives to make this work properly. So he came up with the await() system call, a direct precursor to select(). It is documented in BBN report 3911 (http://bit.ly/2iU1TNK), including man pages. With the awtenb() and awtdis() one would manage the monitored descriptors (like the bit vectors going into a select), and await() would then wait for an event or time out.

Related to this was the capac() system call, to get the 'capacity' of a descriptor. This returns the amount of data that can safely be written to or read from a descriptor. I suppose it is an improved version of empty(). There is no equivalent of that in the later BSD sockets, perhaps because non-blocking I/O in the current sense was about to arrive. With port(), await() and capac() it becomes possible to write single threaded network programs.

An example may be found here, the first TCP/IP (version 4) stack in C for Unix, from early 1979: http://digital2.library.ucla.edu/viewItem.do?ark=21198/zz002gvzqg (scroll down past IMP stuff). It's documented in IEN98 (https://www.rfc-editor.org/ien/ien98.txt). I'm currently retyping this source so that it can be better studied.

The await() call is not in the TCP/IP code done for 4.1 BSD by BBN. I'm puzzled by this as it is evidently useful and Jack Haverty and Rob Gurwitz worked in the same corridor at BBN at the time. In 4.1a the select() call appears and it seems to be an improved version of await(), with the need for awtenb() en awtdis() replaced by the use of bit vectors. I am not sure if Bill Joy was aware of await() or whether it was independent invention. Here we can ask, but I have no contact details.

Hope the above is of interest. I'm still learning new things about these topics every day, so please advise if my above understanding is wrong.


As a side note, I am still looking for:

- surviving copies of UoI "Network Unix" (I'm currently no further than papers and bits of source that lingered in other code bases)

- surviving copies of the 4.1a BSD distribution tape (Kirk McKusick's tape was damaged)

- surviving source of the kernel code of port(), await() and capac(); (could possibly be recreated from documentation)

Any and all help very much appreciated.

Paul

[TUHS] History of select(2)

[Clem Cole]

Paul -- this is a great stuff and fills in some pieces of my memories.


FYI:   CMU had the the Rand Ports code including empty() in some
implementations of kernels in use in the mid 1970s.   The later UNIX cu(1)
program did not exist, and a couple of people hacked together a program
called "connect" which was used to connect between serial ports for
downloading PPT images to microprocessors such as a KIM-1 and communicating
with it.   I say a "couple of people," because I do not remember who wrote
the original version.   It could have been a number of us.  I know I hacked
on it a good bit, as did others over the course of time (and we did not use
source control in those days).   The sources were there for all to work
with and we all added stuff as we though of things that would help us out.


Anyway, besides microprocessor support, we also used connect(1) to
"front-end" the serial lines from the PDP-10's in CS and were doing some
rudimentary networking stuff over parallel DR-11C's, in the pre-TCP/IP
world [a discussion a couple of us have had separately - more in a minute].
  But the point is that the dates line up with my being there and what we
were doing at the time.  One version of connect(1) used the empty() system
call and Rand ports to do its thing.    I also remember reading the
connect(1) code as an early education in network technology and concepts.
We would later take hunks of into a TCP stack [3-4 years later Phil Karn
would write the infamous KA9Q TCP stack and a few years later, Stan Smith
and I would write the VMS TCP etc in BLISS but it model on some C network
stuff from CMU... -- amazing circle].

Anyway, around the same time (either sept 76 or 77 would be my guess) the
late Ted Kowalski came to CMU for his OYOC time (I've forgotten which it
was to be honest) and brought a pre-UNIX/TS system with him - TS begat V7.
Ted brought that system up on the 11/34 in EE and I remember connect(1) was
the most important program that immediately broke.   I also remember a
large argument between Ted and one of the other hackers (I've forgotten
whom), Ted saying we did not need it, it was wasteful, etc... and not in
the official editions.   I remember he re-implemented the connect(1)
program one night with multiple process and EE systems were from them on
based on Ted's system [although, I would later get to know Chesson and Greg
would give me the mpx() code a year or two later for some networking stuff
I would work on but that is a different story].

The point is that while I have no memory of capac(), but I can confirm that
I definitely programmed with the empty() system call and Rand ports on a v6
based kernel in the mid-1970s and that it was definitely at places besides
Rand themselves.

Another thing I want thank you  for it confirming something I have been
saying for  few years and some people have had a hard time believing.   The
specifications for what would become IP and TCP were kicking around the
ARPAnet in the late 1970s.   We definitely had them at CMU and that's where
I first was introduced to them, long before the planned cut over in the
early 1980s.   I probably was not aware of the global politics involved
outside of the ARPA community because I certainly thought at the time IP
was we were headed and it was what we were thinking about and considering
how to implement.

Anyway - thanks again for a great piece of hunt up some good stuff.

Clem

On Mon, Jan 9, 2017 at 2:36 AM, Paul Ruizendaal <pnr at planet.nl> wrote:

> On 9 Jan 2017, at 3:35 , Warren Toomey wrote:
>
> > Also, I came across this history of select(2) a while back:
> >
> > https://idea.popcount.org/2016-11-01-a-brief-history-of-select2/
> >
> > Cheers, Warren
>
> That is an interesting blog post, but I think it is a bit short on the
> history of things before 4.2BSD. Below my current understanding of what
> came before select().
>
> In March 1975 the first networked Unix was created at the University of
> Illinois, initially based on 5th edition, but soon ported to 6th edition.
> It is described in RFC681 and a paper by Greg Chesson. Note that UoI was
> the very first Unix licensee. Its primary authors were Steve Holmgren,
> Steve Bunch and Gary Grossman. Greg Chesson was also involved. Grossman had
> already done two earlier Arpanet implementations (the ANTS and ANTS II
> systems) on bare metal and had a deep understanding of what a good
> implementation needed.
>
> Their implementation was compact (about a thousand lines added to the
> kernel, and another thousand in the connection daemon) and - I'm my opinion
> at least - conceptually well integrated into the existing file API. It
> became the leading Unix Arpanet implementation with wide use from 1975 to
> 1981. Two things stand out: (i) no accept(); and (ii) no select(). The
> original authors are still with us, with the exception of Greg, and I asked
> for their input as well.
>
> (i) no accept()
>
> Listening sockets worked a bit different from today. If one opened a
> listening socket it would not return a descriptor but block instead; when a
> connection was made it would return with the listening socket now bound to
> the new connection. Server applications would open a listening socket and
> do a double fork for the client connection (i.e. getting process 1 as its
> parent); the main process would loop around and open a new listening socket
> (this can all be verified in surviving application sources). According to
> Steve Holmgren this was not perceived as a big problem at the time. Network
> speeds were still so low that the brief gap in listening did not matter
> much, and the double fork was just a few lines of code.
>
> This changed when the CSRG team moved from a long-haul, Arpanet, 56Kb/s
> context to a local, Ethernet, 3Mb/s context and Sam Leffler came up with
> the concept of accept(). In 4.1a BSD and 2.9BSD the queue of pending
> connections was fixed (possibly 1, I have to check). In 4.1c BSD listen()
> was introduced; before then whether a socket was active or listening was a
> flag to opening the socket. The second parameter to listen() specified the
> maximum number of pending connections [as an aside, note that I'm using
> 'socket' in the BSD sense; the term socket changed meaning several times
> between 1973 and 1983].
>
> (ii) no select()
>
> This was the real pain (Holmgren reconfirmed that). This is what Dennis
> must have referred to in his retrospect paper. Various solutions were
> thought of, but in Network Unix the model remained using separate processes
> for simultaneous reading and writing. Progress in this area came from two
> other places involved in Unix and Arpanet: Rand and BBN.
>
> In 1977 Rand was taking on this problem (see
> http://www.dtic.mil/dtic/tr/fulltext/u2/a044200.pdf and
> http://www.dtic.mil/dtic/tr/fulltext/u2/a044201.pdf). They considered a
> solution with a new system call 'empty()' that would tell if there was any
> data available on a file descriptor, a crude form of non-blocking I/O if
> you like. As this would consume precious CPU cycles it proved inadequate.
> Instead they came up with "ports". A port was a (possibly named) pipe with
> multiple writers and a single reader, and it was created with a 'port()'
> system call. The reader would see each write preceded by a header block
> identifying the reader. The implementation (see second PDF) was simple,
> apparently only taking 200 words of kernel code. Rand ports are a
> simplistic version of the 'mpx' facility done by Greg Chesson at Bell Labs
> (in 1978?). I am not sure whether this was independent invention or that
> Greg was aware of Rand ports. Unfortunately we cannot ask him anymore.
>
> Later in 1977, over at BBN, Jack Haverty was doing an experimental TCP/IP
> stack for Unix (this was TCP 2.5, not TCP 4). He had a working stack
> written in PDP11 assembler for a different OS and was making this run on
> Unix. He was using Rand ports to connect clients to the network stack, but
> still lacked the required primitives to make this work properly. So he came
> up with the await() system call, a direct precursor to select(). It is
> documented in BBN report 3911 (http://bit.ly/2iU1TNK), including man
> pages. With the awtenb() and awtdis() one would manage the monitored
> descriptors (like the bit vectors going into a select), and await() would
> then wait for an event or time out.
>
> Related to this was the capac() system call, to get the 'capacity' of a
> descriptor. This returns the amount of data that can safely be written to
> or read from a descriptor. I suppose it is an improved version of empty().
> There is no equivalent of that in the later BSD sockets, perhaps because
> non-blocking I/O in the current sense was about to arrive. With port(),
> await() and capac() it becomes possible to write single threaded network
> programs.
>
> An example may be found here, the first TCP/IP (version 4) stack in C for
> Unix, from early 1979: http://digital2.library.ucla.e
> du/viewItem.do?ark=21198/zz002gvzqg (scroll down past IMP stuff). It's
> documented in IEN98 (https://www.rfc-editor.org/ien/ien98.txt). I'm
> currently retyping this source so that it can be better studied.
>
> The await() call is not in the TCP/IP code done for 4.1 BSD by BBN. I'm
> puzzled by this as it is evidently useful and Jack Haverty and Rob Gurwitz
> worked in the same corridor at BBN at the time. In 4.1a the select() call
> appears and it seems to be an improved version of await(), with the need
> for awtenb() en awtdis() replaced by the use of bit vectors. I am not sure
> if Bill Joy was aware of await() or whether it was independent invention.
> Here we can ask, but I have no contact details.
>
> Hope the above is of interest. I'm still learning new things about these
> topics every day, so please advise if my above understanding is wrong.
>
>
> As a side note, I am still looking for:
>
> - surviving copies of UoI "Network Unix" (I'm currently no further than
> papers and bits of source that lingered in other code bases)
>
> - surviving copies of the 4.1a BSD distribution tape (Kirk McKusick's tape
> was damaged)
>
> - surviving source of the kernel code of port(), await() and capac();
> (could possibly be recreated from documentation)
>
> Any and all help very much appreciated.
>
> Paul
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170111/5e07d4e9/attachment.html>

[TUHS] History of select(2)

[Paul Ruizendaal]

On 12 Jan 2017, at 4:54 , Clem Cole wrote:

> Paul -- this is a great stuff and fills in some pieces of my memories.

Thanks!

> The point is that while I have no memory of capac(), but I can confirm that I definitely programmed with the empty() system call and Rand ports on a v6 based kernel in the mid-1970s and that it was definitely at places besides Rand themselves.

Thank you for confirming that. If anybody knows of surviving source for these extensions I'd love to hear about it. Although the description in the implementation report is clear enough to recreate it (it would seem to be one file similar to pipe.c and a pseudo device driver similar in size to mem.c), original code is better. It is also possible that the code in pipe.c was modified to drive both pipes and ports -- there would have been a lot of similarity between the two, and kernel space was at a premium.

> [...] confirming something I have been saying for few years and some people have had a hard time believing. The specifications for what would become IP and TCP were kicking around the ARPAnet in the late 1970s.

My understanding is that all RFC's and IEN's were available to all legit users of the Arpanet. By 1979 there were 90 nodes (IMP's) and about 200 hosts connected. I don't get the impression that stuff was always easy to find, with Postel making a few posts about putting together "protocol information binders". Apparently nobody had the idea to put all RFC's in a directory and give FTP access to it.

I am not sure how available this stuff was outside the Arpanet community. I think I should put a question out about this, over on the internet history mailing list.

As an aside: IMHO, conceptually the difference between NCP and TCP wasn't all that big. In my current understanding the big difference was that NCP assumes in-order, reliable delivery of packets (as was the case between IMP's) and that TCP allows for unreliable links. Otherwise, the connection build-up and tear-down and the flow control were similar. See for instance RFC54 and RFC55 from 1970. My point is: yes, these concepts were kicking around for over a decade in academia before BSD.

Paul

[TUHS] Early Internet work (Was: History of select(2))

[Paul Ruizendaal]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 7376 bytes --]


On 14 Jan 2017, at 17:41 , Michael Kjörling wrote:

> On 13 Jan 2017 10:13 +0100, from pnr at planet.nl (Paul Ruizendaal):
>> over on the internet history mailing list.
> 
> Interesting. Care to give me a pointer toward it?


That mailing list is here:
http://mailman.postel.org/mailman/listinfo/internet-history


On 14 Jan 2017, at 23:02 , Johnny Billquist wrote:

> IMPs did not talk IP, just for the record.
Yes, this is true of course. The software of the IMP was resurrected from printouts some time ago:
http://walden-family.com/impcode/

> The problem is that all the RFCs are available, and they are later than this. The ARPAnet existed in 1979, but it was not using TCP/IP. If you look at the early drafts of TCP/IP, from around 1980-1981, you will also see that there are significant differences compared to the TCP/IP we know today. There was no ICMP, for example. Error handling and passing around looked different.

Once again: yes. When exactly was the TCP/IP specification completed? That is an issue where reasonable people can hold different opinions. What software first implemented this on Unix? Here too reasonable people can hold different opinions. Below my take on this, based on my current understanding (and I keep repeating that I'm learning new things about this stuff almost every day and please advise if I'm missing things).

Development of TCP/IP

The specification that became TCP/IP apparently finds its roots in 1974, and it is gradually developed over the next years with several trial implementations. By March 1977 we get to TCP2 and more trials.  Next it would seem that there was a flurry of activity from January to August 1978, resulting in specifications for TCP4 (IEN54 and IEN55). Then, up to March more implementations follow, as documented in IEN98. With those implementations tested, also for interoperability, more changes to the protocol and implementations follow and I guess by April 1981 (RFC777) we reach a point where things are specified to a level where implementations would interoperate with today's implementations. This is not where it stops, 'modern' congestion control only goes back to the late 80's (see for instance Craig Partridge http://www.netarch2009.net/slides/Netarch09_Craig_Partridge.pdf, it is an interesting read).

Early Unix code bases

(1) The Mathis/Haverty stack

In 1977 Jim Mathis at SRI writes a "TCP2.5" stack in PDP11 assembler, running on the MOS operating system (for the LSI11). In 1978 Haverty is assigned to take this stack and make it run on V6 Unix. He builds a kernel with ports, await and capac to achieve this. It is a mixed success (see http://mailman.postel.org/pipermail/internet-history/2016-October/004073.html), but he maintains it through 1978 and 1979 as a skunkworks project and the code eventually supports TCP4 (as defined in IEN54/55). The source has survived in Jack Haverty's basement as a printout, but it is not online. As far as I know know, this the first TCP/IP on Unix (a tie with the Wingfield implementation, and only if one accepts IEN54/55 as 'TCP/IP').

(2) The Grossman (DTI) stack

IEN98 mentions a TCP3 stack done for Network Unix (by then called ENFE/INFE) in 1978 by DTI / Gary Grossman. I don't currently have information about that implementation. As at March 1979 it did not appear to support TCP4.

(3) The Wingfield/Cain stack

In 1978 BBN / Michael Wingfield, was commissioned by DCEC / Ed Cain to write a TCP4 tack in C for Unix V6. As it stood in March 1979 this code supported IEN54/55 with the AUTODIN II security extensions that heralded back to 1975. It is a partial implementation: it does not support IP fragmentation and it has a simplistic approach to incoming out-of-order packets (drop everything until the right one arrives). However, it worked and kept being maintained by Ed Cain, who by October 1981 has added support for ICMP and GGN (https://www.rfc-editor.org/rfc/museum/tcp-ip-digest/tcp-ip-digest.v1n1.1). He is still supporting it as late as 1985 (https://www.rfc-editor.org/rfc/museum/tcp-ip-implementations.txt.1)

As far as I know know, only the March 1979 code survives. I'm currently retyping it, about halfway through. I'm not sure what compiler it was written for: it uses longs, but apparently this is still somewhat broken (with comments in the source amounting to 'this WTF necessary to work around compiler bugs'); at the same time it also uses old-style assignments ('=+' instead of '+='). Could this be "typesetter C"? The code feels like it might be based on earlier work, for instance the BBN BCPL implementation for TENEX a few years earlier, but that is pure speculation. It could also be that Wingfield was new to C and bringing habits from other languages with him. Once all done, I'll ask Michael about it.

I'm on thin ice here, but my current guess would be that the 5,000 line code base would need some 500 lines of new code to make it interoperable with today's implementations. From the above I would support the moniker "first TCP/IP in C on Unix" as claimed by UCLA, either for the March 1979 version if one takes the view that '90% modern' is good enough, or for the October 1981 version if support for RFC777 is the benchmark. In the latter view it beats the Gurwitz stack by about a month or two. However, it is not a full implementation of the specifications.

(4) The Gurwitz stack

Last in the list of candidates is the Rob Gurwitz stack for BSD4.1 (see IEN168), started in January 1981. It is a full implementation of the protocols that looks like it was done from scratch (as confirmed by Gurwitz and Haverty) and consolidates the earlier learnings. In my opinion, it is the first implementation where the source code has a distinct 'early unix smell' (please excuse the phrase), arguably more so that the later Joy code. The first iterations of the code don't support (the as yet non-existent) RFC777. By November 1981 there is a beta distribution tape that does, and this code looks to interoperate with modern TCP/IP implementations as-is. If the benchmark is a full implementation interoperating with today's code, the first TCP/IP on Unix would I think be the Gurwitz stack. Possibly it is the first TCP/IP in that definition on any OS.

Note that this is also where TCP/IP and Network Unix join back up [but this view might change as I learn more about the Grossman / DTI version]: the Gurwitz code uses an API closely based on that of UoI's Network Unix and the provided user land programs (Telnet, FTP, MTP, etc.) appear ports from that code base (or perhaps from a BBN development of the UoI work).

===

In any case, I think it is fair to say that TCP/IP as we know it today did not drop from the sky in 1981. There was a vast amount of prior work done in the second half of the 70's on a variety of hardware and operating systems, experience that all feeded into the well known RFC's.

One thing that I'm unclear about is why all this Arpanet work was not filtering more into the versions of Unix done at Bell Labs. The folks involved were certainly aware of each other and the work that was going on. With universities the cost of 'always on' long distance lines may have been too great, but within Bell Labs that would have been less of an issue and there is a clear link with the core business of the Bell System.

Would anybody have some background on that?

Paul