From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.2 Received: (qmail 29880 invoked from network); 31 Mar 2020 02:45:00 -0000 Received-SPF: pass (minnie.tuhs.org: domain of minnie.tuhs.org designates 45.79.103.53 as permitted sender) receiver=inbox.vuxu.org; client-ip=45.79.103.53 envelope-from= Received: from minnie.tuhs.org (45.79.103.53) by inbox.vuxu.org with UTF8ESMTPZ; 31 Mar 2020 02:45:00 -0000 Received: by minnie.tuhs.org (Postfix, from userid 112) id 7AF439D712; Tue, 31 Mar 2020 12:44:57 +1000 (AEST) Received: from minnie.tuhs.org (localhost [127.0.0.1]) by minnie.tuhs.org (Postfix) with ESMTP id BBF2F9CD93; Tue, 31 Mar 2020 12:44:14 +1000 (AEST) Authentication-Results: minnie.tuhs.org; dkim=pass (1024-bit key; unprotected) header.d=planet.nl header.i=@planet.nl header.b="Ybi3By5h"; dkim-atps=neutral Received: by minnie.tuhs.org (Postfix, from userid 112) id 6EB4D9CD93; Tue, 31 Mar 2020 12:44:11 +1000 (AEST) Received: from cpsmtpb-ews03.kpnxchange.com (cpsmtpb-ews03.kpnxchange.com [213.75.39.6]) by minnie.tuhs.org (Postfix) with ESMTP id DA5F19CD7C for ; Tue, 31 Mar 2020 12:44:09 +1000 (AEST) Received: from cpsps-ews08.kpnxchange.com ([10.94.84.175]) by cpsmtpb-ews03.kpnxchange.com with Microsoft SMTPSVC(8.5.9600.16384); Tue, 31 Mar 2020 04:44:06 +0200 X-Brand: 7abm2Q== X-KPN-SpamVerdict: e1=0;e2=0;e3=0;e4=(e1=10;e3=10;e2=11;e4=10);EVW:Whi te;BM:NotScanned;FinalVerdict:Clean X-CMAE-Analysis: v=2.3 cv=WNfkokkR c=1 sm=1 tr=0 cx=a_idp_e a=dZ5u/0G9QtS9WKCcNUBnHQ==:117 a=soxbC+bCkqwFbqeW/W/r+Q==:17 a=jpOVt7BSZ2e4Z31A5e1TngXxSK0=:19 a=x1i13A_MHe4A:10 a=IkcTkHD0fZMA:10 a=SS2py6AdgQ4A:10 a=AoeSMSUmAAAA:8 a=yiCho3l6AAAA:20 a=jEIl9BPeAAAA:8 a=K6vUhs8TGlEsKMe8kLsA:9 a=GYmYGOREGpoH5n9m:21 a=5LmN6sSe14zISzk-:21 a=QEXdDO2ut3YA:10 a=lVEnoaUTKgsA:10 a=2UY7SMgi64q-0UtCmZ5F:22 a=UbykG2d9i8y_PxD0dgl3:22 X-CM-AcctID: kpn@feedback.cloudmark.com Received: from smtp.kpnmail.nl ([195.121.84.13]) by cpsps-ews08.kpnxchange.com over TLS secured channel with Microsoft SMTPSVC(8.5.9600.16384); Tue, 31 Mar 2020 04:44:06 +0200 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=planet.nl; s=planet01; h=to:message-id:date:from:subject:mime-version:content-type; bh=RI1syCo64lPGK3iPDv2GFriBkqTT0ySMO3XwX+Qes9A=; b=Ybi3By5hdgQOXItnhXaC20+4t9MRQWr10U06eq4HIpmtYg+rD2v60i8erLe2FCWzOBuOBHIGY4kRP +Y1BhWr65vyqwthC7SAwhNnS3A+Fjt6moi783oyM42+k5sr2vIzuOKBIXFL23R+LFADDhlrlodgQPh kLEXxIGKSat9PqG0= X-KPN-VerifiedSender: Yes X-CMASSUN: 33|knHh7YAUn+A7rPJ4VcnfejtDJ2i9gB/DZ/1qv7Cdj/4SZZXQvshRIwn2buHAD6f l+MOCe4O3cQLYNJKlXd2QWQ== X-Originating-IP: 80.101.112.122 Received: from mba2.fritz.box (sqlite.xs4all.nl [80.101.112.122]) by smtp.kpnmail.nl (Halon) with ESMTPSA id 7d6abb01-72f9-11ea-a1a6-005056998788; Tue, 31 Mar 2020 04:44:06 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) From: Paul Ruizendaal In-Reply-To: Date: Tue, 31 Mar 2020 04:44:06 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <67CAA7B3-798E-4F19-A06A-E41D95D22684@planet.nl> References: <20C3B8BE-E371-4694-8A34-EEC6A5461FAD@planet.nl> <202003291404.02TE4dAI022916@freefriends.org> <2298456D-A786-40C2-9C68-26C99E2002E1@planet.nl> <6AE2011D-32EF-4E1E-A231-9DFAC683FE48@planet.nl> To: TUHS main list X-Mailer: Apple Mail (2.3445.104.11) X-OriginalArrivalTime: 31 Mar 2020 02:44:06.0849 (UTC) FILETIME=[3F606B10:01D60706] X-RcptDomain: minnie.tuhs.org Subject: Re: [TUHS] 8th Edition timeline X-BeenThere: tuhs@minnie.tuhs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: The Unix Heritage Society mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: tuhs-bounces@minnie.tuhs.org Sender: "TUHS" > [...] the whole mbuf's stuff is something [...] I would prefer not to go into the BSD code history at this point in = time, as I=E2=80=99d like to stay focused on what was going on at Bell = Labs between 7th and 8th edition, as far as networking is concerned. Of = course it was all interconnecting, and the topic of mbuf's is a nice arc = to take us ultimately back too streams. Looking at network buffering in Unix, I think the story begins in 1973 = with Spider. In Spider, the network driver was a character mode device, = but used disk block buffering for storage. This worked well, because the = Spider interface sent/received data through DMA, in cells of 256 bytes = and at speeds comparable to contemporary disks: https://minnie.tuhs.org/cgi-bin/utree.pl?file=3DV4/nsys/dmr/tdir/tiu.c. In 1975 the UoI guys doing Arpa Unix had to interface to an IMP, which = was much slower than a disk but much faster than a terminal. They = concluded that neither clists not disk buffers were a good match and = Steve Bunch came up with =E2=80=99netbuf=E2=80=99s, which were 64 byte = blocks (carved out of a pool of disk blocks). Later (1978?) this code = was modified to optionally use separate buffer heads and buffer data, as = part of the UoC buffer mod=E2=80=99s (to alleviate the pressure on a = limited address space): https://minnie.tuhs.org/cgi-bin/utree.pl?file=3DSRI-NOSC/h/net/netbuf.h In 1979 the first Datakit software was done (Chesson), where I don=E2=80=99= t have much information. My hypothesis is that this used clists, but = I=E2=80=99m still trying to find a copy of his paper about this = implementation. Maybe the driver used a precursor to a streams like data = structure. Maybe this is also where MPX files fit in. Unfortunately, = currently I just don=E2=80=99t know. In 1980 new TCP software for the VAX is being written, and again it = needs to find a data buffer approach. This is =E2=80=98mbuf=E2=80=99s, = which are similar to the earlier =E2=80=99netbuf=E2=80=99s. The main = changes are that the size is increased to 128 bytes and that a start = offset is added, which makes it possible to remove headers without = copying the buffer. The =E2=80=98memberof=E2=80=99 field is replaced by = a second linkage pointer. The UoC idea to separate the header and data = is dropped again. They (mbufs) may have other origins as well, but as = far as Unix networking is concerned they are an evolution of the Arpa = Unix =E2=80=98netbuf=E2=80=99s: https://minnie.tuhs.org/cgi-bin/utree.pl?file=3DBBN-Vax-TCP/bbnnet/mbuf.h Then we get to (late) 1981. On the Labs side streams are created, in my = current understanding mainly to support networking and in particular = Datakit. The difference with Spider is that the Datakit interface was = not DMA based and that the data arrives in 9 bit envelopes instead of 8 = bit bytes. The 9th bit distinguishes between data bytes and control = bytes. Again neither clists nor disk buffers are a good match for = buffering and a new solution is created. Looking at buffers, these = consist of a header and a separate datablock. The datablock can be of = various sizes: 4, 16, 64 and 1024 bytes - a match with the varying = nature of network traffic. Each block has a type (a.o. DATA and CONTROL) = which has various uses, but certainly gives a home to the 9th Datakit = bit where needed: = https://github.com/Alhadis/Research-Unix-v8/blob/master/v8/usr/include/sys= /stream.h = https://github.com/Alhadis/Research-Unix-v8/blob/master/v8/usr/sys/dev/str= eam.c Over on the west coast the mbuf structure is extended with a new field, = =E2=80=9Ctype=E2=80=9D at the same time, and it transitions from being a = pure network data buffer to a more generic buffer. It remains at a fixed = size of 128 bytes with inline data; the option to store data in a = separate block of varying size will only come back late in 4.3BSD. For = the 4.2 version: = https://minnie.tuhs.org/cgi-bin/utree.pl?file=3D4.2BSD/usr/src/sys/h/mbuf.= h When viewed at this code level it would seem to me that by various = routes everybody had arrived at more or less the same the point in = network buffering approach by 1981. That has some interesting = implications. At this level, is there really a difference between a pipe = implemented as cross connected streams and a pipe implemented as cross = connected sockets? Is an unbound socket not about the same as an = unconnected stream head? But before getting into such discussions, I=E2=80=99d really like to get = the development story from 7th to 8th edition as clear and precise as = possible after 40 years. > On 30 Mar 2020, at 21:31, Clem Cole wrote: >=20 >=20 >=20 > On Mon, Mar 30, 2020 at 12:02 PM Paul Ruizendaal = wrote: > Comments in line: >=20 > > On 30 Mar 2020, at 15:26, Clem Cole wrote: > Your date of June '81 for the 4.1BSD release seems late, but I'll = accept it. 3BSD was 1979, and I thought 4BSD was a year later, with = 4.1BSD a few months after 4BSD (few people actually got 4BSD) >=20 > I am aware of that. =E2=80=9C20 years of Berkely Unix=E2=80=9D says = November 1980 for 4BSD and June 1981 for 4.1BSD. =46rom the SCCS log = I=E2=80=99d say November 9th and July 1st respectively. > Ok, that sounds right 6-7 months between them. That was the time of = the 'FASTVAX' work and the fight with Stanford over if BSD or VMS was to = be the official Arpa OS. I remember Joy was incredibly prolific during = that time. =46rom a user standpoint, 4BSD and 4.1BSD are really = similar, but he was hacking the kernel. It seemed like a new change = came out all the time. The CAD group (where I was) always wanted the = fastest system, so there was pressure on me to follow him, but I was = trying to make progress on the AP work (my thesis) so I was a little = loath to take the churn. >=20 > =20 >=20 >=20 > As far as I can tell CSRG integrated the BBN stack with (almost) 4.1 = in April 1981 - most of that is in a separate tree, with a few = #ifdef=E2=80=99s in the main tree. > I'm pretty sure that was mostly Eric Cooper (ecc) IIRC. We had a tape = directly from Gurwitz running in the CAD lab in Cory Hall, running on = Xerox 3M Ethernet and because we needed it between our three 780s = systems. We had ordered the 3C501's from 3COM and we were going to be = the Beta for the Interlan board - which I wrote the original driver and = gave it to Sam. I'm not sure when IngVAX arrived, Eric Schmidt had = Ing70 downstairs (and that was the ArpaNet connect via a VDH up the hill = to the IMP at LBL. We had been running BerkNET over serial lines. = Ernie and Kim were the two 780s over in Evan's (the big pile of 750s and = the C30 IMP were still in the future). >=20 >=20 > =20 > =46rom SCCS it seems that implementation work on sockets started in = Oct/Nov 1981 and 4.1a was released in April or March 1982. > That sounds about right. It was a bunch of wnj marathon hacking = sessions. >=20 > =20 > =46rom the partial coverage in SCCS I=E2=80=99d say there is no BBN = code in 4.1a anymore. > Mumble ... I sort of have a hard time with that. I knew and worked = with both of them in my day and have heard both sides of the story. Joy = had Gurwitz's code and was hacking it - that is just a fact. Claiming = pure authorship is a tad extreme. For instance, Andy VanDam told me = that the whole mbuf's stuff is something Rob originally wrote for a = project for him at Brown when he was an undergrad. Rob brought it BBN = and he used it for the OS-independent TCP (i.e. the HP3000 version) = originally and it landed in the UNIX version. Joy hacked on it, but = that's a data structure directly from Rob. On the other hand, Joy = types open curly brace, close curly brace, and hacks in code the fastest = of anyone I ever knew (someone, maybe Mary Ann Horton, once said he = 'wrote code at 9600'). But if you look at it, its sometimes not pretty. = It works and its usually fast, which is hard to argue with. >=20 >=20 > =20 > The socket API in that release was different from what ended being in = 4.1c and 4.2, > Definitely. I thought other hands besides Joy started to mess with = it (like Sam) >=20 > =20 > and seems to have had some wider visibility in 2.9BSD and UniPlus = System V - both of which used the 4.1a API (and probably code base). > Yeah, Keith was taking things was Sam at pretty hectic pace trying to = keep the 11's running. I would guess that took more of a 'good = enough' attitude, they got the TCP stack in the limited address space = and did not want to try much more >=20 >=20 > Yes, Summer 81 - Summer 82 must have been a vibrant year, with much = interaction between various Unix groups. > It was a fun time to be apart of it all. >=20 > =20 > By the way, dmr was on the 4.2BSD steering group. > Yes, I know. That's how I first got to know him.=20