From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, MAILING_LIST_MULTI autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 21264 invoked from network); 12 Aug 2023 09:06:49 -0000 Received: from minnie.tuhs.org (50.116.15.146) by inbox.vuxu.org with ESMTPUTF8; 12 Aug 2023 09:06:49 -0000 Received: from minnie.tuhs.org (localhost [IPv6:::1]) by minnie.tuhs.org (Postfix) with ESMTP id 3119740F84; Sat, 12 Aug 2023 19:06:43 +1000 (AEST) Received: from ewsoutbound.kpnmail.nl (ewsoutbound.kpnmail.nl [195.121.94.168]) by minnie.tuhs.org (Postfix) with ESMTPS id 3162C40F73 for ; Sat, 12 Aug 2023 19:06:33 +1000 (AEST) X-KPN-MessageId: 5885e3a3-38ef-11ee-8e3d-005056aba152 Received: from smtp.kpnmail.nl (unknown [10.31.155.37]) by ewsoutbound.so.kpn.org (Halon) with ESMTPS id 5885e3a3-38ef-11ee-8e3d-005056aba152; Sat, 12 Aug 2023 11:05:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=planet.nl; s=planet01; h=to:message-id:date:from:subject:mime-version:content-type; bh=QXyhQS9kWa2Ka30dhiw6B+6zrMQDHNz+15VUesVq5vU=; b=XoOIp3B1Wj/2XTNr8zXJDO/iJCT9IpYHY4rvLxipCT131lVjUYIXYnk3AfiBlChiTFxbVzbJGmeRL 2wjHY9mXyvow4+kcYJqBudGKaXhg9kFK0SIQFpVgez1032hWGi1Fq5P3ixKXduZxM/dD253yFymTDA bWxv3cJzYYYBtMqE= X-KPN-MID: 33|QVIB8//BZvPXaTSjjm+rruXMLakK/0mt/Y2CD1OGlI/n9yb8EM324eE6ppJtKQO gIZifANHqLvI8juaOEt/IaYH6SRIAtLYiXLWIYDJgQig= X-KPN-VerifiedSender: Yes X-CMASSUN: 33|JSxB3Y/nQQXbmawYnbXquigd+Y+RYExFIt0sHaQ4o8p29DVNbKPo6AIYQ1bDvp3 0VfH5TGqjVf/GZdMZc5HLzw== X-Originating-IP: 77.172.38.96 Received: from smtpclient.apple (77-172-38-96.fixed.kpn.net [77.172.38.96]) by smtp.kpnmail.nl (Halon) with ESMTPSA id 83cdc286-38ef-11ee-8a21-005056ab1411; Sat, 12 Aug 2023 11:06:25 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) From: Paul Ruizendaal In-Reply-To: Date: Sat, 12 Aug 2023 11:06:25 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <6EA66356-0C94-4802-81B6-1CF891EF6EC2@planet.nl> To: Warner Losh X-Mailer: Apple Mail (2.3654.120.0.1.13) Message-ID-Hash: US3F6PGTCENZDNCO7UWQXOJXLS5MU7XO X-Message-ID-Hash: US3F6PGTCENZDNCO7UWQXOJXLS5MU7XO X-MailFrom: pnr@planet.nl X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: "tuhs@tuhs.org" X-Mailman-Version: 3.3.6b1 Precedence: list Subject: [TUHS] Re: when did v8 or later get networking? List-Id: The Unix Heritage Society mailing list Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: > On 12 Aug 2023, at 07:08, Warner Losh wrote: >=20 > On Fri, Aug 11, 2023 at 3:05=E2=80=AFAM Paul Ruizendaal = wrote: > Bill Joy of CSRG concluded that the BBN stack did not perform = according to his expectations. Note that CSRG was focused on usage over = (thick) ethernet links, and BBN was focused on usage over Arpanet and = other wide-area networks (with much lower bandwidth, and higher latency = and error rates). He then in 1982 rewrote the stack to match the CSRG = environment, changing the design to use software interrupts instead of a = kernel thread and optimising the code (e.g. checksumming and fast code = paths). It was a matter of debate how new the code was, with the = extremes being that it was written from scratch using the spec versus it = being mostly copied. Looking at it with a nearly 50 year distance, it = seems in between: small bits of surviving SCCS suggest CSRG starting = with parts of BBN code followed by rapid, massive modification; the end = result is quite different but retained the =E2=80=98mbuf=E2=80=99 core = data structure and a BBN bug (off-by-one for OOB TCP segments). >=20 > When Kirk McKusick tells the story, UCB got a beta release (or early = access) of the BBN stack. UCB was supposed to add the socket interface = to whatever was there. But Bill Joy found it performed terribly = (multiple seconds to connect sometimes, single digit kB over 10Mb media, = etc). He optimized it to make it perform well. This was a combination of = rewriting chunks and tweaking other chunks, which matches your analysis = of SCCS. When BBN came back with their new, release ready stack Bill = supposedly said something like 'no thanks, we already got one that works = way better.' This is why much of the structure of the original BBN stack = survived the rewrite: if there wasn't a big issue with them, the design = and mechanisms wound up being conserved by this effort. It was too much = work to move from mbuf to something else, and too little gain. >=20 > I tried to find a good link, but they are in his BSD history = retrospective talks to differing degrees. Sorry I don't have an exact = reference. >=20 > Warner =3D=3D=3D > UCB got a beta release (or early access) of the BBN stack. This is certainly true. There are four surviving BBN tapes in the CSRG = archive, from memory the first was from August 1981 and the last from = early 1982. Again from memory, the oldest SCCS entries for the rewrite = are from Oct =E2=80=9981. All of this is on Kirk=E2=80=99s DVD. Indeed the first tape is early code, written to just interface with = Arpanet IMP=E2=80=99s. Things like routing are rudimentary or = hard-coded, etc. This is all filled out by the time of the last tape, = which also includes a token ring driver (written by Noel Chiappa, if I = remember well). There are two editions of Mike Muuss=E2=80=99 TCP-DIGEST mailing list = with posts on performance from Joy and Gurwitz respectively (both from = late 1981): https://groups.google.com/g/fa.tcp-ip/c/WNE_j4mAbAE/m/3nCB79uvNcUJ https://groups.google.com/g/fa.tcp-ip/c/zfYZh-kRlMg/m/pl-5oLQtYxIJ Jonathan Gray sent me a good link of one of Kirk=E2=80=99s talks that = addresses this bit of history: https://youtu.be/DEEr6dT-4uQ?t=3D706 =3D=3D=3D Maybe Kirk was using some hyperbole in that talk. As far as I can tell = the main issues were the below: - BBN coded the checksum routine in C and it compiled badly. Even Joy's = hand-optimised version still took some 25% of CPU when traffic maxed = out. - The time-out constants were 2s and 5s. This makes sense for the = Arpanet of the time. For local ethernet this was changed to 0.2s and = 0.5s. - The BBN code used at lot of bit fields. This too compiled badly, and = was later changed to and/or with #define=E2=80=99d constants - The BBN code took a very layered approach, often abstracting small = bits of functionality into a separate routine. Without compiler inlining = and a somewhat slow VAX subroutine mechanism this had a cost. Some of = these functions where later changed to #define=E2=80=99s instead of = functions. - Although Kirk singles out replacing the state machine with a big = switch statement in the above talk, I=E2=80=99m not sure this was a = major performance boost. It is certainly the most visible/recognisable = change in the source though. Somehow, this seems to have become core to = the debate at the time (ref. the BBN talk at the Summer '84 Usenix = conference). Maybe I underestimate the impact of this aspect. - The TCP management process ran as a kernel thread with normal = scheduling. On a loaded machine this meant that it could take seconds = for this process to be scheduled again. Changing this to a software = interrupt mechanism (somewhat similar to the runrun flag, although Joy = appears to have credited VMS for the idea) made things more responsive = and avoided context switches. Of all the above, it would seem to me that only the last point was = fundamental to the design. The other things appear relatively easy to = fix. =3D=3D=3D Doug McIlroy wrote: "I recall expressions of surprise (dismay?) at the = size of the BSD internet code, but without it we'd have lost our place = in the Unix community." An interesting question is how much of this size is unavoidable. Just = counting file lines (i.e. =E2=80=98wc -l=E2=80=99), 4.2BSD has some 2200 = lines of TCP code (i.e. just the TCP part), the =E2=80=9882 BBN = implementation 2400 lines, 8th edition had some 2400 lines, and the = first version of Plan9 some 2200 lines. Some parts of TCP could perhaps be simplified (pseudo headers and = =E2=80=98urgent' segments come to mind), but not much. Maybe this is = just the code size that it takes. URP is smaller, but if I remember well it does not handle out-of-order = packets or packet duplicates (both of which do not occur in a (virtual) = line switched context). On the other hand, these sizes compare with 900 lines for the IL = protocol in early Plan9.