The Unix Heritage Society mailing list
 help / color / mirror / Atom feed
* [TUHS] Re: Research Datakit notes
@ 2022-06-26  2:19 Noel Chiappa
  2022-06-26  9:46 ` steve jenkin
                   ` (2 more replies)
  0 siblings, 3 replies; 38+ messages in thread
From: Noel Chiappa @ 2022-06-26  2:19 UTC (permalink / raw)
  To: tuhs; +Cc: jnc

    > From: Paul Ruizendaal

    > it would seem to me that Sandy had figured out a core problem some 30
    > years before the TCP/IP world would come up with a similar solution. I
    > would not even be surprised if I learned that modern telco routers
    > transparantly set up virtual circuits for tcp traffic.

To fully explore this topic would take a book, which I don't have the energy
to write, and nobody would bother to read, but... 

Anyway, I'm not upon the latest and greatest high-speed routers: I saw some
stuff from one major vendor under NDA about a decade ago, but that's my most
recent - but at that point there was nothing that looked even _vaguely_ like
virtual circuits. (The stuff Craig was alluding to was just about
connectivity for getting bitts from _interface_ to _interface_ - if you don't
have a giant crossbar - which is going to require buffering on each input
anyway - how exactly do you get bits from board A to board Q - a single
shared bus isn't going to do it...)

A problem with anything like VC's in core switches is the growth of per-VC
state - a major high-speed node will have packets from _millions_ of TCP
connections flowing through it at any time. In the late-80's/early-90's - well
over 30 years ago - I came up with an advanced routing architecture called
Nimrod (see RFC-1992, "The Nimrod Routing Architecture"; RFC-1753 may be of
interest too); it had things called 'flows' which were half way between pure
datagrams (i.e. no setup - you just stick the right destination address in the
header and send it off) and VCs (read the RFCs if you want to kow why), and it
went to a lot of trouble to allow flow aggregation in traffic going to core
switches _precisely_ to limit the growth of state in core switches, which
would have traffic from millions of connections going through them.

I have barely begun to even scratch the surface, here.

	Noel

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-26  2:19 [TUHS] Re: Research Datakit notes Noel Chiappa
@ 2022-06-26  9:46 ` steve jenkin
  2022-06-26 20:35   ` Erik Fair
  2022-06-26 10:16 ` Paul Ruizendaal via TUHS
  2022-06-26 13:07 ` John Cowan
  2 siblings, 1 reply; 38+ messages in thread
From: steve jenkin @ 2022-06-26  9:46 UTC (permalink / raw)
  To: TUHS; +Cc: Noel Chiappa

> On 26 Jun 2022, at 12:19, Noel Chiappa <jnc@mercury.lcs.mit.edu> wrote:
> 
>> From: Paul Ruizendaal
> 
>> it would seem to me that Sandy had figured out a core problem some 30
>> years before the TCP/IP world would come up with a similar solution. I
>> would not even be surprised if I learned that modern telco routers
>> transparantly set up virtual circuits for tcp traffic.
> 
> To fully explore this topic would take a book, which I don't have the energy
> to write, and nobody would bother to read, but... 

packet switching won over Virtual Circuits in the now distant past
but in small, local and un-congested networks without reliability constraints,
any solution can look good. If “Performance is Hard”, so are Manageability & Reliability.

Packet switching hasn’t scaled well to Global size, at least IMHO.

Ethernet only became a viable LAN technology with advent of Twisted pair: point to point + Switches.
The One Big Idea for common channels, collision detection/avoidance, 
	became moot, even irrelevant with switch buffering & collision-less connectivity.

Watching networking types deal with ethernet multi-path routing & link failover - quite problematic for 802.1 layer-2 nets,
I can’t but think “If you don’t design a feature in, why expect it to show up?”.

All those voluminous CCITT / ITU-T standards were solving the problem “keep the network up & working”,
they knew it wasn’t easy ‘at scale’. Harder if you have to billing :)

Something I’ve never understood is how to design large-scale VM farms ,
where DB’s, Filestores & ’services’ (eg Web) have to be migrated between physical hosts,
made more difficult when services have to be split out of existing instances: who gets to keep the MAC & IP address?

I can’t see either a layer-2 or IP addressing scheme that’s simple and works well / reliably for large VM fleets.
Whereas, ‘well known’ Virtual Circuit endpoints can be permanently allocated & tied to an instance/service.

Packet encapsulation & separation techniques, like MPLS, VPN’s and VLAN’s, are now commonplace in IP networks.
Even the ubiquitous (multi-level) use of NAT for traffic isolation & preventing unwanted ingress shows, IMO, a design failure.

Why would I advertise my internal network with IPv6 and allow every bad actor to probe my entire network? Hellish security admin for little benefit.
I’ve never understood this reasoning.

It seems to me, IP networks are trying to provide the functionality of Virtual Circuits - dedicated paths, transparent failover & even committed data rates.

There are two very valuable functions that ‘dedicated paths’, implying topology-aware switches, can do that packet-switch can’t:

	- Network-based “Call Transfer”
		There’s no packet-switch mechanism I know,
		that allows a connection to an advertised ‘gateway’ or authenticator,
		that then allows invisible network rerouting of the connection to a ’service provider’ or host,
		without giving away information to the caller.

		This would be really handy for “follow me” connections, moving between user devices, even different access networks.
		Apple have just announced something like this - moving running sessions between Mac, iPad, iPhone as needed.

		The inverse, “blocked caller id”, can’t be done in IP without a repeating host, ence VPN’s.
		[ In telephony, all A & B parties are identified for billing & forensics / interception. Addrs only hidden from other users. ]


	- “Multicast” / Conference calls
		Packet switched ‘conferencing’ is rarely done by the Network, despite multicast IP standards 25yrs of more old, they are rarely used.
		If major router vendors had good solutions, they’d be pushing them.
		The Pandemic brought forward the need for video calling everywhere, created a market for “Conferencing” products & “Streaming”.

		Isn’t Multicast of Free To Air TV & radio over Internet both obvious & desirable?

		It should be the modern equivalent of & replacement for ‘wireless’ broadcasting.
		Should be trivial if everyone has one or more mobile devices and a home Internet connection.
		The current Point-Point transmission model of Streaming Services - no scheduled broadcasts, On-demand only -
			seems to be the most resource intensive, hence costly, solution possible.

		Flash storage is under $0.20/GB, a small PVR with 1TB of “circular buffer” could cheaply convert “Scheduled” to “On Demand” at the customer premises.
		[ Samsung have announced a 1.5TB micro-SD card, targeted at surveillance recording. Purportedly, “100 days of video”. Cost? NFI ]


One of the modern challenges for Enterprise Customers is “change network provider”.
One major Aussie firm just took two years to move Service Provider and it was ’news’.

25yrs ago, and probably now as well, interconnecting / splitting corporate networks following merges / de-merges was a huge task.
IP networks require re-numbering and basic IP services, like DNS, email, web, need re-provisioning / re-platforming.

The Worst Case error - happens too often - is Network Admins cutting themselves off from the Remote Admin Network.
‘Recovery’ is slow & painful - all sites / all devices may have to be visited, even if you have a backup network link.
Did Microsoft have a fault like this in the last 12 mths after it acquired a vendor? There was a big outage.

Post merger, Enterprise phone switch & voice network are relatively simple & fast to change and rarely result in Admins losing Remote Admin access.

It’s not that I don’t like cheap, easy Ethernet devices that mostly Just Work and at small scale are simple to setup and easy to manage.
30 years of hardware development has been a boon to us all, in almost all ways.

Rob Pike wrote a great piece (in the 1980’s) about cheap hardware forcing systems software to take up the slack, an example I thought of Ashby’s Law of Requite Variety.

Have we arrived in this Packet Switch nightmare because of "Microsoft is Best” thinking?

I don’t have a proper name for this blinkered view of the world, backed by arrogance & the assumption “What I don’t know can’t matter”.

regards
steve j


--
Steve Jenkin, IT Systems and Design 
0412 786 915 (+61 412 786 915)
PO Box 38, Kippax ACT 2615, AUSTRALIA

mailto:sjenkin@canb.auug.org.au http://members.tip.net.au/~sjenkin


^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-26  2:19 [TUHS] Re: Research Datakit notes Noel Chiappa
  2022-06-26  9:46 ` steve jenkin
@ 2022-06-26 10:16 ` Paul Ruizendaal via TUHS
  2022-06-26 13:07 ` John Cowan
  2 siblings, 0 replies; 38+ messages in thread
From: Paul Ruizendaal via TUHS @ 2022-06-26 10:16 UTC (permalink / raw)
  To: Noel Chiappa; +Cc: tuhs


> On 26 Jun 2022, at 04:19, Noel Chiappa <jnc@mercury.lcs.mit.edu> wrote:
> 
> I have barely begun to even scratch the surface, here.

I feared as much. I will have to think about the modern network as a cloud that just works, without really understanding the in-depth how and why.

Will read those RFC’s, though -- thank you for pointing them out.

Paul


^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-26  2:19 [TUHS] Re: Research Datakit notes Noel Chiappa
  2022-06-26  9:46 ` steve jenkin
  2022-06-26 10:16 ` Paul Ruizendaal via TUHS
@ 2022-06-26 13:07 ` John Cowan
  2022-06-26 13:35   ` Larry McVoy
  2 siblings, 1 reply; 38+ messages in thread
From: John Cowan @ 2022-06-26 13:07 UTC (permalink / raw)
  To: Noel Chiappa; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 460 bytes --]

On Sat, Jun 25, 2022 at 10:20 PM Noel Chiappa <jnc@mercury.lcs.mit.edu>
wrote:


> it had things called 'flows' which were half way between pure
> datagrams (i.e. no setup - you just stick the right destination address in
> the
> header and send it off) and VCs (read the RFCs if you want to kow why),
>

In that connection I have always admired Padlipsky's RFC 962, which
exploits the existing TCP architecture to do just this.  So simple, so
easy, so Unixy.

[-- Attachment #2: Type: text/html, Size: 1108 bytes --]

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-26 13:07 ` John Cowan
@ 2022-06-26 13:35   ` Larry McVoy
  2022-06-26 13:58     ` John Cowan
  0 siblings, 1 reply; 38+ messages in thread
From: Larry McVoy @ 2022-06-26 13:35 UTC (permalink / raw)
  To: John Cowan; +Cc: Noel Chiappa, The Eunuchs Hysterical Society

On Sun, Jun 26, 2022 at 09:07:49AM -0400, John Cowan wrote:
> On Sat, Jun 25, 2022 at 10:20 PM Noel Chiappa <jnc@mercury.lcs.mit.edu>
> wrote:
> 
> 
> > it had things called 'flows' which were half way between pure
> > datagrams (i.e. no setup - you just stick the right destination address in
> > the
> > header and send it off) and VCs (read the RFCs if you want to kow why),
> >
> 
> In that connection I have always admired Padlipsky's RFC 962, which
> exploits the existing TCP architecture to do just this.  So simple, so
> easy, so Unixy.

I knew Mike, interesting dude.  His "The Elements of Networking Style" is
a very fun read but also, for me, just getting to understand networking,
it snapped a bunch of stuff into focus.  I think you have to read it
at just the right spot in your career and I did.  Great little book and
full of jabs like "If you know what you are doing, 3 layers are enough.
If you don't, 7 aren't."

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-26 13:35   ` Larry McVoy
@ 2022-06-26 13:58     ` John Cowan
  0 siblings, 0 replies; 38+ messages in thread
From: John Cowan @ 2022-06-26 13:58 UTC (permalink / raw)
  To: Larry McVoy; +Cc: Noel Chiappa, The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 1205 bytes --]

I'll check it out eventually.  $10 on Ebay (the cheapest per bookfinder.com)
is a little steep nowadays.  Maybe I'll spring for a <$17 new copy.  No
Kindle, alas.

On Sun, Jun 26, 2022 at 9:35 AM Larry McVoy <lm@mcvoy.com> wrote:

> On Sun, Jun 26, 2022 at 09:07:49AM -0400, John Cowan wrote:
> > On Sat, Jun 25, 2022 at 10:20 PM Noel Chiappa <jnc@mercury.lcs.mit.edu>
> > wrote:
> >
> >
> > > it had things called 'flows' which were half way between pure
> > > datagrams (i.e. no setup - you just stick the right destination
> address in
> > > the
> > > header and send it off) and VCs (read the RFCs if you want to kow why),
> > >
> >
> > In that connection I have always admired Padlipsky's RFC 962, which
> > exploits the existing TCP architecture to do just this.  So simple, so
> > easy, so Unixy.
>
> I knew Mike, interesting dude.  His "The Elements of Networking Style" is
> a very fun read but also, for me, just getting to understand networking,
> it snapped a bunch of stuff into focus.  I think you have to read it
> at just the right spot in your career and I did.  Great little book and
> full of jabs like "If you know what you are doing, 3 layers are enough.
> If you don't, 7 aren't."
>

[-- Attachment #2: Type: text/html, Size: 1850 bytes --]

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-26  9:46 ` steve jenkin
@ 2022-06-26 20:35   ` Erik Fair
  2022-06-26 21:53     ` Steve Jenkin
  0 siblings, 1 reply; 38+ messages in thread
From: Erik Fair @ 2022-06-26 20:35 UTC (permalink / raw)
  To: steve jenkin; +Cc: TUHS, Noel Chiappa


> On Jun 26, 2022, at 02:46, steve jenkin <sjenkin@canb.auug.org.au> wrote:
> 
> One of the modern challenges for Enterprise Customers is “change network provider”.
> One major Aussie firm just took two years to move Service Provider and it was ’news’.
> 
> 25yrs ago, and probably now as well, interconnecting / splitting corporate networks following merges / de-merges was a huge task.
> IP networks require re-numbering and basic IP services, like DNS, email, web, need re-provisioning / re-platforming.


I take issue with this, and have experience to backup my view: it is not hard to change Internet Service Providers (ISP) for large corporations - I did it several times during my tenure as the “Internet guy” for Apple Computer, Inc., an ~$8bn revenue multinational corporation. You just have to plan for it properly, handle transitions gracefully (ideally, with overlap), and keep control of (do not outsource) key assets like domain names, public IP network address assignments, and services like e-mail (SMTP).

Apple's primary “face” to the world in July 1988 when I arrived was a DEC VAX-11/780 running 4.3 BSD Unix. I renumbered it once, when we changed from CSNET’s X25NET (9.6Kb/s IP-over-X.25 via TELENET/SPRINTlink) to a 56Kb/s (DS0) leased line - we started using an assigned class B network: 130.43/16 - the VAX became 130.43.2.2. It retained its name as “apple.com” until it was decommissioned in the late 1990s.

Apple was on CSNET primarily (where the VAX was connected), and had a separated BARRNET T1 that belonged to the A/UX group (A/UX was Apple’s version of Unix). I connected our two external “perimeter” networks , set up fail-over IP routing (initially with RIP (ugh), later deployed BGP), upgraded CSNET in California, added a second site in Cambridge, MA to CSNET, later moved Cambridge to NEARNET, replaced CSNET in California with CERFNET, helped our European offices in Zeist, NL connect to SURFnet, and our customer service division in Austin, TX to SPRINTNET (they had to forcibly (disruptively) renumber, but they screwed up by not listening to me during their network planning).

We did have to clean up an internal IP address mess when I got there: lots of “picked out of the air” net numbers in use: 90/8, 92/8, 95/8, etc.; those were all renumbered into properly assigned net 17/8, which Apple still uses today.

Before the WWW, “ftp.apple.com” offered up MacOS software distribution via anonymous FTP from a Mac IIcx running A/UX under my desk, and to prevent our connectivity from being overwhelmed when MacOS releases were published, I wrote an ftp-listener to restrict the number of FTP connections per peer classful network number for that server. I later installed that code at the Smithsonian Institution on a Unix machine they set up for public anonymous FTP of their digitally-scanned historical photography archive, because they had the same problem: limited connectivity, popular content.

As for mergers, acquisitions, and spinoffs, I was involved in networking for Coral Software (acquired; purveyors of Macintosh Common Lisp) which became Apple Cambridge; Taligent which was an Apple/IBM joint venture (an Apple OS & software group spin-out; they were set up with a separate public class B network number and domain name from the get-go to provide for maximum future flexibility, even though they were initially in a building on the Apple Campus in Cupertino, CA and thus on the dark fiber plant, and Apple (me) was their ISP) and ultimately IBM bought out Apple; The Apple Engineering secured connection to the “Somerset Design Center” in Austin, TX (a joint venture between Apple, IBM, and Motorola (the AIM alliance) from whence the PowerPC processor came), which was tricky.

I’ll grant that when I was doing this work, Internet connectivity downtime didn’t mean immediate revenues losses (e.g., from not being able to accept orders in the Apple Online store, which did not (yet) exist), but e-mail was critical to Apple’s world-wide operations, and making e-mail robust & reliable was the first job I was hired to do: fix sendmail(8), take control of DNS (CSNET controlled the apple.com zone file when I arrived), and naturally, that extended to making Internet connectivity & routing robust as well.

The main problem that “modern” mergers & acquisitions face in coordinating networking is a direct result of something else I fought against in the IETF: private IP address space (RFC 1918), and Network Address Translation (NAT) - see RFC 1627. Unfortunately, I and my colleagues lost that debate, which is why one now sees double/triple NAT setups to try and make overlapping private IPv4 addressed networks talk to each other. You’ll notice that I eschewed use of private address space while at Apple, and having unique public IP address space made most things simpler - funny how following the architecture works better than fighting it.

Unix is tied into all of this because it has been the platform where (most often) the first implementation of many of these protocols or hacks is written. It takes a flexible operating system to make such a wide range of applications possible. However, properly setting up the rest of the communications infrastructure (in which Unix must communicate) is important too.

	Erik Fair
 

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-26 20:35   ` Erik Fair
@ 2022-06-26 21:53     ` Steve Jenkin
  0 siblings, 0 replies; 38+ messages in thread
From: Steve Jenkin @ 2022-06-26 21:53 UTC (permalink / raw)
  To: TUHS; +Cc: Erik Fair

Erik,

Thanks for the reply & the account of your work. 
You seemed to do a lot less work than I saw networking teams forced into.
As you say, “work with it”.

I worked at the ANU for a short time. 
They used a Class-B, with address ranges delegated to (many) local admin, but Central IT demanded every device connect to their switch, which allowed them to monitor ports.
I think they’d have a hard time renumbering, my section didn’t even have DHCP.

Here’s the reference I spared the list initially.
Very thin on details. The only more detailed info is a video.

Australia Post's telco transformation named top IT project
		<https://www.itnews.com.au/news/australia-posts-telco-transformation-named-top-it-project-581371>

	Billed as the largest project of its type, the project wrapped up in November last year after more than two years of work to address critical network performances challenges and high operating costs.
	The telecommunication transformation saw the rollout a new software-defined wide area network (SD-WAN) across 4000 sites.

regards
steve j

> On 27 Jun 2022, at 06:35, Erik Fair <fair@netbsd.org> wrote:
> 
>> 
>> On Jun 26, 2022, at 02:46, steve jenkin <sjenkin@canb.auug.org.au> wrote:
>> 
>> One of the modern challenges for Enterprise Customers is “change network provider”.
>> One major Aussie firm just took two years to move Service Provider and it was ’news’.
>> 
>> 25yrs ago, and probably now as well, interconnecting / splitting corporate networks following merges / de-merges was a huge task.
>> IP networks require re-numbering and basic IP services, like DNS, email, web, need re-provisioning / re-platforming.
> 
> 
> I take issue with this, and have experience to backup my view: it is not hard to change Internet Service Providers (ISP) for large corporations - I did it several times during my tenure as the “Internet guy” for Apple Computer, Inc., an ~$8bn revenue multinational corporation. You just have to plan for it properly, handle transitions gracefully (ideally, with overlap), and keep control of (do not outsource) key assets like domain names, public IP network address assignments, and services like e-mail (SMTP).

--
Steve Jenkin, IT Systems and Design 
0412 786 915 (+61 412 786 915)
PO Box 38, Kippax ACT 2615, AUSTRALIA

mailto:sjenkin@canb.auug.org.au http://members.tip.net.au/~sjenkin


^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-07-02  2:51       ` Grant Taylor via TUHS
@ 2022-07-02  2:57         ` Larry McVoy
  0 siblings, 0 replies; 38+ messages in thread
From: Larry McVoy @ 2022-07-02  2:57 UTC (permalink / raw)
  To: Grant Taylor; +Cc: tuhs

On Fri, Jul 01, 2022 at 08:51:53PM -0600, Grant Taylor via TUHS wrote:
> On 6/25/22 7:17 PM, Paul Ruizendaal wrote:
> >In his video (https://www.youtube.com/watch?v=ojRtJ1U6Qzw), Sandy explains
> >why he became dissatisfied with Spider and the main reason was that doing
> >switching/routing on a mini computer was just plain inefficient as
> >compared to a telephone switch (at 37:06). This was 1972. The result was a
> >new design, Datakit, that could route/switch packets at high speed and in
> >parallel.
> 
> I didn't realize, or even fathom that something preceded ATM.
> 
> Now it seems like Spider preceded Datakit which preceded ATM.  Very
> interesting.

They were Bell Labs, funded by the phone company.  It makes sense that
they would look at networking through the phone company lens.

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-26  1:17     ` Paul Ruizendaal
@ 2022-07-02  2:51       ` Grant Taylor via TUHS
  2022-07-02  2:57         ` Larry McVoy
  0 siblings, 1 reply; 38+ messages in thread
From: Grant Taylor via TUHS @ 2022-07-02  2:51 UTC (permalink / raw)
  To: tuhs

[-- Attachment #1: Type: text/plain, Size: 633 bytes --]

On 6/25/22 7:17 PM, Paul Ruizendaal wrote:
> In his video (https://www.youtube.com/watch?v=ojRtJ1U6Qzw), Sandy 
> explains why he became dissatisfied with Spider and the main reason 
> was that doing switching/routing on a mini computer was just plain 
> inefficient as compared to a telephone switch (at 37:06). This was 
> 1972. The result was a new design, Datakit, that could route/switch 
> packets at high speed and in parallel.

I didn't realize, or even fathom that something preceded ATM.

Now it seems like Spider preceded Datakit which preceded ATM.  Very 
interesting.



-- 
Grant. . . .
unix || die


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4017 bytes --]

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-26  1:41 ` Anthony Martin
  2022-06-26  9:52   ` Ralph Corderoy
  2022-06-26 11:04   ` Paul Ruizendaal
@ 2022-06-29 20:21   ` Paul Ruizendaal
  2 siblings, 0 replies; 38+ messages in thread
From: Paul Ruizendaal @ 2022-06-29 20:21 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society


> Would you happen to know where I can find copies of these three
> papers?
> 
> A. G. Fraser, "Datakit - A Modular Network for Synchronous and
> Asynchronous Traffic", Proc. ICC 79, June 1979, Boston, Ma.,
> pp.20.1.1-20.1.3
> 
> G. L. Chesson, "Datakit Software Architecture", Proc. ICC 79, June
> 1979, Boston Ma., pp.20.2.1-20.2.5
> 
> G. L. Chesson and A. G. Fraser, "Datakit Network Architecture," Proc.
> Compcon 80, February 1980, San Fransisco CA., pp.59-61


I just remembered that I had received a copy of a file note (50+ pages) that Greg Chesson wrote in 1982 about the "CMC” control software for Datakit. I think it covers the same ground as the 1979 paper, but in far greater detail and with two more years of development. In short, the connection protocol in CMC is based on the exchange of binary messages. That was replaced (for the most part) by text-based messages in the later TDK control software.

It is here (it is a 16MB pdf):

https://www.jslite.net/notes/dk3.pdf

To compare, here are the first two design documents on sockets. I looked for these for many years (even had the Berkeley library manually search the boxes with CSRG documents that Kirk McKusick had sent there - to no avail), and then in 2021 Rich Morin found them in the papers of Jim Joyce. I’m still very thankful for this.

These two papers were written in the summer of 1981 and circulated to the newly formed steering committee for what was to become 4.2BSD (note: ~5MB pdf each).

The first is specifically on networking:

https://www.jslite.net/notes/joy1.pdf

The second outlines the overall ambitions for the new version (including a summary of the above document). It has an interesting view of John Reiser’s VM code in its section 3.17 as well:

https://www.jslite.net/notes/joy2.pdf

What was proposed is not quite the sockets we know, but the general direction is set and the reasoning is explained. Reading the Chesson and Joy paper side by side makes for an interesting comparison of thinking on these topics in the early 80’s.

Maybe they are worth storing in the TUHS archive.

Wbr,

Paul




^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-28 21:34               ` Richard Salz
@ 2022-06-29  6:07                 ` Stuart Remphrey
  0 siblings, 0 replies; 38+ messages in thread
From: Stuart Remphrey @ 2022-06-29  6:07 UTC (permalink / raw)
  To: Richard Salz; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 727 bytes --]

Yes, I'd thought of Sun's Jini here, part of their Java push -- which IIRC
exposed network (un)reliability and tried to address it through service
lookup and limited-lifetime lease/renew for remote resource access.
Though I'm not sure it went anywhere?


(or how TUHS-relevant this is/was... apologies if not)


-----
Hmm, apparently Jini became Apache River; last release in 2016, then
retired Feb 2022:
https://en.wikipedia.org/wiki/Jini


On Wed, 29 Jun 2022, 05:36 Richard Salz, <rich.salz@gmail.com> wrote:

> > My theory is that networking is different because it breaks all the time.
>
> This Sun research paper from 1994
> https://scholar.harvard.edu/waldo/publications/note-distributed-computing
> is a classic.
>
>
>

[-- Attachment #2: Type: text/html, Size: 1490 bytes --]

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-28 21:19             ` Lawrence Stewart
@ 2022-06-28 21:34               ` Richard Salz
  2022-06-29  6:07                 ` Stuart Remphrey
  0 siblings, 1 reply; 38+ messages in thread
From: Richard Salz @ 2022-06-28 21:34 UTC (permalink / raw)
  To: Lawrence Stewart; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 199 bytes --]

> My theory is that networking is different because it breaks all the time.

This Sun research paper from 1994
https://scholar.harvard.edu/waldo/publications/note-distributed-computing
is a classic.

[-- Attachment #2: Type: text/html, Size: 366 bytes --]

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-28 15:50 Noel Chiappa
@ 2022-06-28 21:32 ` Lawrence Stewart
  0 siblings, 0 replies; 38+ messages in thread
From: Lawrence Stewart @ 2022-06-28 21:32 UTC (permalink / raw)
  To: Noel Chiappa; +Cc: tuhs

I’ll argue there is quite a lot known about where to put network functionality, much
of it from HPC.  If you want minimum latency and minimum variance of latency, both
of which are important to big applications, you make the network reliable and move
functionality into the host adapters and the switches.  The code path at each end
of a very good MPI implementation will be under 200 machine instructions, all in
user mode.  There is no time to do retry or variable code paths.

Doesn’t work on WANs of course, or at consumer price points.

(I think there is still a lot to do, because the best networks still hover around 
800 nanoseconds calling SEND to returning from RECV, and I think it could be 100).

-L

> On 2022, Jun 28, at 11:50 AM, Noel Chiappa <jnc@mercury.lcs.mit.edu> wrote:
> 
>> From: Rob Pike
> 
>> having the switch do some of the call validation and even maybe
>> authentication (I'm not sure...) sounds like it takes load off the host.
> 
> I don't have enough information to express a judgement in this particular
> case, but I can say a few things about how one would go about analyzing
> questions of 'where should I put function [X]; in the host, or in the
> 'network' (which almost inevitably means 'in the switches')'.
> 


^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-28 13:33           ` Dan Cross
@ 2022-06-28 21:19             ` Lawrence Stewart
  2022-06-28 21:34               ` Richard Salz
  0 siblings, 1 reply; 38+ messages in thread
From: Lawrence Stewart @ 2022-06-28 21:19 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On 2022, Jun 28, at 9:33 AM, Dan Cross <crossd@gmail.com> wrote:
> 
> On Tue, Jun 28, 2022 at 8:46 AM Rob Pike <robpike@gmail.com> wrote:
>> One of the reasons I'm not a networking expert may be relevant here.
>> With networks, I never found an abstraction to hang my hat on.
> 

My theory is that networking is different because it breaks all the time.

One can get away with writing local file system programs without checking the return codes on every single call and figuring out what you should do in the case of every possible error.

It is much harder to get away with that sort of thing when writing network applications.

And of course there is a long history of things that look like file systems but which have network failure modes, and treating them like they are reliable often does not end well.

Full up reliable applications with replicated storage and multiple network availability zones and raft/paxos/byzantine generals level coding are pretty arcane.
I know I am not qualified to write one.

File systems are “good enough” that you can depend on them, mostly.  Networks are not.

-L


^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-28 12:45         ` Rob Pike
  2022-06-28 13:33           ` Dan Cross
  2022-06-28 16:11           ` Tom Teixeira
@ 2022-06-28 18:28           ` John Floren
  2 siblings, 0 replies; 38+ messages in thread
From: John Floren @ 2022-06-28 18:28 UTC (permalink / raw)
  To: tuhs

On 6/28/22 05:45, Rob Pike wrote:
> [...] I know it's amazing engineering and
> all that, but why aren't we allowed to program the I/O without all that
> fuss? What makes networks so _different_? A telling detail is that the
> original sockets interface had send and recv, not read and write. From
> day 1 in Unix land at least, networking was special, and it remains so,
> but I fail to see why it needs to be.
>
> It just seems there has to be a better way. Sockets are just so
> unpleasant, and the endless nonsense around network configuration doubly so.
>

I was pretty sad when netchans were discontinued. A colleague was
sufficiently attached to them that he kept his own branch of the library
going for a while.



john


^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-28 17:05             ` Adam Thornton
@ 2022-06-28 17:43               ` John Labovitz
  0 siblings, 0 replies; 38+ messages in thread
From: John Labovitz @ 2022-06-28 17:43 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

I’m generally a lurker here, but this has been an interesting conversation to observe. I often hesitate to post, as to not offend you folks who literally invented this stuff, but I thought it might be helpful to share some experiences with socket-based I/O.

For the first time in ~20 years, I’ve recently been writing low-level (eg, not through a library layer/framework) socket code in straight C, for an art project based on an ESP32 embedded/SoC. I first played around with the socket API in the mid-80s, and then wrote a lot of socket-using code in the 1990s for the Watchguard Firebox, the first Linux-based appliance firewall.

I have to say that I really enjoying programming with sockets. I feel that it *does* make a lot of sense if I'm thinking directly about the TCP/IP stack, *and* if my code has a good 'impedance match' to the protocols. If I’m writing a server, I’m dealing with connections and queues and various-sized packets/messages/blocks, which have to fit into some decision of memory usage (often true in embedded systems). Usually I’m not simply writing, say, a file server that simply reads from disc and sends bytes out through a stream and then calls close().

I also believe that the sockets API really comes into its own with high-capacity, non-threaded, non-blocking servers or clients — that is, ones that use select() or poll() and then recv() and send() or their variants. I’m sure that if I didn’t have a sockets API and only had open(), read(), write(), etc., I could make it work, but there’s something almost beautiful that happens at a large scale with many non-blocking sockets (see: the reactor pattern) that I don’t think would translate as well with a typical everything-is-a-file model.

My opinion solely, of course. But I’m simply happy that both socket- and file-based APIs exist. Each has their purpose.

—John


> On Jun 28, 2022, at 19:05, Adam Thornton <athornton@gmail.com> wrote:
> 
> 
> 
>> On Jun 28, 2022, at 6:13 AM, Marc Donner <marc.donner@gmail.com> wrote:
>> 
>> What I don't understand is whether Rob's observation about networking is *fundamental* to the space or *incidental* to the implementation.  I would love to be educated on that.
> 
> And there it is!  THAT was the sentence--well, ahort paragraph--that jogged my memory as to why this seemed familiar.
> 
> If you go back to _The Unix-Hater's Handbook_ (I know, I know, bear with me), one of the things I noticed and pointed out in my review (https://athornton.dreamwidth.org/14272.html) is how many of the targets of hatred, twenty years down the line, turned out to be unix-adjacent, and not fundamental.
> 
> In the book, these were things like Usenet and sendmail.cf (indeed, those were the two big ones).
> 
> But the current discussion: is the thing we don't like Berkeley Sockets?  Is it TCP/IP itself?  Is the the lack of a Unixy abstraction layer over some lower-level technology?  To what degree is it inherent?
> 
> I mean, obviously, to some degree it's all three, and I think a large but fairly unexamined part of it is that TCP/IP these days almost always at least pretends to be sitting on top of Ethernet at the bottom...but of course Classic Ethernet largely died in the...early 2000s, I guess?...when even extremely cheap home multiple-access-devices became switches rather than hubs.
> 
> Some sort of inter-machine networking is clearly inherent in a modern concept of Unix.  I think we're stuck with the sockets interface and IP, whether we like them or not.  They don't bother me a great deal, but, yes, they do not feel as unixy as, say, /dev/tcp does.  But the interesting thing is that I think that is Unix-adjacent or, like the UHH distate for Unix filesystems, it's at least incidental and could be replaced if the desire arose.  And I think we already have the answer about what the abstraction is, albeit at an application rather than the kernel level.
> 
> To answer Rob's question: I think the abstraction is now much farther up the stack.  To a pretty good first approximation, almost all applications simply definte their own semantics on top of HTTP(S) (OK, OK, Websockets muddy the waters again) and three-to-five verbs.  There's an incantation to establish a circuit (or a "session" if you're under the age of 50, I guess), and then you GET, DELETE, and at least one of PUT/POST/PATCH, for "read", "unlink", and "write".  This does seem to be a more record-oriented (kids these days get snippy if you call them "records" rather than "objects" but w/e) format than a stream of bytes (or at least you put an abstraction layer in between your records and the stream-of-octets that's happening).
> 
> This is certainly not efficient at a wire protocol level, but it's a fairly small cognitive burden for people who just want to write applications that communicate with each other.
> 
> Adam


^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-28 13:13           ` Marc Donner
  2022-06-28 14:41             ` Clem Cole
@ 2022-06-28 17:05             ` Adam Thornton
  2022-06-28 17:43               ` John Labovitz
  1 sibling, 1 reply; 38+ messages in thread
From: Adam Thornton @ 2022-06-28 17:05 UTC (permalink / raw)
  To: Marc Donner; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 2887 bytes --]



> On Jun 28, 2022, at 6:13 AM, Marc Donner <marc.donner@gmail.com> wrote:
> 
> What I don't understand is whether Rob's observation about networking is *fundamental* to the space or *incidental* to the implementation.  I would love to be educated on that.

And there it is!  THAT was the sentence--well, ahort paragraph--that jogged my memory as to why this seemed familiar.

If you go back to _The Unix-Hater's Handbook_ (I know, I know, bear with me), one of the things I noticed and pointed out in my review (https://athornton.dreamwidth.org/14272.html) is how many of the targets of hatred, twenty years down the line, turned out to be unix-adjacent, and not fundamental.

In the book, these were things like Usenet and sendmail.cf (indeed, those were the two big ones).

But the current discussion: is the thing we don't like Berkeley Sockets?  Is it TCP/IP itself?  Is the the lack of a Unixy abstraction layer over some lower-level technology?  To what degree is it inherent?

I mean, obviously, to some degree it's all three, and I think a large but fairly unexamined part of it is that TCP/IP these days almost always at least pretends to be sitting on top of Ethernet at the bottom...but of course Classic Ethernet largely died in the...early 2000s, I guess?...when even extremely cheap home multiple-access-devices became switches rather than hubs.

Some sort of inter-machine networking is clearly inherent in a modern concept of Unix.  I think we're stuck with the sockets interface and IP, whether we like them or not.  They don't bother me a great deal, but, yes, they do not feel as unixy as, say, /dev/tcp does.  But the interesting thing is that I think that is Unix-adjacent or, like the UHH distate for Unix filesystems, it's at least incidental and could be replaced if the desire arose.  And I think we already have the answer about what the abstraction is, albeit at an application rather than the kernel level.

To answer Rob's question: I think the abstraction is now much farther up the stack.  To a pretty good first approximation, almost all applications simply definte their own semantics on top of HTTP(S) (OK, OK, Websockets muddy the waters again) and three-to-five verbs.  There's an incantation to establish a circuit (or a "session" if you're under the age of 50, I guess), and then you GET, DELETE, and at least one of PUT/POST/PATCH, for "read", "unlink", and "write".  This does seem to be a more record-oriented (kids these days get snippy if you call them "records" rather than "objects" but w/e) format than a stream of bytes (or at least you put an abstraction layer in between your records and the stream-of-octets that's happening).

This is certainly not efficient at a wire protocol level, but it's a fairly small cognitive burden for people who just want to write applications that communicate with each other.

Adam

[-- Attachment #2: Type: text/html, Size: 3874 bytes --]

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-28 12:45         ` Rob Pike
  2022-06-28 13:33           ` Dan Cross
@ 2022-06-28 16:11           ` Tom Teixeira
  2022-06-28 18:28           ` John Floren
  2 siblings, 0 replies; 38+ messages in thread
From: Tom Teixeira @ 2022-06-28 16:11 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On 6/28/22 8:45 AM, Rob Pike wrote:
> One of the reasons I'm not a networking expert may be relevant here. 
> With networks, I never found an abstraction to hang my hat on. Unlike 
> with file systems and files, or even Unix character devices, which 
> provide a level of remove from the underlying blocks and sectors and 
> so on, the Unix networking interface always seemed too low-level and 
> fiddly, analogous to making users write files by managing the blocks 
> and sectors themselves. It could be all sockets' fault, but when I 
> hear networking people talk about the protocols and stacks and routing 
> and load shedding and ....my ears droop. I know it's amazing 
> engineering and all that, but why aren't we allowed to program the I/O 
> without all that fuss? What makes networks so _different_? A telling 
> detail is that the original sockets interface had send and recv, not 
> read and write. From day 1 in Unix land at least, networking was 
> special, and it remains so, but I fail to see why it needs to be.

Two observations:

At the time, I think everyone was searching for the right abstraction. I 
don't remember the whole talk, but just an anecdote by, I think, David 
Clark. I don't remember if this was just a seminar at MIT LCS or perhaps 
at SIGOPS. In any case, David talked about trying to get some database 
people to use the virtual memory and file systems abstractions that had 
been built by Multics. They agreed that these were nice abstractions, 
but in the mean time, "get out of our way and let us at the disk."

Since networking developers and users were all searching for the right 
abstraction, and new hardware, protocols, and software interfaces were 
being proposed on what seemed like a weekly basis, many of the proposed 
interfaces tried to expose low level mechanisms as well as high level 
stream abstractions, preserving the hope that something like TCP could 
be implemented at user code level rather than the kernel.

Secondly, I had to dig up a reference for the Chaosnet software (MIT AI 
memo 628 available at 
http://bitsavers.trailing-edge.com/pdf/mit/ai/AIM-628_chaosnet.pdf and 
probably other places). The Unix implementation used the rest of the 
path name to specify connection setup parameters in the typical case 
which seemed more unix-like than sockets. But the Chaosnet software was 
definitely swept away in the Ethernet/sockets storm surge.




^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-28 14:41             ` Clem Cole
@ 2022-06-28 15:54               ` Tom Teixeira
  0 siblings, 0 replies; 38+ messages in thread
From: Tom Teixeira @ 2022-06-28 15:54 UTC (permalink / raw)
  To: tuhs

[-- Attachment #1: Type: text/plain, Size: 1765 bytes --]

Christensen-style disruption sounds rather like Gresham's Law ("bad 
money drives out good"), but I don't think the mechanism is the same: 
one can hoard old silver coins and sell those at a profit for the silver 
content, but there's no premium I know of for better technology -- 
probably because "better technology" seems to imply aesthetics, but 
newer, "lessor technology" is likely to be much faster while using less 
energy.

On 6/28/22 10:41 AM, Clem Cole wrote:
>
>
> On Tue, Jun 28, 2022 at 9:15 AM Marc Donner <marc.donner@gmail.com> wrote:
>
>     My perception of the debate at the time was that it pitted
>     proprietary networking (SNA, DECNet, ...) against open networking
>     (TCP/IP).  The hardware vendors wanted proprietary networking to
>     lock customers into their equipment, but that dog would not hunt.
>
> Metcalfe's law: "/value of a network is proportional to the square of 
> the number of connected users of the system/."The problem with a 
> walled garden is that it can only grow as large as the walls allow.
>
>
>     It was pretty clear that except for the clever encapsulation stuff
>     that Vint had done with IP, the TCP/IP world was quick and dirty
>     and quite slapdash.  But it was non-proprietary and that is what
>     won the race.
>
> Point taken, but I actually think it is more of a 
> Christensen-style disruption where the 'lessor technology' outstrips 
> the more sophisticated one because it finds/creates a new market that 
> values that new technology for what it is and cares less about the 
> ways it may be 'lessor.'
>
> I described this in a talk I did at Asilomar a few years back.  This 
> is the most important slide:
> ColesLaw20190222.png
>
> ᐧ


[-- Attachment #2.1: Type: text/html, Size: 5850 bytes --]

[-- Attachment #2.2: ColesLaw20190222.png --]
[-- Type: image/png, Size: 252558 bytes --]

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
@ 2022-06-28 15:50 Noel Chiappa
  2022-06-28 21:32 ` Lawrence Stewart
  0 siblings, 1 reply; 38+ messages in thread
From: Noel Chiappa @ 2022-06-28 15:50 UTC (permalink / raw)
  To: tuhs; +Cc: jnc

    > From: Rob Pike

    > having the switch do some of the call validation and even maybe
    > authentication (I'm not sure...) sounds like it takes load off the host.

I don't have enough information to express a judgement in this particular
case, but I can say a few things about how one would go about analyzing
questions of 'where should I put function [X]; in the host, or in the
'network' (which almost inevitably means 'in the switches')'.

It seems to me that one has to examine three points:

- What is the 'cost' to actually _do_ the thing (which might be in
transmission usage, or computing power, or memory, or delay), in each
alternative; these costs obviously generally cannot be amortized across
multiple similar transactions.

- What is the 'cost' of providing the _mechanism_ to do the thing, in each
alternative. This comes in three parts. The first is the engineering cost of
_designing_ the thing, in detail; this obviously is amortized across muiple
instances. The second is _producing_ the mechanism, in the places where it is
needed (for mechanisms in software, this cost is essentially zero, unless it
needs a lot of memory/computes/etc); this is not amortized across many. The
third is harder to measure: it's complexity.

This is probably a book by itself, but it has costs that are hard to
quantify, and are also very disparate: e.g. more complex designs are more
likely to have unforseen bugs, which is very different from the 'cost' that
more complex designs are probaly harder to evolve for new uses.

So far I haven't said anything that isn't applicable across a broad range of
information sytems. The last influence on where one puts functions is much
more common in communication systems: the Saltzer/Clark/Reed 'End-to-end
Arguments in System Design' questions. If one _has_ to put a function in the
host to get 'acceptable' performace of that function, the
operation/implementation/design cost implications are irrelevant: one has to
grit one's teeth and bear them.

This may then feed back to design questions in the other areas. E.g. the
Version 2 ring at MIT deliberately left out hardware packet checksums -
because it was mostly intended for use with TCP/IP traffic, which provided a
pseudo-End-to-End checksum, so the per-unit hardware costs didn't buy enough
to be worth the costs of a hardware CRC. (Which was the right call; I don't
recall the lack of a hardware checksum ever causing a problem.)

And then there's the 'techology is a moving target' point: something that
might be unacceptably expensive (in computing cost) in year X might be fine
in year X+10, when we're lighting our cigars with unneeded computing power.
So when one is designing a communication system with a likely lifetime in
many decades, one tends to bias one's judgement toward things like End-to-End
analysis - because those factors will be forever.


Sorry if I haven't offered any answer to your initial query: "having the
switch do some of the call validation ... sounds like it takes load off the
host", but as I have tried to explain, these 'where should one do [X]'
questions are very complicated, and one would need a lot more detail before
one could give a good answer.

But, in general, "tak[ing] load off the host" doesn't seem to rate
highly as a goal these days... :-) :-(

	Noel

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-28 13:13           ` Marc Donner
@ 2022-06-28 14:41             ` Clem Cole
  2022-06-28 15:54               ` Tom Teixeira
  2022-06-28 17:05             ` Adam Thornton
  1 sibling, 1 reply; 38+ messages in thread
From: Clem Cole @ 2022-06-28 14:41 UTC (permalink / raw)
  To: Marc Donner; +Cc: The Eunuchs Hysterical Society


[-- Attachment #1.1: Type: text/plain, Size: 1209 bytes --]

On Tue, Jun 28, 2022 at 9:15 AM Marc Donner <marc.donner@gmail.com> wrote:

> My perception of the debate at the time was that it pitted proprietary
> networking (SNA, DECNet, ...) against open networking (TCP/IP).  The
> hardware vendors wanted proprietary networking to lock customers into their
> equipment, but that dog would not hunt.
>
Metcalfe's law: "*value of a network is proportional to the square of the
number of connected users of the system*." The problem with a walled garden
is that it can only grow as large as the walls allow.

>
> It was pretty clear that except for the clever encapsulation stuff that
> Vint had done with IP, the TCP/IP world was quick and dirty and quite
> slapdash.  But it was non-proprietary and that is what won the race.
>
Point taken, but I actually think it is more of a Christensen-style disruption
where the 'lessor technology' outstrips the more sophisticated one because
it finds/creates a new market that values that new technology for what it
is and cares less about the ways it may be 'lessor.'

I described this in a talk I did at Asilomar a few years back.  This is the
most important slide:
[image: ColesLaw20190222.png]

ᐧ

[-- Attachment #1.2: Type: text/html, Size: 3544 bytes --]

[-- Attachment #2: ColesLaw20190222.png --]
[-- Type: image/png, Size: 252558 bytes --]

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-28 12:45         ` Rob Pike
@ 2022-06-28 13:33           ` Dan Cross
  2022-06-28 21:19             ` Lawrence Stewart
  2022-06-28 16:11           ` Tom Teixeira
  2022-06-28 18:28           ` John Floren
  2 siblings, 1 reply; 38+ messages in thread
From: Dan Cross @ 2022-06-28 13:33 UTC (permalink / raw)
  To: Rob Pike; +Cc: The Eunuchs Hysterical Society

On Tue, Jun 28, 2022 at 8:46 AM Rob Pike <robpike@gmail.com> wrote:
> One of the reasons I'm not a networking expert may be relevant here.
> With networks, I never found an abstraction to hang my hat on.

Hmm, this raises some design questions.

> Unlike with file systems and files, or even Unix character
> devices, which provide a level of remove from the underlying
> blocks and sectors and so on, the Unix networking interface
> always seemed too low-level and fiddly, analogous to making
> users write files by managing the blocks and sectors themselves.

I can see this. Sockets in particular require filling in abstruse
and mostly opaque data structures and then passing pointers
to them into the kernel. It's not so terribly onerous once one
gets it going, particularly for a streaming protocol like TCP,
but eronomically it also doesn't sit particularly well in the larger
paradigm. Something like plan9's dial() seems more in line with
the Unix model than what Unix actually gives us in terms of
TLI/Sockets/whatever. I'll note that 10th Edition had its IPC
library that handled some of this.

> It could be all sockets' fault, but when I hear networking people talk
> about the protocols and stacks and routing and load shedding and
> ....my ears droop. I know it's amazing engineering and all that, but
> why aren't we allowed to program the I/O without all that fuss?
> What makes networks so _different_? A telling detail is that the
> original sockets interface had send and recv, not read and write.
> From day 1 in Unix land at least, networking was special, and it
> remains so, but I fail to see why it needs to be.

Of course, the semantics of networking are a little different than
the (mostly) stream-oriented file model of Unix, in that datagram
protocols must be accommodated somehow and they have metadata
in the form of sender/receiver information that accompanies each
IO request. How does one model that neatly in the read/write case,
except by prepending a header or having another argument?

But the same is true of streaming to/from files and file-like things
as well. I can't `seek` on a pipe or a serial device, for obvious
reasons, but that implies that that model is not completely regular.
Similarly, writes to, say, a raw disk device that are not a multiple
of the sector size have weird semantics. The best we have done
is document this and add it to the oral lore.

One may argue that the disk device thing is a special case that
is so uncomment and only relevant to extremely low-level systems
programs that it doesn't count, but the semantics of seeking are
universal: programs that want to work as Unix filters have to
accommodate this somehow. In practice this doesn't matter much;
most filters just don't seek on their input.

Once again I am in awe of how Unix got it right for 90% of use
cases, and makes the last 10% possible, even if painful.

> It just seems there has to be a better way. Sockets are just so
> unpleasant, and the endless nonsense around network
> configuration doubly so.

No argument there.

> Rhetorical questions. I'm not asking or wanting an answer.
> I'm happy to remain a greenhorn, oblivious to the wonder.

As we continue forward, I wonder how much this matters.  We
talk about sockets, but how many programmers _actually_ reach
for that interface when they want to talk over a network?  I'd
wager that _most_ of the time now days it's hidden behind a
library interface that shields the consumer from the gritty details.
Sure, that library probably uses sockets internally, but most
people probably never look under that rock.

> To adapt a reference some may recognize, I just want to read 5 terabytes.

Believe it or not, I actually had Borgmon readability.

        - Dan C.


> On Tue, Jun 28, 2022 at 10:36 PM Rob Pike <robpike@gmail.com> wrote:
>>
>> I am not a networking expert. I said that already. The issue could well be a property more of sockets than TCP/IP itself, but having the switch do some of the call validation and even maybe authentication (I'm not sure...) sounds like it takes load off the host.
>>
>> -rob
>>
>>
>> On Tue, Jun 28, 2022 at 8:39 PM Derek Fawcus <dfawcus+lists-tuhs@employees.org> wrote:
>>>
>>> On Sun, Jun 26, 2022 at 09:57:17AM +1000, Rob Pike wrote:
>>> > One of the things we liked about Datakit was that the computer didn't have
>>> > to establish the connection before it could reject the call, unlike TCP/IP
>>> > where all validation happens after the connection is made.
>>>
>>> Nor does TCP, one can send a RST to a SYN, and reject the call before it is
>>> established.  That would then look to the caller just like a non listening
>>> endpoint, unless one added data with the RST.
>>>
>>> So this is really just a consequence of the sockets API, and the current implementations.
>>> I've a vague recall of folks suggesting ways to expose that facility via the sockets
>>> layer, possibly using setsockopt(), but don't know if anyone ever did it.
>>>
>>> As I recall that TCP capability was actually exposed via the TLI/XTI API,
>>> and (for some STREAMS based TCP stacks) it did function. Although I may be
>>> thinking of embedded STREAMS TCP stacks, not unix based stacks.
>>>
>>> Or by 'connection' are you referring to an end-to-end packet delivery,
>>> and that Datakit allowed a closer switch to reject a call before the packet
>>> got to the far end?
>>>
>>> DF

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-28 12:47         ` Rich Morin
@ 2022-06-28 13:13           ` Marc Donner
  2022-06-28 14:41             ` Clem Cole
  2022-06-28 17:05             ` Adam Thornton
  0 siblings, 2 replies; 38+ messages in thread
From: Marc Donner @ 2022-06-28 13:13 UTC (permalink / raw)
  To: Rich Morin; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 2387 bytes --]

In the mid-1980s I returned to IBM Research after finishing up at CMU.  I
smuggled a bunch of Sun machines in and strung Ethernet between my office
and my lab so that the desktop could talk to the server.

Then I went around IBM giving talks about TCP/IP and why IBM should commit
to it.  At the time IBM Research was the center of development of IBM's SNA
stuff, so there was some (!) tension.  (Particularly because Paul Greene,
one of the key leaders of the SNA work, was very close to my undergraduate
mentor, so I had socialized with him.)  They proposed running TCP/IP
encapsulated in SNA, but I told them that the best they could expect was to
encapsulate SNA in TCP/IP.  That turned out to be what happened.

My perception of the debate at the time was that it pitted proprietary
networking (SNA, DECNet, ...) against open networking (TCP/IP).  The
hardware vendors wanted proprietary networking to lock customers into their
equipment, but that dog would not hunt.

Meanwhile, our community had recently discovered how horrible proprietary
tech was for our careers ... the mid-1980s recession led to serious layoffs
in the system programmer community and the newly unemployed geeks
discovered that the skills so assiduously honed were not portable.  Enter
FSK and the open source movement.

It was pretty clear that except for the clever encapsulation stuff that
Vint had done with IP, the TCP/IP world was quick and dirty and quite
slapdash.  But it was non-proprietary and that is what won the race.

What I don't understand is whether Rob's observation about networking is
*fundamental* to the space or *incidental* to the implementation.  I would
love to be educated on that.

Marc
=====
nygeek.net
mindthegapdialogs.com/home <https://www.mindthegapdialogs.com/home>


On Tue, Jun 28, 2022 at 8:48 AM Rich Morin <rdm@cfcl.com> wrote:

> > On Jun 28, 2022, at 05:36, Rob Pike <robpike@gmail.com> wrote:
> >
> > I am not a networking expert. I said that already. The issue could well
> be a property more of sockets than TCP/IP itself, but having the switch do
> some of the call validation and even maybe authentication (I'm not sure...)
> sounds like it takes load off the host.
>
> Some years ago, we set up a front end email server to reject incoming
> message attempts that didn't match our list of valid users.  This resulted
> in a better then 90% reduction.
>
> -r
>
>

[-- Attachment #2: Type: text/html, Size: 4147 bytes --]

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-28 12:36       ` Rob Pike
  2022-06-28 12:45         ` Rob Pike
@ 2022-06-28 12:47         ` Rich Morin
  2022-06-28 13:13           ` Marc Donner
  1 sibling, 1 reply; 38+ messages in thread
From: Rich Morin @ 2022-06-28 12:47 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

> On Jun 28, 2022, at 05:36, Rob Pike <robpike@gmail.com> wrote:
> 
> I am not a networking expert. I said that already. The issue could well be a property more of sockets than TCP/IP itself, but having the switch do some of the call validation and even maybe authentication (I'm not sure...) sounds like it takes load off the host.

Some years ago, we set up a front end email server to reject incoming message attempts that didn't match our list of valid users.  This resulted in a better then 90% reduction.

-r


^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-28 12:36       ` Rob Pike
@ 2022-06-28 12:45         ` Rob Pike
  2022-06-28 13:33           ` Dan Cross
                             ` (2 more replies)
  2022-06-28 12:47         ` Rich Morin
  1 sibling, 3 replies; 38+ messages in thread
From: Rob Pike @ 2022-06-28 12:45 UTC (permalink / raw)
  To: Derek Fawcus; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 2934 bytes --]

One of the reasons I'm not a networking expert may be relevant here. With
networks, I never found an abstraction to hang my hat on. Unlike with file
systems and files, or even Unix character devices, which provide a level of
remove from the underlying blocks and sectors and so on, the Unix
networking interface always seemed too low-level and fiddly, analogous to
making users write files by managing the blocks and sectors themselves. It
could be all sockets' fault, but when I hear networking people talk about
the protocols and stacks and routing and load shedding and ....my ears
droop. I know it's amazing engineering and all that, but why aren't we
allowed to program the I/O without all that fuss? What makes networks so
_different_? A telling detail is that the original sockets interface had
send and recv, not read and write. From day 1 in Unix land at least,
networking was special, and it remains so, but I fail to see why it needs
to be.

It just seems there has to be a better way. Sockets are just so unpleasant,
and the endless nonsense around network configuration doubly so.

Rhetorical questions. I'm not asking or wanting an answer. I'm happy to
remain a greenhorn, oblivious to the wonder.

To adapt a reference some may recognize, I just want to read 5 terabytes.

-rob


On Tue, Jun 28, 2022 at 10:36 PM Rob Pike <robpike@gmail.com> wrote:

> I am not a networking expert. I said that already. The issue could well be
> a property more of sockets than TCP/IP itself, but having the switch do
> some of the call validation and even maybe authentication (I'm not sure...)
> sounds like it takes load off the host.
>
> -rob
>
>
> On Tue, Jun 28, 2022 at 8:39 PM Derek Fawcus <
> dfawcus+lists-tuhs@employees.org> wrote:
>
>> On Sun, Jun 26, 2022 at 09:57:17AM +1000, Rob Pike wrote:
>> > One of the things we liked about Datakit was that the computer didn't
>> have
>> > to establish the connection before it could reject the call, unlike
>> TCP/IP
>> > where all validation happens after the connection is made.
>>
>> Nor does TCP, one can send a RST to a SYN, and reject the call before it
>> is
>> established.  That would then look to the caller just like a non listening
>> endpoint, unless one added data with the RST.
>>
>> So this is really just a consequence of the sockets API, and the current
>> implementations.
>> I've a vague recall of folks suggesting ways to expose that facility via
>> the sockets
>> layer, possibly using setsockopt(), but don't know if anyone ever did it.
>>
>> As I recall that TCP capability was actually exposed via the TLI/XTI API,
>> and (for some STREAMS based TCP stacks) it did function. Although I may be
>> thinking of embedded STREAMS TCP stacks, not unix based stacks.
>>
>> Or by 'connection' are you referring to an end-to-end packet delivery,
>> and that Datakit allowed a closer switch to reject a call before the
>> packet
>> got to the far end?
>>
>> DF
>>
>

[-- Attachment #2: Type: text/html, Size: 3745 bytes --]

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-28 10:38     ` Derek Fawcus
@ 2022-06-28 12:36       ` Rob Pike
  2022-06-28 12:45         ` Rob Pike
  2022-06-28 12:47         ` Rich Morin
  0 siblings, 2 replies; 38+ messages in thread
From: Rob Pike @ 2022-06-28 12:36 UTC (permalink / raw)
  To: Derek Fawcus; +Cc: The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 1516 bytes --]

I am not a networking expert. I said that already. The issue could well be
a property more of sockets than TCP/IP itself, but having the switch do
some of the call validation and even maybe authentication (I'm not sure...)
sounds like it takes load off the host.

-rob


On Tue, Jun 28, 2022 at 8:39 PM Derek Fawcus <
dfawcus+lists-tuhs@employees.org> wrote:

> On Sun, Jun 26, 2022 at 09:57:17AM +1000, Rob Pike wrote:
> > One of the things we liked about Datakit was that the computer didn't
> have
> > to establish the connection before it could reject the call, unlike
> TCP/IP
> > where all validation happens after the connection is made.
>
> Nor does TCP, one can send a RST to a SYN, and reject the call before it is
> established.  That would then look to the caller just like a non listening
> endpoint, unless one added data with the RST.
>
> So this is really just a consequence of the sockets API, and the current
> implementations.
> I've a vague recall of folks suggesting ways to expose that facility via
> the sockets
> layer, possibly using setsockopt(), but don't know if anyone ever did it.
>
> As I recall that TCP capability was actually exposed via the TLI/XTI API,
> and (for some STREAMS based TCP stacks) it did function. Although I may be
> thinking of embedded STREAMS TCP stacks, not unix based stacks.
>
> Or by 'connection' are you referring to an end-to-end packet delivery,
> and that Datakit allowed a closer switch to reject a call before the packet
> got to the far end?
>
> DF
>

[-- Attachment #2: Type: text/html, Size: 1942 bytes --]

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-25 23:57   ` Rob Pike
  2022-06-26  1:17     ` Paul Ruizendaal
@ 2022-06-28 10:38     ` Derek Fawcus
  2022-06-28 12:36       ` Rob Pike
  1 sibling, 1 reply; 38+ messages in thread
From: Derek Fawcus @ 2022-06-28 10:38 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On Sun, Jun 26, 2022 at 09:57:17AM +1000, Rob Pike wrote:
> One of the things we liked about Datakit was that the computer didn't have
> to establish the connection before it could reject the call, unlike TCP/IP
> where all validation happens after the connection is made.

Nor does TCP, one can send a RST to a SYN, and reject the call before it is
established.  That would then look to the caller just like a non listening
endpoint, unless one added data with the RST.

So this is really just a consequence of the sockets API, and the current implementations.
I've a vague recall of folks suggesting ways to expose that facility via the sockets
layer, possibly using setsockopt(), but don't know if anyone ever did it.

As I recall that TCP capability was actually exposed via the TLI/XTI API,
and (for some STREAMS based TCP stacks) it did function. Although I may be
thinking of embedded STREAMS TCP stacks, not unix based stacks.

Or by 'connection' are you referring to an end-to-end packet delivery,
and that Datakit allowed a closer switch to reject a call before the packet
got to the far end?

DF

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-27 21:40 Noel Chiappa
@ 2022-06-27 22:40 ` George Michaelson
  0 siblings, 0 replies; 38+ messages in thread
From: George Michaelson @ 2022-06-27 22:40 UTC (permalink / raw)
  To: Noel Chiappa; +Cc: tuhs

I did an analysis of the DFZ with Emile Aben at RIPE. There may be a
million now, but at least half of these are TE and functionally
irrelevant to 90%+ of the rest of the BGP speakers, being aimed at
immediate peers only.

If we renumbered, the count of real announcements becomes very much
smaller, close to the count of ASN, modulo some necessary
unaggegatable outcomes.

Geoff has done work on this too, the ratio between noisy speakers and
the stable speakers appear to be constants modulo natural growth.
(Geoff says hello btw)

-G

On Tue, Jun 28, 2022 at 7:41 AM Noel Chiappa <jnc@mercury.lcs.mit.edu> wrote:
>
>     > From: Paul Ruizendaal
>
>     > Will read those RFC's, though -- thank you for pointing them out.
>
> Oh, I wouldn't bother - unless you are really into routing (i.e. path
> selection).
>
> RFC-1992 in particular; it's got my name on it, but it was mostly written by
> Martha and Isidro, and I'm not entirely happy with it. E.g. CSC mode and CSS
> mode (roughly, strict source route and loose source route); I wasn't really
> sold on them, but I was too tired to argue about it. Nimrod was complicated
> enough without adding extra bells and whistles - and indeed, LSR and SSR are
> basically unused to this day in the Internet (at least, at the internet
> layer; MPLS does provide the ability to specify paths, which I gather is used
> to some degree). I guess it's an OK overview of the architecture, though.
>
> RFC-1753 is not the best overview, but it has interesting bits. E.g. 2.2
> Packet Format Fields, Option 2: "The packet contains a stack of flow-ids,
> with the current one on the top." If this reminds you of MPLS, it should!
> (One can think of MPLS as Nimrod's packet-carrying subsystem, in some ways.)
>
> I guess I should mention that Nimrod covers more stuff - a lot more - than
> just path selection. That's because I felt that the architecture embodied in
> IPv4 was missing lots of things which one would need to do the internet layer
> 'right' in a global-scale Internet (e.g. variable length 'addresses' - for
> which we were forced to invent the term 'locator' because many nitwits in the
> IETF couldn't wrap their minds around 'addresses' which weren't in every
> packet header). And separation of location and identity; and the introduction
> of traffic aggregates as first-class objects at the internet layer. Etc, etc,
> etc.
>
> Nimrod's main focus was really on i) providing a path-selection system which
> allowed things like letting users have more input to selecting the path their
> traffic took (just as when one gets into a car, one gets to pick the path
> one's going to use), and ii) controlling the overhead of the routing.
>
> Of course, on the latter point, in the real world, people just threw
> resources (memory, computing power, bandwidth) at the problem. I'm kind of
> blown away< that there are almost 1 million routes in the DFZ these days.
> Boiling frogs...
>
>         Noel

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
@ 2022-06-27 21:40 Noel Chiappa
  2022-06-27 22:40 ` George Michaelson
  0 siblings, 1 reply; 38+ messages in thread
From: Noel Chiappa @ 2022-06-27 21:40 UTC (permalink / raw)
  To: tuhs; +Cc: jnc

    > From: Paul Ruizendaal

    > Will read those RFC's, though -- thank you for pointing them out.

Oh, I wouldn't bother - unless you are really into routing (i.e. path
selection).

RFC-1992 in particular; it's got my name on it, but it was mostly written by
Martha and Isidro, and I'm not entirely happy with it. E.g. CSC mode and CSS
mode (roughly, strict source route and loose source route); I wasn't really
sold on them, but I was too tired to argue about it. Nimrod was complicated
enough without adding extra bells and whistles - and indeed, LSR and SSR are
basically unused to this day in the Internet (at least, at the internet
layer; MPLS does provide the ability to specify paths, which I gather is used
to some degree). I guess it's an OK overview of the architecture, though.

RFC-1753 is not the best overview, but it has interesting bits. E.g. 2.2
Packet Format Fields, Option 2: "The packet contains a stack of flow-ids,
with the current one on the top." If this reminds you of MPLS, it should!
(One can think of MPLS as Nimrod's packet-carrying subsystem, in some ways.)

I guess I should mention that Nimrod covers more stuff - a lot more - than
just path selection. That's because I felt that the architecture embodied in
IPv4 was missing lots of things which one would need to do the internet layer
'right' in a global-scale Internet (e.g. variable length 'addresses' - for
which we were forced to invent the term 'locator' because many nitwits in the
IETF couldn't wrap their minds around 'addresses' which weren't in every
packet header). And separation of location and identity; and the introduction
of traffic aggregates as first-class objects at the internet layer. Etc, etc,
etc.

Nimrod's main focus was really on i) providing a path-selection system which
allowed things like letting users have more input to selecting the path their
traffic took (just as when one gets into a car, one gets to pick the path
one's going to use), and ii) controlling the overhead of the routing.

Of course, on the latter point, in the real world, people just threw
resources (memory, computing power, bandwidth) at the problem. I'm kind of
blown away< that there are almost 1 million routes in the DFZ these days.
Boiling frogs...

	Noel

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-27  0:43 Noel Chiappa
@ 2022-06-27  3:00 ` Erik Fair
  0 siblings, 0 replies; 38+ messages in thread
From: Erik Fair @ 2022-06-27  3:00 UTC (permalink / raw)
  To: Noel Chiappa; +Cc: tuhs

[-- Attachment #1: Type: text/plain, Size: 789 bytes --]


> On Jun 26, 2022, at 17:43, Noel Chiappa <jnc@mercury.lcs.mit.edu> wrote:
> 
> Interfaces are forever; like the screw in light-bulb. These days, it's likely
> an LED bulb on one side, powered by a reactor on the other - two technologies
> which were unforseen (and unforseeable) when the interface was defined, well
> over 100 years ago.

To be specific: the “Edison screw” (or “Edison medium base” or now E26) patented by Thomas Edison in 1881, https://en.wikipedia.org/wiki/Edison_screw <https://en.wikipedia.org/wiki/Edison_screw>

It’s been said that “hardware comes & goes, but software is forever” but I like to add “software comes & goes, but protocols are forever.”

We should take care in how we design/define protocols and interfaces.

	Erik


[-- Attachment #2: Type: text/html, Size: 1522 bytes --]

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
@ 2022-06-27  0:43 Noel Chiappa
  2022-06-27  3:00 ` Erik Fair
  0 siblings, 1 reply; 38+ messages in thread
From: Noel Chiappa @ 2022-06-27  0:43 UTC (permalink / raw)
  To: tuhs; +Cc: jnc

Just as the topic of TUHS isn't 'how _I_ could/would build a _better_ OS', but
'history of the OS that was _actually built_' (something that many posters
here seem to lose track of, to the my great irritation), so too the topic
isn't 'how to build a better network' - or actually, anything network-centric.

I'll make a few comments on a couple of things, though.


    > From: steve jenkin

    > packet switching won over Virtual Circuits in the now distant past but
    > in small, local and un-congested networks without reliability
    > constraints, any solution can look good. ... Packet switching
    > hasn't scaled well to Global size, at least IMHO.

The internetworking architecture, circa 1978, has not scaled as well as would
have been optimal, for a number of reasons, among them:

- pure scaling effects (e.g. algorithms won't scale up; subsystems which
  handle several different needs will often need to be separated out at a larger
  scale; etc)

- inherent lack of hindsight (unknown unknowns, to use Rumsfeld's phrase; some
  things you only learn in hindsight)

- insufficiently detailed knowledge of complete requirements for a
  global-scale network (including O+M, eventual business model, etc)

- limited personnel resources at the time (some things we _knew_ were going to
  be a problem we had to ignore because we didn't have people to throw at the
  problem, then and there)

- rapid technological innovation (and nobody's crystal ball is 100% perfect)

It has been possible to fix some aspects of the ca. 1978 system - e.g. the
addition of DNS, which I think has worked _reasonably_ well - but in other
areas, changes weren't really adequate, often because they were constrained by
things like upward compatibility requirements (e.g. BGP, which, among numerous
other issues, had to live with existing IP addressing).

Having said all that, I think your assertion that virtual circuits would have
worked better in a global-scale network is questionable. The whole point of
networks which use unreliable datagrams as a fundamental building block is
that by moving a lot of functionality into the edge nodes, it makes the
switches a lot simpler. Contemporary core routers may be complex - but they
would be much worse if the network used virtual circuits.

Something I suspect you may be unaware of is that most of the people who
devised the unreliable datagram approach of the internetworking architecture
_had experience with an actual moderately-sized, operational virtual circuit
network_ - the ARPANET. (Yes, it was basically a VC network. Look at things
like RFNMs, links {the specific ARPANET mechanism referred to by this term,
not the general concept}, etc.) So they _knew_ what a VC network would
involve.

So, think about the 'core routers' in a network which used VC's. I guess a
typical core router tese days uses a couple of OC768 links. Assume an average
packet size of 100 bytes (probably roughly accurate, with the bimodal
distribution between data and acks). With 4 OC768's, that's 4*38.5G/800 =
~155M packets/second. I'm not sure of the average TCP connection length in
packets these days, but assume it's 100 packets or so (that's a 100KB Web
object). That's still roughly _1 million cicuit setups per second_.

If the answer is 'oh, we'll use aggregation so core routers don't see
individual connections - or their setup/tear-down' - well, the same
can be done with a datagram system; that's what MPLS does. Work
through the details - VCs were not preferred, for good reasons.


    > Ethernet only became a viable LAN technology with advent of Twisted
    > pair: point to point + Switches.

It's really irritating that a lot of things labelled 'Ethernet' these days
_aren't_ _real_ Ethernet (i.e. a common broadcast bus allocated via CSMA-CD).

They use the same _packet format_ as Ethernet (especially the 48-bit
globally-unique address, which can usefully be blown into things at
manufacture time), but it's not Ethernet. In some cases, they also retain the
host interface<->network physical interface - but the thing on the other side
of the interface is totally different (such as the hub-based systems commmon
now - as you indicate, it's a bunch of small datagram packet switches plugged
together with point-point links).

Interfaces are forever; like the screw in light-bulb. These days, it's likely
an LED bulb on one side, powered by a reactor on the other - two technologies
which were unforseen (and unforseeable) when the interface was defined, well
over 100 years ago.

	Noel

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-26  1:41 ` Anthony Martin
  2022-06-26  9:52   ` Ralph Corderoy
@ 2022-06-26 11:04   ` Paul Ruizendaal
  2022-06-29 20:21   ` Paul Ruizendaal
  2 siblings, 0 replies; 38+ messages in thread
From: Paul Ruizendaal @ 2022-06-26 11:04 UTC (permalink / raw)
  To: Anthony Martin; +Cc: tuhs


> On 26 Jun 2022, at 03:41, Anthony Martin <ality@pbrane.org> wrote:
> 
> Really nice outline. I felt some déjà vu when opening the document: I
> remember reading your notes in April of 2020.

Thank you. Yes, I have not updated these notes since then.

> Would you happen to know where I can find copies of these three
> papers?
> 
> A. G. Fraser, "Datakit - A Modular Network for Synchronous and
> Asynchronous Traffic", Proc. ICC 79, June 1979, Boston, Ma.,
> pp.20.1.1-20.1.3
> 
> G. L. Chesson, "Datakit Software Architecture", Proc. ICC 79, June
> 1979, Boston Ma., pp.20.2.1-20.2.5
> 
> G. L. Chesson and A. G. Fraser, "Datakit Network Architecture," Proc.
> Compcon 80, February 1980, San Fransisco CA., pp.59-61

Two of these three were in Sandy Fraser’s archives:

https://www.jslite.net/notes/dk1.pdf

https://www.jslite.net/notes/dk2.pdf

The middle one is I fear lost - unless a paper copy of the proceedings is still lurking in some university library. I think it will be mostly about the “CMC” switch control software, which is somewhat described in the Luderer paper as well. As far as I can tell, there is no surviving Unix code that was designed for the CMC version of Datakit (unless someone is sitting on a V7 tape with the Datakit drivers still included).

> To connect to one of the Plan 9 cpu servers at Bell Labs, you would
> dial "nj/astro/helix!9fs”.

Out of interest, would you know if the switch-host interface for Datakit in the days of Plan9 worked more or less the same as in 8th Edition? Or had it much evolved by then?








^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-26  1:41 ` Anthony Martin
@ 2022-06-26  9:52   ` Ralph Corderoy
  2022-06-26 11:04   ` Paul Ruizendaal
  2022-06-29 20:21   ` Paul Ruizendaal
  2 siblings, 0 replies; 38+ messages in thread
From: Ralph Corderoy @ 2022-06-26  9:52 UTC (permalink / raw)
  To: Anthony Martin; +Cc: tuhs

Hi Anthony,

> Would you happen to know where I can find copies of these three
> papers?
>
> A. G. Fraser, "Datakit - A Modular Network for Synchronous and
> Asynchronous Traffic", Proc. ICC 79, June 1979, Boston, Ma.,
> pp.20.1.1-20.1.3
>
> G. L. Chesson, "Datakit Software Architecture", Proc. ICC 79, June
> 1979, Boston Ma., pp.20.2.1-20.2.5
>
> G. L. Chesson and A. G. Fraser, "Datakit Network Architecture," Proc.
> Compcon 80, February 1980, San Fransisco CA., pp.59-61

I had no luck looking for those.  I did find other Datakit ones
which may interest the list; all Bell Labs.

- A Virtual Circuit Switch as the Basis for Distributed Systems.
  Luderer, Che, Marshall, Bell Labs, Murray Hill.  ~1981.
  ‘...we have implemented... a new distributed operating system derived
  from Unix’.
  https://dl.acm.org/doi/pdf/10.1145/1013879.802670

- Methods for routing, performance management, and service problem
  relief in Datakit networks.
  Fendick, Harshavardhana, Jidarian, Bell Labs, Holmdel.  1991.
  https://libgen.rocks/get.php?md5=27c34fe1fce3ed20b79aaf65362cb5f9&key=VXFE8W3J24GAYURB&doi=10.1109/icc.1991.162268

- FIBERKIT: a Datakit compatible 500 megabit virtual circuit switch.
  Follett, Levy, Sobin, Tourgee, AT&T Bell Labs.  1988.
  https://libgen.rocks/get.php?md5=577344ef77634022868f52969dbda62b&key=COCJ9PQZSNIJ2U3B&doi=10.1109/glocom.1988.25918

-- 
Cheers, Ralph.

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-25 23:01 [TUHS] " Paul Ruizendaal
  2022-06-25 23:09 ` [TUHS] " Larry McVoy
@ 2022-06-26  1:41 ` Anthony Martin
  2022-06-26  9:52   ` Ralph Corderoy
                     ` (2 more replies)
  1 sibling, 3 replies; 38+ messages in thread
From: Anthony Martin @ 2022-06-26  1:41 UTC (permalink / raw)
  To: Paul Ruizendaal; +Cc: tuhs

Paul Ruizendaal <pnr@planet.nl> once said:
> Wanted to post my notes as plain text, but the bullets / sub-bullets get lost.
>
> Here is a 2 page PDF with my notes on Research Datakit:
>
> https://www.jslite.net/notes/rdk.pdf

Really nice outline. I felt some déjà vu when opening the document: I
remember reading your notes in April of 2020.

Would you happen to know where I can find copies of these three
papers?

A. G. Fraser, "Datakit - A Modular Network for Synchronous and
Asynchronous Traffic", Proc. ICC 79, June 1979, Boston, Ma.,
pp.20.1.1-20.1.3

G. L. Chesson, "Datakit Software Architecture", Proc. ICC 79, June
1979, Boston Ma., pp.20.2.1-20.2.5

G. L. Chesson and A. G. Fraser, "Datakit Network Architecture," Proc.
Compcon 80, February 1980, San Fransisco CA., pp.59-61

> A secondary takeaway is that Research Datakit eventually settled on a
> three-level ascii namespace: “area/trunk/switch”. On each switch, the
> hosts would be known by name, and each connection request had a
> service name as parameter. In an alternate reality we would maybe have
> used “ca/stclara/mtnview!google!www” to do a search.

To connect to one of the Plan 9 cpu servers at Bell Labs, you would
dial "nj/astro/helix!9fs". I do wonder how the relative hierarchical
naming would have evolved to encompass the entire Internet. Would
it have been more like "com/google/search!http"? Who knows? ☺

Thanks,
  Anthony

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-25 23:57   ` Rob Pike
@ 2022-06-26  1:17     ` Paul Ruizendaal
  2022-07-02  2:51       ` Grant Taylor via TUHS
  2022-06-28 10:38     ` Derek Fawcus
  1 sibling, 1 reply; 38+ messages in thread
From: Paul Ruizendaal @ 2022-06-26  1:17 UTC (permalink / raw)
  To: Rob Pike; +Cc: The Eunuchs Hysterical Society


> On 26 Jun 2022, at 01:57, Rob Pike <robpike@gmail.com> wrote:
> 
> One of the things we liked about Datakit was that the computer didn't have to establish the connection before it could reject the call, unlike TCP/IP where all validation happens after the connection is made. This is also why sockets and Datakit never worked together; sockets pretty much assume Ethernet-like connection rules.
> 
> I am not a networking expert, but to me in this regard at least Datakit seemed like a prettier picture. I suppose you can DOS-attack the network, but not the machines. Datakit had other issues, for sure, like the expensive racks of hardware, but then that's because, for better and worse, it was designed by phone engineers rather than.... however you'd characterize Ethernet and its original "I scream while listening to your whisper", 5V into 50Ω Schmidt-triggered craziness. Ethernet's come a long way, but the engineering of the original Radio Shack parts was not favored by the Bell Labs crowd.

I was not putting Datakit down, just trying to explain why the V8 approach to networking may seem a little odd from a 1980’s TCP/IP perspective, but makes perfect sense from a Datakit perspective.

In the end technology often becomes a hybrid of various solutions, and maybe in this case as well. By coincidence there was a post in the Internet History mailing list earlier today that appears to make this point.

In his video (https://www.youtube.com/watch?v=ojRtJ1U6Qzw), Sandy explains why he became dissatisfied with Spider and the main reason was that doing switching/routing on a mini computer was just plain inefficient as compared to a telephone switch (at 37:06). This was 1972. The result was a new design, Datakit, that could route/switch packets at high speed and in parallel.

On the internet history list, someone quipped: "Yeah, back then the joke was that McQuillan was the only one making money from ATM. :-) That did change in a big way (for a while) in the late 90s and early 2000s, before router silicon caught up."

To this Craig Partridge responded: "Wasn't just router silicon -- it was router design.  What made ATM appealing is that it made the inside of the router or switch parallel, which was necessary to push into multigigabit rates. Folks had to figure out how to rework an Internet router to be parallel and it took at least two major innovations: fully-standalone forwarding tables with associating forwarding engines and breaking packets apart (essentially into cells), squirting those parts through the parallel backplane, and then reassembling the packet at the outbound interface for transmission." This was around 2000.

It is not my field of expertise, but it would seem to me that Sandy had figured out a core problem some 30 years before the TCP/IP world would come up with a similar solution. I would not even be surprised if I learned that modern telco routers transparantly set up virtual circuits for tcp traffic.


^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-25 23:09 ` [TUHS] " Larry McVoy
@ 2022-06-25 23:57   ` Rob Pike
  2022-06-26  1:17     ` Paul Ruizendaal
  2022-06-28 10:38     ` Derek Fawcus
  0 siblings, 2 replies; 38+ messages in thread
From: Rob Pike @ 2022-06-25 23:57 UTC (permalink / raw)
  To: Larry McVoy; +Cc: Paul Ruizendaal, The Eunuchs Hysterical Society

[-- Attachment #1: Type: text/plain, Size: 3104 bytes --]

One of the things we liked about Datakit was that the computer didn't have
to establish the connection before it could reject the call, unlike TCP/IP
where all validation happens after the connection is made. This is also why
sockets and Datakit never worked together; sockets pretty much assume
Ethernet-like connection rules.

I am not a networking expert, but to me in this regard at least Datakit
seemed like a prettier picture. I suppose you can DOS-attack the network,
but not the machines. Datakit had other issues, for sure, like the
expensive racks of hardware, but then that's because, for better and worse,
it was designed by phone engineers rather than.... however you'd
characterize Ethernet and its original "I scream while listening to your
whisper", 5V into 50Ω Schmidt-triggered craziness. Ethernet's come a long
way, but the engineering of the original Radio Shack parts was not favored
by the Bell Labs crowd.

-rob


On Sun, Jun 26, 2022 at 9:09 AM Larry McVoy <lm@mcvoy.com> wrote:

> Nice.  Any chance you want to do a TCP/XTP comparison?
>
> On Sun, Jun 26, 2022 at 01:01:07AM +0200, Paul Ruizendaal wrote:
> > Wanted to post my notes as plain text, but the bullets / sub-bullets get
> lost.
> >
> > Here is a 2 page PDF with my notes on Research Datakit:
> >
> > https://www.jslite.net/notes/rdk.pdf
> >
> > The main takeaway is that connection build-up and tear-down is
> considerably more expensive than with TCP. The first cost is in the
> network, which builds up a dedicated path for each connection. Bandwidth is
> not allocated/reserved, but a path is and routing information is set up at
> each hop. The other cost is in the relatively verbose switch-host
> communication in this phase. This compares to the 3 packets exchanged at
> the hosts??? driver level to set up a TCP connection, with no permanent
> resources consumed in the network.
> >
> > In compensation, the cost to use a connection is considerably lower: the
> routing is known and the host-host link protocol (???URP") can be
> light-weight, as the network guarantees in-order delivery without
> duplicates but packets may be corrupted or lost (i.e. as if the connection
> is a phone line with a modem). No need to deal with packet fragmentation,
> stream reassembly and congestion storms as in the TCP of the early 80???s.
> >
> > Doing UDP traffic to a fixed remote host is easily mapped to using URP
> with no error correction and no flow control. Doing UDP where the remote
> host is different all the time is not practical on a Datakit network (i.e.
> a virtual circuit would be set up anyway).
> >
> > A secondary takeaway is that Research Datakit eventually settled on a
> three-level ascii namespace: ???area/trunk/switch???. On each switch, the
> hosts would be known by name, and each connection request had a service
> name as parameter. In an alternate reality we would maybe have used
> ???ca/stclara/mtnview!google!www??? to do a search.
>
> --
> ---
> Larry McVoy            Retired to fishing
> http://www.mcvoy.com/lm/boat
>

[-- Attachment #2: Type: text/html, Size: 4143 bytes --]

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [TUHS] Re: Research Datakit notes
  2022-06-25 23:01 [TUHS] " Paul Ruizendaal
@ 2022-06-25 23:09 ` Larry McVoy
  2022-06-25 23:57   ` Rob Pike
  2022-06-26  1:41 ` Anthony Martin
  1 sibling, 1 reply; 38+ messages in thread
From: Larry McVoy @ 2022-06-25 23:09 UTC (permalink / raw)
  To: Paul Ruizendaal; +Cc: tuhs

Nice.  Any chance you want to do a TCP/XTP comparison?

On Sun, Jun 26, 2022 at 01:01:07AM +0200, Paul Ruizendaal wrote:
> Wanted to post my notes as plain text, but the bullets / sub-bullets get lost.
> 
> Here is a 2 page PDF with my notes on Research Datakit:
> 
> https://www.jslite.net/notes/rdk.pdf
> 
> The main takeaway is that connection build-up and tear-down is considerably more expensive than with TCP. The first cost is in the network, which builds up a dedicated path for each connection. Bandwidth is not allocated/reserved, but a path is and routing information is set up at each hop. The other cost is in the relatively verbose switch-host communication in this phase. This compares to the 3 packets exchanged at the hosts??? driver level to set up a TCP connection, with no permanent resources consumed in the network.
> 
> In compensation, the cost to use a connection is considerably lower: the routing is known and the host-host link protocol (???URP") can be light-weight, as the network guarantees in-order delivery without duplicates but packets may be corrupted or lost (i.e. as if the connection is a phone line with a modem). No need to deal with packet fragmentation, stream reassembly and congestion storms as in the TCP of the early 80???s.
> 
> Doing UDP traffic to a fixed remote host is easily mapped to using URP with no error correction and no flow control. Doing UDP where the remote host is different all the time is not practical on a Datakit network (i.e. a virtual circuit would be set up anyway).
> 
> A secondary takeaway is that Research Datakit eventually settled on a three-level ascii namespace: ???area/trunk/switch???. On each switch, the hosts would be known by name, and each connection request had a service name as parameter. In an alternate reality we would maybe have used ???ca/stclara/mtnview!google!www??? to do a search.

-- 
---
Larry McVoy            Retired to fishing           http://www.mcvoy.com/lm/boat

^ permalink raw reply	[flat|nested] 38+ messages in thread

end of thread, other threads:[~2022-07-02  2:57 UTC | newest]

Thread overview: 38+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-06-26  2:19 [TUHS] Re: Research Datakit notes Noel Chiappa
2022-06-26  9:46 ` steve jenkin
2022-06-26 20:35   ` Erik Fair
2022-06-26 21:53     ` Steve Jenkin
2022-06-26 10:16 ` Paul Ruizendaal via TUHS
2022-06-26 13:07 ` John Cowan
2022-06-26 13:35   ` Larry McVoy
2022-06-26 13:58     ` John Cowan
  -- strict thread matches above, loose matches on Subject: below --
2022-06-28 15:50 Noel Chiappa
2022-06-28 21:32 ` Lawrence Stewart
2022-06-27 21:40 Noel Chiappa
2022-06-27 22:40 ` George Michaelson
2022-06-27  0:43 Noel Chiappa
2022-06-27  3:00 ` Erik Fair
2022-06-25 23:01 [TUHS] " Paul Ruizendaal
2022-06-25 23:09 ` [TUHS] " Larry McVoy
2022-06-25 23:57   ` Rob Pike
2022-06-26  1:17     ` Paul Ruizendaal
2022-07-02  2:51       ` Grant Taylor via TUHS
2022-07-02  2:57         ` Larry McVoy
2022-06-28 10:38     ` Derek Fawcus
2022-06-28 12:36       ` Rob Pike
2022-06-28 12:45         ` Rob Pike
2022-06-28 13:33           ` Dan Cross
2022-06-28 21:19             ` Lawrence Stewart
2022-06-28 21:34               ` Richard Salz
2022-06-29  6:07                 ` Stuart Remphrey
2022-06-28 16:11           ` Tom Teixeira
2022-06-28 18:28           ` John Floren
2022-06-28 12:47         ` Rich Morin
2022-06-28 13:13           ` Marc Donner
2022-06-28 14:41             ` Clem Cole
2022-06-28 15:54               ` Tom Teixeira
2022-06-28 17:05             ` Adam Thornton
2022-06-28 17:43               ` John Labovitz
2022-06-26  1:41 ` Anthony Martin
2022-06-26  9:52   ` Ralph Corderoy
2022-06-26 11:04   ` Paul Ruizendaal
2022-06-29 20:21   ` Paul Ruizendaal

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).