[TUHS] Research Datakit notes

[TUHS] Research Datakit notes

[Paul Ruizendaal]

Wanted to post my notes as plain text, but the bullets / sub-bullets get lost.

Here is a 2 page PDF with my notes on Research Datakit:

https://www.jslite.net/notes/rdk.pdf

The main takeaway is that connection build-up and tear-down is considerably more expensive than with TCP. The first cost is in the network, which builds up a dedicated path for each connection. Bandwidth is not allocated/reserved, but a path is and routing information is set up at each hop. The other cost is in the relatively verbose switch-host communication in this phase. This compares to the 3 packets exchanged at the hosts’ driver level to set up a TCP connection, with no permanent resources consumed in the network.

In compensation, the cost to use a connection is considerably lower: the routing is known and the host-host link protocol (“URP") can be light-weight, as the network guarantees in-order delivery without duplicates but packets may be corrupted or lost (i.e. as if the connection is a phone line with a modem). No need to deal with packet fragmentation, stream reassembly and congestion storms as in the TCP of the early 80’s.

Doing UDP traffic to a fixed remote host is easily mapped to using URP with no error correction and no flow control. Doing UDP where the remote host is different all the time is not practical on a Datakit network (i.e. a virtual circuit would be set up anyway).

A secondary takeaway is that Research Datakit eventually settled on a three-level ascii namespace: “area/trunk/switch”. On each switch, the hosts would be known by name, and each connection request had a service name as parameter. In an alternate reality we would maybe have used “ca/stclara/mtnview!google!www” to do a search.

[TUHS] Re: Research Datakit notes

[Larry McVoy]

Nice.  Any chance you want to do a TCP/XTP comparison?

On Sun, Jun 26, 2022 at 01:01:07AM +0200, Paul Ruizendaal wrote:
> Wanted to post my notes as plain text, but the bullets / sub-bullets get lost.
> 
> Here is a 2 page PDF with my notes on Research Datakit:
> 
> https://www.jslite.net/notes/rdk.pdf
> 
> The main takeaway is that connection build-up and tear-down is considerably more expensive than with TCP. The first cost is in the network, which builds up a dedicated path for each connection. Bandwidth is not allocated/reserved, but a path is and routing information is set up at each hop. The other cost is in the relatively verbose switch-host communication in this phase. This compares to the 3 packets exchanged at the hosts??? driver level to set up a TCP connection, with no permanent resources consumed in the network.
> 
> In compensation, the cost to use a connection is considerably lower: the routing is known and the host-host link protocol (???URP") can be light-weight, as the network guarantees in-order delivery without duplicates but packets may be corrupted or lost (i.e. as if the connection is a phone line with a modem). No need to deal with packet fragmentation, stream reassembly and congestion storms as in the TCP of the early 80???s.
> 
> Doing UDP traffic to a fixed remote host is easily mapped to using URP with no error correction and no flow control. Doing UDP where the remote host is different all the time is not practical on a Datakit network (i.e. a virtual circuit would be set up anyway).
> 
> A secondary takeaway is that Research Datakit eventually settled on a three-level ascii namespace: ???area/trunk/switch???. On each switch, the hosts would be known by name, and each connection request had a service name as parameter. In an alternate reality we would maybe have used ???ca/stclara/mtnview!google!www??? to do a search.

-- 
---
Larry McVoy            Retired to fishing           http://www.mcvoy.com/lm/boat

[TUHS] Re: Research Datakit notes

[Rob Pike]

[-- Attachment #1: Type: text/plain, Size: 3104 bytes --]

One of the things we liked about Datakit was that the computer didn't have
to establish the connection before it could reject the call, unlike TCP/IP
where all validation happens after the connection is made. This is also why
sockets and Datakit never worked together; sockets pretty much assume
Ethernet-like connection rules.

I am not a networking expert, but to me in this regard at least Datakit
seemed like a prettier picture. I suppose you can DOS-attack the network,
but not the machines. Datakit had other issues, for sure, like the
expensive racks of hardware, but then that's because, for better and worse,
it was designed by phone engineers rather than.... however you'd
characterize Ethernet and its original "I scream while listening to your
whisper", 5V into 50Ω Schmidt-triggered craziness. Ethernet's come a long
way, but the engineering of the original Radio Shack parts was not favored
by the Bell Labs crowd.

-rob


On Sun, Jun 26, 2022 at 9:09 AM Larry McVoy <lm@mcvoy.com> wrote:

> Nice.  Any chance you want to do a TCP/XTP comparison?
>
> On Sun, Jun 26, 2022 at 01:01:07AM +0200, Paul Ruizendaal wrote:
> > Wanted to post my notes as plain text, but the bullets / sub-bullets get
> lost.
> >
> > Here is a 2 page PDF with my notes on Research Datakit:
> >
> > https://www.jslite.net/notes/rdk.pdf
> >
> > The main takeaway is that connection build-up and tear-down is
> considerably more expensive than with TCP. The first cost is in the
> network, which builds up a dedicated path for each connection. Bandwidth is
> not allocated/reserved, but a path is and routing information is set up at
> each hop. The other cost is in the relatively verbose switch-host
> communication in this phase. This compares to the 3 packets exchanged at
> the hosts??? driver level to set up a TCP connection, with no permanent
> resources consumed in the network.
> >
> > In compensation, the cost to use a connection is considerably lower: the
> routing is known and the host-host link protocol (???URP") can be
> light-weight, as the network guarantees in-order delivery without
> duplicates but packets may be corrupted or lost (i.e. as if the connection
> is a phone line with a modem). No need to deal with packet fragmentation,
> stream reassembly and congestion storms as in the TCP of the early 80???s.
> >
> > Doing UDP traffic to a fixed remote host is easily mapped to using URP
> with no error correction and no flow control. Doing UDP where the remote
> host is different all the time is not practical on a Datakit network (i.e.
> a virtual circuit would be set up anyway).
> >
> > A secondary takeaway is that Research Datakit eventually settled on a
> three-level ascii namespace: ???area/trunk/switch???. On each switch, the
> hosts would be known by name, and each connection request had a service
> name as parameter. In an alternate reality we would maybe have used
> ???ca/stclara/mtnview!google!www??? to do a search.
>
> --
> ---
> Larry McVoy            Retired to fishing
> http://www.mcvoy.com/lm/boat
>

[-- Attachment #2: Type: text/html, Size: 4143 bytes --]

[TUHS] Re: Research Datakit notes

[Paul Ruizendaal]

> On 26 Jun 2022, at 01:57, Rob Pike <robpike@gmail.com> wrote:
> 
> One of the things we liked about Datakit was that the computer didn't have to establish the connection before it could reject the call, unlike TCP/IP where all validation happens after the connection is made. This is also why sockets and Datakit never worked together; sockets pretty much assume Ethernet-like connection rules.
> 
> I am not a networking expert, but to me in this regard at least Datakit seemed like a prettier picture. I suppose you can DOS-attack the network, but not the machines. Datakit had other issues, for sure, like the expensive racks of hardware, but then that's because, for better and worse, it was designed by phone engineers rather than.... however you'd characterize Ethernet and its original "I scream while listening to your whisper", 5V into 50Ω Schmidt-triggered craziness. Ethernet's come a long way, but the engineering of the original Radio Shack parts was not favored by the Bell Labs crowd.

I was not putting Datakit down, just trying to explain why the V8 approach to networking may seem a little odd from a 1980’s TCP/IP perspective, but makes perfect sense from a Datakit perspective.

In the end technology often becomes a hybrid of various solutions, and maybe in this case as well. By coincidence there was a post in the Internet History mailing list earlier today that appears to make this point.

In his video (https://www.youtube.com/watch?v=ojRtJ1U6Qzw), Sandy explains why he became dissatisfied with Spider and the main reason was that doing switching/routing on a mini computer was just plain inefficient as compared to a telephone switch (at 37:06). This was 1972. The result was a new design, Datakit, that could route/switch packets at high speed and in parallel.

On the internet history list, someone quipped: "Yeah, back then the joke was that McQuillan was the only one making money from ATM. :-) That did change in a big way (for a while) in the late 90s and early 2000s, before router silicon caught up."

To this Craig Partridge responded: "Wasn't just router silicon -- it was router design.  What made ATM appealing is that it made the inside of the router or switch parallel, which was necessary to push into multigigabit rates. Folks had to figure out how to rework an Internet router to be parallel and it took at least two major innovations: fully-standalone forwarding tables with associating forwarding engines and breaking packets apart (essentially into cells), squirting those parts through the parallel backplane, and then reassembling the packet at the outbound interface for transmission." This was around 2000.

It is not my field of expertise, but it would seem to me that Sandy had figured out a core problem some 30 years before the TCP/IP world would come up with a similar solution. I would not even be surprised if I learned that modern telco routers transparantly set up virtual circuits for tcp traffic.

[TUHS] Re: Research Datakit notes

[Grant Taylor via TUHS]

[-- Attachment #1: Type: text/plain, Size: 633 bytes --]

On 6/25/22 7:17 PM, Paul Ruizendaal wrote:
> In his video (https://www.youtube.com/watch?v=ojRtJ1U6Qzw), Sandy 
> explains why he became dissatisfied with Spider and the main reason 
> was that doing switching/routing on a mini computer was just plain 
> inefficient as compared to a telephone switch (at 37:06). This was 
> 1972. The result was a new design, Datakit, that could route/switch 
> packets at high speed and in parallel.

I didn't realize, or even fathom that something preceded ATM.

Now it seems like Spider preceded Datakit which preceded ATM.  Very 
interesting.



-- 
Grant. . . .
unix || die


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4017 bytes --]

[TUHS] Re: Research Datakit notes

[Larry McVoy]

On Fri, Jul 01, 2022 at 08:51:53PM -0600, Grant Taylor via TUHS wrote:
> On 6/25/22 7:17 PM, Paul Ruizendaal wrote:
> >In his video (https://www.youtube.com/watch?v=ojRtJ1U6Qzw), Sandy explains
> >why he became dissatisfied with Spider and the main reason was that doing
> >switching/routing on a mini computer was just plain inefficient as
> >compared to a telephone switch (at 37:06). This was 1972. The result was a
> >new design, Datakit, that could route/switch packets at high speed and in
> >parallel.
> 
> I didn't realize, or even fathom that something preceded ATM.
> 
> Now it seems like Spider preceded Datakit which preceded ATM.  Very
> interesting.

They were Bell Labs, funded by the phone company.  It makes sense that
they would look at networking through the phone company lens.

[TUHS] Re: Research Datakit notes

[Derek Fawcus]

On Sun, Jun 26, 2022 at 09:57:17AM +1000, Rob Pike wrote:
> One of the things we liked about Datakit was that the computer didn't have
> to establish the connection before it could reject the call, unlike TCP/IP
> where all validation happens after the connection is made.

Nor does TCP, one can send a RST to a SYN, and reject the call before it is
established.  That would then look to the caller just like a non listening
endpoint, unless one added data with the RST.

So this is really just a consequence of the sockets API, and the current implementations.
I've a vague recall of folks suggesting ways to expose that facility via the sockets
layer, possibly using setsockopt(), but don't know if anyone ever did it.

As I recall that TCP capability was actually exposed via the TLI/XTI API,
and (for some STREAMS based TCP stacks) it did function. Although I may be
thinking of embedded STREAMS TCP stacks, not unix based stacks.

Or by 'connection' are you referring to an end-to-end packet delivery,
and that Datakit allowed a closer switch to reject a call before the packet
got to the far end?

DF

[TUHS] Re: Research Datakit notes

[Rob Pike]

[-- Attachment #1: Type: text/plain, Size: 1516 bytes --]

I am not a networking expert. I said that already. The issue could well be
a property more of sockets than TCP/IP itself, but having the switch do
some of the call validation and even maybe authentication (I'm not sure...)
sounds like it takes load off the host.

-rob


On Tue, Jun 28, 2022 at 8:39 PM Derek Fawcus <
dfawcus+lists-tuhs@employees.org> wrote:

> On Sun, Jun 26, 2022 at 09:57:17AM +1000, Rob Pike wrote:
> > One of the things we liked about Datakit was that the computer didn't
> have
> > to establish the connection before it could reject the call, unlike
> TCP/IP
> > where all validation happens after the connection is made.
>
> Nor does TCP, one can send a RST to a SYN, and reject the call before it is
> established.  That would then look to the caller just like a non listening
> endpoint, unless one added data with the RST.
>
> So this is really just a consequence of the sockets API, and the current
> implementations.
> I've a vague recall of folks suggesting ways to expose that facility via
> the sockets
> layer, possibly using setsockopt(), but don't know if anyone ever did it.
>
> As I recall that TCP capability was actually exposed via the TLI/XTI API,
> and (for some STREAMS based TCP stacks) it did function. Although I may be
> thinking of embedded STREAMS TCP stacks, not unix based stacks.
>
> Or by 'connection' are you referring to an end-to-end packet delivery,
> and that Datakit allowed a closer switch to reject a call before the packet
> got to the far end?
>
> DF
>

[-- Attachment #2: Type: text/html, Size: 1942 bytes --]

[TUHS] Re: Research Datakit notes

[Rob Pike]

[-- Attachment #1: Type: text/plain, Size: 2934 bytes --]

One of the reasons I'm not a networking expert may be relevant here. With
networks, I never found an abstraction to hang my hat on. Unlike with file
systems and files, or even Unix character devices, which provide a level of
remove from the underlying blocks and sectors and so on, the Unix
networking interface always seemed too low-level and fiddly, analogous to
making users write files by managing the blocks and sectors themselves. It
could be all sockets' fault, but when I hear networking people talk about
the protocols and stacks and routing and load shedding and ....my ears
droop. I know it's amazing engineering and all that, but why aren't we
allowed to program the I/O without all that fuss? What makes networks so
_different_? A telling detail is that the original sockets interface had
send and recv, not read and write. From day 1 in Unix land at least,
networking was special, and it remains so, but I fail to see why it needs
to be.

It just seems there has to be a better way. Sockets are just so unpleasant,
and the endless nonsense around network configuration doubly so.

Rhetorical questions. I'm not asking or wanting an answer. I'm happy to
remain a greenhorn, oblivious to the wonder.

To adapt a reference some may recognize, I just want to read 5 terabytes.

-rob


On Tue, Jun 28, 2022 at 10:36 PM Rob Pike <robpike@gmail.com> wrote:

> I am not a networking expert. I said that already. The issue could well be
> a property more of sockets than TCP/IP itself, but having the switch do
> some of the call validation and even maybe authentication (I'm not sure...)
> sounds like it takes load off the host.
>
> -rob
>
>
> On Tue, Jun 28, 2022 at 8:39 PM Derek Fawcus <
> dfawcus+lists-tuhs@employees.org> wrote:
>
>> On Sun, Jun 26, 2022 at 09:57:17AM +1000, Rob Pike wrote:
>> > One of the things we liked about Datakit was that the computer didn't
>> have
>> > to establish the connection before it could reject the call, unlike
>> TCP/IP
>> > where all validation happens after the connection is made.
>>
>> Nor does TCP, one can send a RST to a SYN, and reject the call before it
>> is
>> established.  That would then look to the caller just like a non listening
>> endpoint, unless one added data with the RST.
>>
>> So this is really just a consequence of the sockets API, and the current
>> implementations.
>> I've a vague recall of folks suggesting ways to expose that facility via
>> the sockets
>> layer, possibly using setsockopt(), but don't know if anyone ever did it.
>>
>> As I recall that TCP capability was actually exposed via the TLI/XTI API,
>> and (for some STREAMS based TCP stacks) it did function. Although I may be
>> thinking of embedded STREAMS TCP stacks, not unix based stacks.
>>
>> Or by 'connection' are you referring to an end-to-end packet delivery,
>> and that Datakit allowed a closer switch to reject a call before the
>> packet
>> got to the far end?
>>
>> DF
>>
>

[-- Attachment #2: Type: text/html, Size: 3745 bytes --]

[TUHS] Re: Research Datakit notes

[Dan Cross]

On Tue, Jun 28, 2022 at 8:46 AM Rob Pike <robpike@gmail.com> wrote:
> One of the reasons I'm not a networking expert may be relevant here.
> With networks, I never found an abstraction to hang my hat on.

Hmm, this raises some design questions.

> Unlike with file systems and files, or even Unix character
> devices, which provide a level of remove from the underlying
> blocks and sectors and so on, the Unix networking interface
> always seemed too low-level and fiddly, analogous to making
> users write files by managing the blocks and sectors themselves.

I can see this. Sockets in particular require filling in abstruse
and mostly opaque data structures and then passing pointers
to them into the kernel. It's not so terribly onerous once one
gets it going, particularly for a streaming protocol like TCP,
but eronomically it also doesn't sit particularly well in the larger
paradigm. Something like plan9's dial() seems more in line with
the Unix model than what Unix actually gives us in terms of
TLI/Sockets/whatever. I'll note that 10th Edition had its IPC
library that handled some of this.

> It could be all sockets' fault, but when I hear networking people talk
> about the protocols and stacks and routing and load shedding and
> ....my ears droop. I know it's amazing engineering and all that, but
> why aren't we allowed to program the I/O without all that fuss?
> What makes networks so _different_? A telling detail is that the
> original sockets interface had send and recv, not read and write.
> From day 1 in Unix land at least, networking was special, and it
> remains so, but I fail to see why it needs to be.

Of course, the semantics of networking are a little different than
the (mostly) stream-oriented file model of Unix, in that datagram
protocols must be accommodated somehow and they have metadata
in the form of sender/receiver information that accompanies each
IO request. How does one model that neatly in the read/write case,
except by prepending a header or having another argument?

But the same is true of streaming to/from files and file-like things
as well. I can't `seek` on a pipe or a serial device, for obvious
reasons, but that implies that that model is not completely regular.
Similarly, writes to, say, a raw disk device that are not a multiple
of the sector size have weird semantics. The best we have done
is document this and add it to the oral lore.

One may argue that the disk device thing is a special case that
is so uncomment and only relevant to extremely low-level systems
programs that it doesn't count, but the semantics of seeking are
universal: programs that want to work as Unix filters have to
accommodate this somehow. In practice this doesn't matter much;
most filters just don't seek on their input.

Once again I am in awe of how Unix got it right for 90% of use
cases, and makes the last 10% possible, even if painful.

> It just seems there has to be a better way. Sockets are just so
> unpleasant, and the endless nonsense around network
> configuration doubly so.

No argument there.

> Rhetorical questions. I'm not asking or wanting an answer.
> I'm happy to remain a greenhorn, oblivious to the wonder.

As we continue forward, I wonder how much this matters.  We
talk about sockets, but how many programmers _actually_ reach
for that interface when they want to talk over a network?  I'd
wager that _most_ of the time now days it's hidden behind a
library interface that shields the consumer from the gritty details.
Sure, that library probably uses sockets internally, but most
people probably never look under that rock.

> To adapt a reference some may recognize, I just want to read 5 terabytes.

Believe it or not, I actually had Borgmon readability.

        - Dan C.


> On Tue, Jun 28, 2022 at 10:36 PM Rob Pike <robpike@gmail.com> wrote:
>>
>> I am not a networking expert. I said that already. The issue could well be a property more of sockets than TCP/IP itself, but having the switch do some of the call validation and even maybe authentication (I'm not sure...) sounds like it takes load off the host.
>>
>> -rob
>>
>>
>> On Tue, Jun 28, 2022 at 8:39 PM Derek Fawcus <dfawcus+lists-tuhs@employees.org> wrote:
>>>
>>> On Sun, Jun 26, 2022 at 09:57:17AM +1000, Rob Pike wrote:
>>> > One of the things we liked about Datakit was that the computer didn't have
>>> > to establish the connection before it could reject the call, unlike TCP/IP
>>> > where all validation happens after the connection is made.
>>>
>>> Nor does TCP, one can send a RST to a SYN, and reject the call before it is
>>> established.  That would then look to the caller just like a non listening
>>> endpoint, unless one added data with the RST.
>>>
>>> So this is really just a consequence of the sockets API, and the current implementations.
>>> I've a vague recall of folks suggesting ways to expose that facility via the sockets
>>> layer, possibly using setsockopt(), but don't know if anyone ever did it.
>>>
>>> As I recall that TCP capability was actually exposed via the TLI/XTI API,
>>> and (for some STREAMS based TCP stacks) it did function. Although I may be
>>> thinking of embedded STREAMS TCP stacks, not unix based stacks.
>>>
>>> Or by 'connection' are you referring to an end-to-end packet delivery,
>>> and that Datakit allowed a closer switch to reject a call before the packet
>>> got to the far end?
>>>
>>> DF

[TUHS] Re: Research Datakit notes

[Lawrence Stewart]

On 2022, Jun 28, at 9:33 AM, Dan Cross <crossd@gmail.com> wrote:
> 
> On Tue, Jun 28, 2022 at 8:46 AM Rob Pike <robpike@gmail.com> wrote:
>> One of the reasons I'm not a networking expert may be relevant here.
>> With networks, I never found an abstraction to hang my hat on.
> 

My theory is that networking is different because it breaks all the time.

One can get away with writing local file system programs without checking the return codes on every single call and figuring out what you should do in the case of every possible error.

It is much harder to get away with that sort of thing when writing network applications.

And of course there is a long history of things that look like file systems but which have network failure modes, and treating them like they are reliable often does not end well.

Full up reliable applications with replicated storage and multiple network availability zones and raft/paxos/byzantine generals level coding are pretty arcane.
I know I am not qualified to write one.

File systems are “good enough” that you can depend on them, mostly.  Networks are not.

-L

[TUHS] Re: Research Datakit notes

[Richard Salz]

[-- Attachment #1: Type: text/plain, Size: 199 bytes --]

> My theory is that networking is different because it breaks all the time.

This Sun research paper from 1994
https://scholar.harvard.edu/waldo/publications/note-distributed-computing
is a classic.

[-- Attachment #2: Type: text/html, Size: 366 bytes --]

[TUHS] Re: Research Datakit notes

[Stuart Remphrey]

[-- Attachment #1: Type: text/plain, Size: 727 bytes --]

Yes, I'd thought of Sun's Jini here, part of their Java push -- which IIRC
exposed network (un)reliability and tried to address it through service
lookup and limited-lifetime lease/renew for remote resource access.
Though I'm not sure it went anywhere?


(or how TUHS-relevant this is/was... apologies if not)


-----
Hmm, apparently Jini became Apache River; last release in 2016, then
retired Feb 2022:
https://en.wikipedia.org/wiki/Jini


On Wed, 29 Jun 2022, 05:36 Richard Salz, <rich.salz@gmail.com> wrote:

> > My theory is that networking is different because it breaks all the time.
>
> This Sun research paper from 1994
> https://scholar.harvard.edu/waldo/publications/note-distributed-computing
> is a classic.
>
>
>

[-- Attachment #2: Type: text/html, Size: 1490 bytes --]

[TUHS] Re: Research Datakit notes

[Tom Teixeira]

On 6/28/22 8:45 AM, Rob Pike wrote:
> One of the reasons I'm not a networking expert may be relevant here. 
> With networks, I never found an abstraction to hang my hat on. Unlike 
> with file systems and files, or even Unix character devices, which 
> provide a level of remove from the underlying blocks and sectors and 
> so on, the Unix networking interface always seemed too low-level and 
> fiddly, analogous to making users write files by managing the blocks 
> and sectors themselves. It could be all sockets' fault, but when I 
> hear networking people talk about the protocols and stacks and routing 
> and load shedding and ....my ears droop. I know it's amazing 
> engineering and all that, but why aren't we allowed to program the I/O 
> without all that fuss? What makes networks so _different_? A telling 
> detail is that the original sockets interface had send and recv, not 
> read and write. From day 1 in Unix land at least, networking was 
> special, and it remains so, but I fail to see why it needs to be.

Two observations:

At the time, I think everyone was searching for the right abstraction. I 
don't remember the whole talk, but just an anecdote by, I think, David 
Clark. I don't remember if this was just a seminar at MIT LCS or perhaps 
at SIGOPS. In any case, David talked about trying to get some database 
people to use the virtual memory and file systems abstractions that had 
been built by Multics. They agreed that these were nice abstractions, 
but in the mean time, "get out of our way and let us at the disk."

Since networking developers and users were all searching for the right 
abstraction, and new hardware, protocols, and software interfaces were 
being proposed on what seemed like a weekly basis, many of the proposed 
interfaces tried to expose low level mechanisms as well as high level 
stream abstractions, preserving the hope that something like TCP could 
be implemented at user code level rather than the kernel.

Secondly, I had to dig up a reference for the Chaosnet software (MIT AI 
memo 628 available at 
http://bitsavers.trailing-edge.com/pdf/mit/ai/AIM-628_chaosnet.pdf and 
probably other places). The Unix implementation used the rest of the 
path name to specify connection setup parameters in the typical case 
which seemed more unix-like than sockets. But the Chaosnet software was 
definitely swept away in the Ethernet/sockets storm surge.

[TUHS] Re: Research Datakit notes

[John Floren]

On 6/28/22 05:45, Rob Pike wrote:
> [...] I know it's amazing engineering and
> all that, but why aren't we allowed to program the I/O without all that
> fuss? What makes networks so _different_? A telling detail is that the
> original sockets interface had send and recv, not read and write. From
> day 1 in Unix land at least, networking was special, and it remains so,
> but I fail to see why it needs to be.
>
> It just seems there has to be a better way. Sockets are just so
> unpleasant, and the endless nonsense around network configuration doubly so.
>

I was pretty sad when netchans were discontinued. A colleague was
sufficiently attached to them that he kept his own branch of the library
going for a while.



john

[TUHS] Re: Research Datakit notes

[Rich Morin]

> On Jun 28, 2022, at 05:36, Rob Pike <robpike@gmail.com> wrote:
> 
> I am not a networking expert. I said that already. The issue could well be a property more of sockets than TCP/IP itself, but having the switch do some of the call validation and even maybe authentication (I'm not sure...) sounds like it takes load off the host.

Some years ago, we set up a front end email server to reject incoming message attempts that didn't match our list of valid users.  This resulted in a better then 90% reduction.

-r

[TUHS] Re: Research Datakit notes

[Marc Donner]

[-- Attachment #1: Type: text/plain, Size: 2387 bytes --]

In the mid-1980s I returned to IBM Research after finishing up at CMU.  I
smuggled a bunch of Sun machines in and strung Ethernet between my office
and my lab so that the desktop could talk to the server.

Then I went around IBM giving talks about TCP/IP and why IBM should commit
to it.  At the time IBM Research was the center of development of IBM's SNA
stuff, so there was some (!) tension.  (Particularly because Paul Greene,
one of the key leaders of the SNA work, was very close to my undergraduate
mentor, so I had socialized with him.)  They proposed running TCP/IP
encapsulated in SNA, but I told them that the best they could expect was to
encapsulate SNA in TCP/IP.  That turned out to be what happened.

My perception of the debate at the time was that it pitted proprietary
networking (SNA, DECNet, ...) against open networking (TCP/IP).  The
hardware vendors wanted proprietary networking to lock customers into their
equipment, but that dog would not hunt.

Meanwhile, our community had recently discovered how horrible proprietary
tech was for our careers ... the mid-1980s recession led to serious layoffs
in the system programmer community and the newly unemployed geeks
discovered that the skills so assiduously honed were not portable.  Enter
FSK and the open source movement.

It was pretty clear that except for the clever encapsulation stuff that
Vint had done with IP, the TCP/IP world was quick and dirty and quite
slapdash.  But it was non-proprietary and that is what won the race.

What I don't understand is whether Rob's observation about networking is
*fundamental* to the space or *incidental* to the implementation.  I would
love to be educated on that.

Marc
=====
nygeek.net
mindthegapdialogs.com/home <https://www.mindthegapdialogs.com/home>


On Tue, Jun 28, 2022 at 8:48 AM Rich Morin <rdm@cfcl.com> wrote:

> > On Jun 28, 2022, at 05:36, Rob Pike <robpike@gmail.com> wrote:
> >
> > I am not a networking expert. I said that already. The issue could well
> be a property more of sockets than TCP/IP itself, but having the switch do
> some of the call validation and even maybe authentication (I'm not sure...)
> sounds like it takes load off the host.
>
> Some years ago, we set up a front end email server to reject incoming
> message attempts that didn't match our list of valid users.  This resulted
> in a better then 90% reduction.
>
> -r
>
>

[-- Attachment #2: Type: text/html, Size: 4147 bytes --]

[TUHS] Re: Research Datakit notes

[Clem Cole]

[-- Attachment #1.1: Type: text/plain, Size: 1209 bytes --]

On Tue, Jun 28, 2022 at 9:15 AM Marc Donner <marc.donner@gmail.com> wrote:

> My perception of the debate at the time was that it pitted proprietary
> networking (SNA, DECNet, ...) against open networking (TCP/IP).  The
> hardware vendors wanted proprietary networking to lock customers into their
> equipment, but that dog would not hunt.
>
Metcalfe's law: "*value of a network is proportional to the square of the
number of connected users of the system*." The problem with a walled garden
is that it can only grow as large as the walls allow.

>
> It was pretty clear that except for the clever encapsulation stuff that
> Vint had done with IP, the TCP/IP world was quick and dirty and quite
> slapdash.  But it was non-proprietary and that is what won the race.
>
Point taken, but I actually think it is more of a Christensen-style disruption
where the 'lessor technology' outstrips the more sophisticated one because
it finds/creates a new market that values that new technology for what it
is and cares less about the ways it may be 'lessor.'

I described this in a talk I did at Asilomar a few years back.  This is the
most important slide:
[image: ColesLaw20190222.png]

ᐧ

[-- Attachment #1.2: Type: text/html, Size: 3544 bytes --]

[-- Attachment #2: ColesLaw20190222.png --]
[-- Type: image/png, Size: 252558 bytes --]

[TUHS] Re: Research Datakit notes

[Tom Teixeira]

[-- Attachment #1: Type: text/plain, Size: 1765 bytes --]

Christensen-style disruption sounds rather like Gresham's Law ("bad 
money drives out good"), but I don't think the mechanism is the same: 
one can hoard old silver coins and sell those at a profit for the silver 
content, but there's no premium I know of for better technology -- 
probably because "better technology" seems to imply aesthetics, but 
newer, "lessor technology" is likely to be much faster while using less 
energy.

On 6/28/22 10:41 AM, Clem Cole wrote:
>
>
> On Tue, Jun 28, 2022 at 9:15 AM Marc Donner <marc.donner@gmail.com> wrote:
>
>     My perception of the debate at the time was that it pitted
>     proprietary networking (SNA, DECNet, ...) against open networking
>     (TCP/IP).  The hardware vendors wanted proprietary networking to
>     lock customers into their equipment, but that dog would not hunt.
>
> Metcalfe's law: "/value of a network is proportional to the square of 
> the number of connected users of the system/."The problem with a 
> walled garden is that it can only grow as large as the walls allow.
>
>
>     It was pretty clear that except for the clever encapsulation stuff
>     that Vint had done with IP, the TCP/IP world was quick and dirty
>     and quite slapdash.  But it was non-proprietary and that is what
>     won the race.
>
> Point taken, but I actually think it is more of a 
> Christensen-style disruption where the 'lessor technology' outstrips 
> the more sophisticated one because it finds/creates a new market that 
> values that new technology for what it is and cares less about the 
> ways it may be 'lessor.'
>
> I described this in a talk I did at Asilomar a few years back.  This 
> is the most important slide:
> ColesLaw20190222.png
>
> ᐧ


[-- Attachment #2.1: Type: text/html, Size: 5850 bytes --]

[-- Attachment #2.2: ColesLaw20190222.png --]
[-- Type: image/png, Size: 252558 bytes --]

[TUHS] Re: Research Datakit notes

[Adam Thornton]

[-- Attachment #1: Type: text/plain, Size: 2887 bytes --]



> On Jun 28, 2022, at 6:13 AM, Marc Donner <marc.donner@gmail.com> wrote:
> 
> What I don't understand is whether Rob's observation about networking is *fundamental* to the space or *incidental* to the implementation.  I would love to be educated on that.

And there it is!  THAT was the sentence--well, ahort paragraph--that jogged my memory as to why this seemed familiar.

If you go back to _The Unix-Hater's Handbook_ (I know, I know, bear with me), one of the things I noticed and pointed out in my review (https://athornton.dreamwidth.org/14272.html) is how many of the targets of hatred, twenty years down the line, turned out to be unix-adjacent, and not fundamental.

In the book, these were things like Usenet and sendmail.cf (indeed, those were the two big ones).

But the current discussion: is the thing we don't like Berkeley Sockets?  Is it TCP/IP itself?  Is the the lack of a Unixy abstraction layer over some lower-level technology?  To what degree is it inherent?

I mean, obviously, to some degree it's all three, and I think a large but fairly unexamined part of it is that TCP/IP these days almost always at least pretends to be sitting on top of Ethernet at the bottom...but of course Classic Ethernet largely died in the...early 2000s, I guess?...when even extremely cheap home multiple-access-devices became switches rather than hubs.

Some sort of inter-machine networking is clearly inherent in a modern concept of Unix.  I think we're stuck with the sockets interface and IP, whether we like them or not.  They don't bother me a great deal, but, yes, they do not feel as unixy as, say, /dev/tcp does.  But the interesting thing is that I think that is Unix-adjacent or, like the UHH distate for Unix filesystems, it's at least incidental and could be replaced if the desire arose.  And I think we already have the answer about what the abstraction is, albeit at an application rather than the kernel level.

To answer Rob's question: I think the abstraction is now much farther up the stack.  To a pretty good first approximation, almost all applications simply definte their own semantics on top of HTTP(S) (OK, OK, Websockets muddy the waters again) and three-to-five verbs.  There's an incantation to establish a circuit (or a "session" if you're under the age of 50, I guess), and then you GET, DELETE, and at least one of PUT/POST/PATCH, for "read", "unlink", and "write".  This does seem to be a more record-oriented (kids these days get snippy if you call them "records" rather than "objects" but w/e) format than a stream of bytes (or at least you put an abstraction layer in between your records and the stream-of-octets that's happening).

This is certainly not efficient at a wire protocol level, but it's a fairly small cognitive burden for people who just want to write applications that communicate with each other.

Adam

[-- Attachment #2: Type: text/html, Size: 3874 bytes --]

[TUHS] Re: Research Datakit notes

[John Labovitz]

I’m generally a lurker here, but this has been an interesting conversation to observe. I often hesitate to post, as to not offend you folks who literally invented this stuff, but I thought it might be helpful to share some experiences with socket-based I/O.

For the first time in ~20 years, I’ve recently been writing low-level (eg, not through a library layer/framework) socket code in straight C, for an art project based on an ESP32 embedded/SoC. I first played around with the socket API in the mid-80s, and then wrote a lot of socket-using code in the 1990s for the Watchguard Firebox, the first Linux-based appliance firewall.

I have to say that I really enjoying programming with sockets. I feel that it *does* make a lot of sense if I'm thinking directly about the TCP/IP stack, *and* if my code has a good 'impedance match' to the protocols. If I’m writing a server, I’m dealing with connections and queues and various-sized packets/messages/blocks, which have to fit into some decision of memory usage (often true in embedded systems). Usually I’m not simply writing, say, a file server that simply reads from disc and sends bytes out through a stream and then calls close().

I also believe that the sockets API really comes into its own with high-capacity, non-threaded, non-blocking servers or clients — that is, ones that use select() or poll() and then recv() and send() or their variants. I’m sure that if I didn’t have a sockets API and only had open(), read(), write(), etc., I could make it work, but there’s something almost beautiful that happens at a large scale with many non-blocking sockets (see: the reactor pattern) that I don’t think would translate as well with a typical everything-is-a-file model.

My opinion solely, of course. But I’m simply happy that both socket- and file-based APIs exist. Each has their purpose.

—John


> On Jun 28, 2022, at 19:05, Adam Thornton <athornton@gmail.com> wrote:
> 
> 
> 
>> On Jun 28, 2022, at 6:13 AM, Marc Donner <marc.donner@gmail.com> wrote:
>> 
>> What I don't understand is whether Rob's observation about networking is *fundamental* to the space or *incidental* to the implementation.  I would love to be educated on that.
> 
> And there it is!  THAT was the sentence--well, ahort paragraph--that jogged my memory as to why this seemed familiar.
> 
> If you go back to _The Unix-Hater's Handbook_ (I know, I know, bear with me), one of the things I noticed and pointed out in my review (https://athornton.dreamwidth.org/14272.html) is how many of the targets of hatred, twenty years down the line, turned out to be unix-adjacent, and not fundamental.
> 
> In the book, these were things like Usenet and sendmail.cf (indeed, those were the two big ones).
> 
> But the current discussion: is the thing we don't like Berkeley Sockets?  Is it TCP/IP itself?  Is the the lack of a Unixy abstraction layer over some lower-level technology?  To what degree is it inherent?
> 
> I mean, obviously, to some degree it's all three, and I think a large but fairly unexamined part of it is that TCP/IP these days almost always at least pretends to be sitting on top of Ethernet at the bottom...but of course Classic Ethernet largely died in the...early 2000s, I guess?...when even extremely cheap home multiple-access-devices became switches rather than hubs.
> 
> Some sort of inter-machine networking is clearly inherent in a modern concept of Unix.  I think we're stuck with the sockets interface and IP, whether we like them or not.  They don't bother me a great deal, but, yes, they do not feel as unixy as, say, /dev/tcp does.  But the interesting thing is that I think that is Unix-adjacent or, like the UHH distate for Unix filesystems, it's at least incidental and could be replaced if the desire arose.  And I think we already have the answer about what the abstraction is, albeit at an application rather than the kernel level.
> 
> To answer Rob's question: I think the abstraction is now much farther up the stack.  To a pretty good first approximation, almost all applications simply definte their own semantics on top of HTTP(S) (OK, OK, Websockets muddy the waters again) and three-to-five verbs.  There's an incantation to establish a circuit (or a "session" if you're under the age of 50, I guess), and then you GET, DELETE, and at least one of PUT/POST/PATCH, for "read", "unlink", and "write".  This does seem to be a more record-oriented (kids these days get snippy if you call them "records" rather than "objects" but w/e) format than a stream of bytes (or at least you put an abstraction layer in between your records and the stream-of-octets that's happening).
> 
> This is certainly not efficient at a wire protocol level, but it's a fairly small cognitive burden for people who just want to write applications that communicate with each other.
> 
> Adam

[TUHS] HTTP (was Re: Re: Research Datakit notes)

[Derek Fawcus]

On Tue, Jun 28, 2022 at 10:05:26AM -0700, Adam Thornton wrote:
> 
> And I think we already have the answer about what the abstraction is, albeit at an application rather than the kernel level.
> 
> To answer Rob's question: I think the abstraction is now much farther up the stack.  To a pretty good first approximation, almost all applications simply definte their own semantics on top of HTTP(S) (OK, OK, Websockets muddy the waters again) and three-to-five verbs.  There's an incantation to establish a circuit (or a "session" if you're under the age of 50, I guess), and then you GET, DELETE, and at least one of PUT/POST/PATCH, for "read", "unlink", and "write".  This does seem to be a more record-oriented (kids these days get snippy if you call them "records" rather than "objects" but w/e) format than a stream of bytes (or at least you put an abstraction layer in between your records and the stream-of-octets that's happening).

Yes it is effectively records, or RPCs w/o state between requests.  So akin to DNS queries over UDP, rather than DNS queries over TCP.

As the whole HTTP model allows for proxies, at which point the request (GET, HEAD, POST, etc) includes
an endpoint address (DNS name and port) and path at that endpoint.  Having 'session' state is built at
a level above HTTP, by cookies or redirecting to magic URIs encoding the session ID after the query (the '?').

The fact that this all happens to run over a TCP connection, simply means that we have stacked sets of
VCs.  An application VC on top of a connectionless HTTP on top of a TCP 'VC'.  One could argue that
Websockets simplifies this by ripping out the HTTP layer, and having the top VC then just being message
framing within the TCP session.

(Ignoring for the moment TLS on top of TCP, or HTTP/3 being on top of QUIC, hence UDP)

> This is certainly not efficient at a wire protocol level, but it's a fairly small cognitive burden for people who just want to write applications that communicate with each other.

Sort of, in that I recently investigated the Go HTTP client/server APIs as someone was asking us about 
implementing a MiTM HTTP(S) "firewall".  Depending upon how one deals with it, it hides the VCs and exposes
the connectionless RPCs.

With such a MiTM in place, the 'nice' RPC in a session is effectivly forced back to the 'nasty' RPC in datagrams, even though the endpoints may be unaware, but they already include the whole datagram remote address in the request.  Responses come back by 'magic' without explicit address.

So maybe from that perspective one could model HTTP request/response as RPCs over SVCs, those being raised and torn down for each exchange.

DF

[TUHS] Re: Research Datakit notes

[Anthony Martin]

Paul Ruizendaal <pnr@planet.nl> once said:
> Wanted to post my notes as plain text, but the bullets / sub-bullets get lost.
>
> Here is a 2 page PDF with my notes on Research Datakit:
>
> https://www.jslite.net/notes/rdk.pdf

Really nice outline. I felt some déjà vu when opening the document: I
remember reading your notes in April of 2020.

Would you happen to know where I can find copies of these three
papers?

A. G. Fraser, "Datakit - A Modular Network for Synchronous and
Asynchronous Traffic", Proc. ICC 79, June 1979, Boston, Ma.,
pp.20.1.1-20.1.3

G. L. Chesson, "Datakit Software Architecture", Proc. ICC 79, June
1979, Boston Ma., pp.20.2.1-20.2.5

G. L. Chesson and A. G. Fraser, "Datakit Network Architecture," Proc.
Compcon 80, February 1980, San Fransisco CA., pp.59-61

> A secondary takeaway is that Research Datakit eventually settled on a
> three-level ascii namespace: “area/trunk/switch”. On each switch, the
> hosts would be known by name, and each connection request had a
> service name as parameter. In an alternate reality we would maybe have
> used “ca/stclara/mtnview!google!www” to do a search.

To connect to one of the Plan 9 cpu servers at Bell Labs, you would
dial "nj/astro/helix!9fs". I do wonder how the relative hierarchical
naming would have evolved to encompass the entire Internet. Would
it have been more like "com/google/search!http"? Who knows? ☺

Thanks,
  Anthony

[TUHS] Re: Research Datakit notes

[Ralph Corderoy]

Hi Anthony,

> Would you happen to know where I can find copies of these three
> papers?
>
> A. G. Fraser, "Datakit - A Modular Network for Synchronous and
> Asynchronous Traffic", Proc. ICC 79, June 1979, Boston, Ma.,
> pp.20.1.1-20.1.3
>
> G. L. Chesson, "Datakit Software Architecture", Proc. ICC 79, June
> 1979, Boston Ma., pp.20.2.1-20.2.5
>
> G. L. Chesson and A. G. Fraser, "Datakit Network Architecture," Proc.
> Compcon 80, February 1980, San Fransisco CA., pp.59-61

I had no luck looking for those.  I did find other Datakit ones
which may interest the list; all Bell Labs.

- A Virtual Circuit Switch as the Basis for Distributed Systems.
  Luderer, Che, Marshall, Bell Labs, Murray Hill.  ~1981.
  ‘...we have implemented... a new distributed operating system derived
  from Unix’.
  https://dl.acm.org/doi/pdf/10.1145/1013879.802670

- Methods for routing, performance management, and service problem
  relief in Datakit networks.
  Fendick, Harshavardhana, Jidarian, Bell Labs, Holmdel.  1991.
  https://libgen.rocks/get.php?md5=27c34fe1fce3ed20b79aaf65362cb5f9&key=VXFE8W3J24GAYURB&doi=10.1109/icc.1991.162268

- FIBERKIT: a Datakit compatible 500 megabit virtual circuit switch.
  Follett, Levy, Sobin, Tourgee, AT&T Bell Labs.  1988.
  https://libgen.rocks/get.php?md5=577344ef77634022868f52969dbda62b&key=COCJ9PQZSNIJ2U3B&doi=10.1109/glocom.1988.25918

-- 
Cheers, Ralph.

[TUHS] Re: Research Datakit notes

[Paul Ruizendaal]

> On 26 Jun 2022, at 03:41, Anthony Martin <ality@pbrane.org> wrote:
> 
> Really nice outline. I felt some déjà vu when opening the document: I
> remember reading your notes in April of 2020.

Thank you. Yes, I have not updated these notes since then.

> Would you happen to know where I can find copies of these three
> papers?
> 
> A. G. Fraser, "Datakit - A Modular Network for Synchronous and
> Asynchronous Traffic", Proc. ICC 79, June 1979, Boston, Ma.,
> pp.20.1.1-20.1.3
> 
> G. L. Chesson, "Datakit Software Architecture", Proc. ICC 79, June
> 1979, Boston Ma., pp.20.2.1-20.2.5
> 
> G. L. Chesson and A. G. Fraser, "Datakit Network Architecture," Proc.
> Compcon 80, February 1980, San Fransisco CA., pp.59-61

Two of these three were in Sandy Fraser’s archives:

https://www.jslite.net/notes/dk1.pdf

https://www.jslite.net/notes/dk2.pdf

The middle one is I fear lost - unless a paper copy of the proceedings is still lurking in some university library. I think it will be mostly about the “CMC” switch control software, which is somewhat described in the Luderer paper as well. As far as I can tell, there is no surviving Unix code that was designed for the CMC version of Datakit (unless someone is sitting on a V7 tape with the Datakit drivers still included).

> To connect to one of the Plan 9 cpu servers at Bell Labs, you would
> dial "nj/astro/helix!9fs”.

Out of interest, would you know if the switch-host interface for Datakit in the days of Plan9 worked more or less the same as in 8th Edition? Or had it much evolved by then?

[TUHS] Re: Research Datakit notes

[Paul Ruizendaal]

> Would you happen to know where I can find copies of these three
> papers?
> 
> A. G. Fraser, "Datakit - A Modular Network for Synchronous and
> Asynchronous Traffic", Proc. ICC 79, June 1979, Boston, Ma.,
> pp.20.1.1-20.1.3
> 
> G. L. Chesson, "Datakit Software Architecture", Proc. ICC 79, June
> 1979, Boston Ma., pp.20.2.1-20.2.5
> 
> G. L. Chesson and A. G. Fraser, "Datakit Network Architecture," Proc.
> Compcon 80, February 1980, San Fransisco CA., pp.59-61


I just remembered that I had received a copy of a file note (50+ pages) that Greg Chesson wrote in 1982 about the "CMC” control software for Datakit. I think it covers the same ground as the 1979 paper, but in far greater detail and with two more years of development. In short, the connection protocol in CMC is based on the exchange of binary messages. That was replaced (for the most part) by text-based messages in the later TDK control software.

It is here (it is a 16MB pdf):

https://www.jslite.net/notes/dk3.pdf

To compare, here are the first two design documents on sockets. I looked for these for many years (even had the Berkeley library manually search the boxes with CSRG documents that Kirk McKusick had sent there - to no avail), and then in 2021 Rich Morin found them in the papers of Jim Joyce. I’m still very thankful for this.

These two papers were written in the summer of 1981 and circulated to the newly formed steering committee for what was to become 4.2BSD (note: ~5MB pdf each).

The first is specifically on networking:

https://www.jslite.net/notes/joy1.pdf

The second outlines the overall ambitions for the new version (including a summary of the above document). It has an interesting view of John Reiser’s VM code in its section 3.17 as well:

https://www.jslite.net/notes/joy2.pdf

What was proposed is not quite the sockets we know, but the general direction is set and the reasoning is explained. Reading the Chesson and Joy paper side by side makes for an interesting comparison of thinking on these topics in the early 80’s.

Maybe they are worth storing in the TUHS archive.

Wbr,

Paul

[TUHS] Research Datakit notes

[Geoff Pool]

[-- Attachment #1: Type: text/plain, Size: 263 bytes --]

I've enjoyed reading this thread as networking has always been a passion of
mine.  Lawrence Livermore had, at one time, their own networking system
they called Spider.  Is this the same Spider technology that Sandy Fraiser
references in his Datakit notes?

Geoff

[-- Attachment #2: Type: text/html, Size: 319 bytes --]