The Unix Heritage Society mailing list
 help / color / mirror / Atom feed
* [TUHS] Re: Research Datakit notes
@ 2022-06-28 15:50 Noel Chiappa
  2022-06-28 21:32 ` Lawrence Stewart
  0 siblings, 1 reply; 38+ messages in thread
From: Noel Chiappa @ 2022-06-28 15:50 UTC (permalink / raw)
  To: tuhs; +Cc: jnc

    > From: Rob Pike

    > having the switch do some of the call validation and even maybe
    > authentication (I'm not sure...) sounds like it takes load off the host.

I don't have enough information to express a judgement in this particular
case, but I can say a few things about how one would go about analyzing
questions of 'where should I put function [X]; in the host, or in the
'network' (which almost inevitably means 'in the switches')'.

It seems to me that one has to examine three points:

- What is the 'cost' to actually _do_ the thing (which might be in
transmission usage, or computing power, or memory, or delay), in each
alternative; these costs obviously generally cannot be amortized across
multiple similar transactions.

- What is the 'cost' of providing the _mechanism_ to do the thing, in each
alternative. This comes in three parts. The first is the engineering cost of
_designing_ the thing, in detail; this obviously is amortized across muiple
instances. The second is _producing_ the mechanism, in the places where it is
needed (for mechanisms in software, this cost is essentially zero, unless it
needs a lot of memory/computes/etc); this is not amortized across many. The
third is harder to measure: it's complexity.

This is probably a book by itself, but it has costs that are hard to
quantify, and are also very disparate: e.g. more complex designs are more
likely to have unforseen bugs, which is very different from the 'cost' that
more complex designs are probaly harder to evolve for new uses.

So far I haven't said anything that isn't applicable across a broad range of
information sytems. The last influence on where one puts functions is much
more common in communication systems: the Saltzer/Clark/Reed 'End-to-end
Arguments in System Design' questions. If one _has_ to put a function in the
host to get 'acceptable' performace of that function, the
operation/implementation/design cost implications are irrelevant: one has to
grit one's teeth and bear them.

This may then feed back to design questions in the other areas. E.g. the
Version 2 ring at MIT deliberately left out hardware packet checksums -
because it was mostly intended for use with TCP/IP traffic, which provided a
pseudo-End-to-End checksum, so the per-unit hardware costs didn't buy enough
to be worth the costs of a hardware CRC. (Which was the right call; I don't
recall the lack of a hardware checksum ever causing a problem.)

And then there's the 'techology is a moving target' point: something that
might be unacceptably expensive (in computing cost) in year X might be fine
in year X+10, when we're lighting our cigars with unneeded computing power.
So when one is designing a communication system with a likely lifetime in
many decades, one tends to bias one's judgement toward things like End-to-End
analysis - because those factors will be forever.


Sorry if I haven't offered any answer to your initial query: "having the
switch do some of the call validation ... sounds like it takes load off the
host", but as I have tried to explain, these 'where should one do [X]'
questions are very complicated, and one would need a lot more detail before
one could give a good answer.

But, in general, "tak[ing] load off the host" doesn't seem to rate
highly as a goal these days... :-) :-(

	Noel

^ permalink raw reply	[flat|nested] 38+ messages in thread
* [TUHS] Re: Research Datakit notes
@ 2022-06-27 21:40 Noel Chiappa
  2022-06-27 22:40 ` George Michaelson
  0 siblings, 1 reply; 38+ messages in thread
From: Noel Chiappa @ 2022-06-27 21:40 UTC (permalink / raw)
  To: tuhs; +Cc: jnc

    > From: Paul Ruizendaal

    > Will read those RFC's, though -- thank you for pointing them out.

Oh, I wouldn't bother - unless you are really into routing (i.e. path
selection).

RFC-1992 in particular; it's got my name on it, but it was mostly written by
Martha and Isidro, and I'm not entirely happy with it. E.g. CSC mode and CSS
mode (roughly, strict source route and loose source route); I wasn't really
sold on them, but I was too tired to argue about it. Nimrod was complicated
enough without adding extra bells and whistles - and indeed, LSR and SSR are
basically unused to this day in the Internet (at least, at the internet
layer; MPLS does provide the ability to specify paths, which I gather is used
to some degree). I guess it's an OK overview of the architecture, though.

RFC-1753 is not the best overview, but it has interesting bits. E.g. 2.2
Packet Format Fields, Option 2: "The packet contains a stack of flow-ids,
with the current one on the top." If this reminds you of MPLS, it should!
(One can think of MPLS as Nimrod's packet-carrying subsystem, in some ways.)

I guess I should mention that Nimrod covers more stuff - a lot more - than
just path selection. That's because I felt that the architecture embodied in
IPv4 was missing lots of things which one would need to do the internet layer
'right' in a global-scale Internet (e.g. variable length 'addresses' - for
which we were forced to invent the term 'locator' because many nitwits in the
IETF couldn't wrap their minds around 'addresses' which weren't in every
packet header). And separation of location and identity; and the introduction
of traffic aggregates as first-class objects at the internet layer. Etc, etc,
etc.

Nimrod's main focus was really on i) providing a path-selection system which
allowed things like letting users have more input to selecting the path their
traffic took (just as when one gets into a car, one gets to pick the path
one's going to use), and ii) controlling the overhead of the routing.

Of course, on the latter point, in the real world, people just threw
resources (memory, computing power, bandwidth) at the problem. I'm kind of
blown away< that there are almost 1 million routes in the DFZ these days.
Boiling frogs...

	Noel

^ permalink raw reply	[flat|nested] 38+ messages in thread
* [TUHS] Re: Research Datakit notes
@ 2022-06-27  0:43 Noel Chiappa
  2022-06-27  3:00 ` Erik Fair
  0 siblings, 1 reply; 38+ messages in thread
From: Noel Chiappa @ 2022-06-27  0:43 UTC (permalink / raw)
  To: tuhs; +Cc: jnc

Just as the topic of TUHS isn't 'how _I_ could/would build a _better_ OS', but
'history of the OS that was _actually built_' (something that many posters
here seem to lose track of, to the my great irritation), so too the topic
isn't 'how to build a better network' - or actually, anything network-centric.

I'll make a few comments on a couple of things, though.


    > From: steve jenkin

    > packet switching won over Virtual Circuits in the now distant past but
    > in small, local and un-congested networks without reliability
    > constraints, any solution can look good. ... Packet switching
    > hasn't scaled well to Global size, at least IMHO.

The internetworking architecture, circa 1978, has not scaled as well as would
have been optimal, for a number of reasons, among them:

- pure scaling effects (e.g. algorithms won't scale up; subsystems which
  handle several different needs will often need to be separated out at a larger
  scale; etc)

- inherent lack of hindsight (unknown unknowns, to use Rumsfeld's phrase; some
  things you only learn in hindsight)

- insufficiently detailed knowledge of complete requirements for a
  global-scale network (including O+M, eventual business model, etc)

- limited personnel resources at the time (some things we _knew_ were going to
  be a problem we had to ignore because we didn't have people to throw at the
  problem, then and there)

- rapid technological innovation (and nobody's crystal ball is 100% perfect)

It has been possible to fix some aspects of the ca. 1978 system - e.g. the
addition of DNS, which I think has worked _reasonably_ well - but in other
areas, changes weren't really adequate, often because they were constrained by
things like upward compatibility requirements (e.g. BGP, which, among numerous
other issues, had to live with existing IP addressing).

Having said all that, I think your assertion that virtual circuits would have
worked better in a global-scale network is questionable. The whole point of
networks which use unreliable datagrams as a fundamental building block is
that by moving a lot of functionality into the edge nodes, it makes the
switches a lot simpler. Contemporary core routers may be complex - but they
would be much worse if the network used virtual circuits.

Something I suspect you may be unaware of is that most of the people who
devised the unreliable datagram approach of the internetworking architecture
_had experience with an actual moderately-sized, operational virtual circuit
network_ - the ARPANET. (Yes, it was basically a VC network. Look at things
like RFNMs, links {the specific ARPANET mechanism referred to by this term,
not the general concept}, etc.) So they _knew_ what a VC network would
involve.

So, think about the 'core routers' in a network which used VC's. I guess a
typical core router tese days uses a couple of OC768 links. Assume an average
packet size of 100 bytes (probably roughly accurate, with the bimodal
distribution between data and acks). With 4 OC768's, that's 4*38.5G/800 =
~155M packets/second. I'm not sure of the average TCP connection length in
packets these days, but assume it's 100 packets or so (that's a 100KB Web
object). That's still roughly _1 million cicuit setups per second_.

If the answer is 'oh, we'll use aggregation so core routers don't see
individual connections - or their setup/tear-down' - well, the same
can be done with a datagram system; that's what MPLS does. Work
through the details - VCs were not preferred, for good reasons.


    > Ethernet only became a viable LAN technology with advent of Twisted
    > pair: point to point + Switches.

It's really irritating that a lot of things labelled 'Ethernet' these days
_aren't_ _real_ Ethernet (i.e. a common broadcast bus allocated via CSMA-CD).

They use the same _packet format_ as Ethernet (especially the 48-bit
globally-unique address, which can usefully be blown into things at
manufacture time), but it's not Ethernet. In some cases, they also retain the
host interface<->network physical interface - but the thing on the other side
of the interface is totally different (such as the hub-based systems commmon
now - as you indicate, it's a bunch of small datagram packet switches plugged
together with point-point links).

Interfaces are forever; like the screw in light-bulb. These days, it's likely
an LED bulb on one side, powered by a reactor on the other - two technologies
which were unforseen (and unforseeable) when the interface was defined, well
over 100 years ago.

	Noel

^ permalink raw reply	[flat|nested] 38+ messages in thread
* [TUHS] Re: Research Datakit notes
@ 2022-06-26  2:19 Noel Chiappa
  2022-06-26  9:46 ` steve jenkin
                   ` (2 more replies)
  0 siblings, 3 replies; 38+ messages in thread
From: Noel Chiappa @ 2022-06-26  2:19 UTC (permalink / raw)
  To: tuhs; +Cc: jnc

    > From: Paul Ruizendaal

    > it would seem to me that Sandy had figured out a core problem some 30
    > years before the TCP/IP world would come up with a similar solution. I
    > would not even be surprised if I learned that modern telco routers
    > transparantly set up virtual circuits for tcp traffic.

To fully explore this topic would take a book, which I don't have the energy
to write, and nobody would bother to read, but... 

Anyway, I'm not upon the latest and greatest high-speed routers: I saw some
stuff from one major vendor under NDA about a decade ago, but that's my most
recent - but at that point there was nothing that looked even _vaguely_ like
virtual circuits. (The stuff Craig was alluding to was just about
connectivity for getting bitts from _interface_ to _interface_ - if you don't
have a giant crossbar - which is going to require buffering on each input
anyway - how exactly do you get bits from board A to board Q - a single
shared bus isn't going to do it...)

A problem with anything like VC's in core switches is the growth of per-VC
state - a major high-speed node will have packets from _millions_ of TCP
connections flowing through it at any time. In the late-80's/early-90's - well
over 30 years ago - I came up with an advanced routing architecture called
Nimrod (see RFC-1992, "The Nimrod Routing Architecture"; RFC-1753 may be of
interest too); it had things called 'flows' which were half way between pure
datagrams (i.e. no setup - you just stick the right destination address in the
header and send it off) and VCs (read the RFCs if you want to kow why), and it
went to a lot of trouble to allow flow aggregation in traffic going to core
switches _precisely_ to limit the growth of state in core switches, which
would have traffic from millions of connections going through them.

I have barely begun to even scratch the surface, here.

	Noel

^ permalink raw reply	[flat|nested] 38+ messages in thread
* [TUHS] Research Datakit notes
@ 2022-06-25 23:01 Paul Ruizendaal
  2022-06-25 23:09 ` [TUHS] " Larry McVoy
  2022-06-26  1:41 ` Anthony Martin
  0 siblings, 2 replies; 38+ messages in thread
From: Paul Ruizendaal @ 2022-06-25 23:01 UTC (permalink / raw)
  To: tuhs

Wanted to post my notes as plain text, but the bullets / sub-bullets get lost.

Here is a 2 page PDF with my notes on Research Datakit:

https://www.jslite.net/notes/rdk.pdf

The main takeaway is that connection build-up and tear-down is considerably more expensive than with TCP. The first cost is in the network, which builds up a dedicated path for each connection. Bandwidth is not allocated/reserved, but a path is and routing information is set up at each hop. The other cost is in the relatively verbose switch-host communication in this phase. This compares to the 3 packets exchanged at the hosts’ driver level to set up a TCP connection, with no permanent resources consumed in the network.

In compensation, the cost to use a connection is considerably lower: the routing is known and the host-host link protocol (“URP") can be light-weight, as the network guarantees in-order delivery without duplicates but packets may be corrupted or lost (i.e. as if the connection is a phone line with a modem). No need to deal with packet fragmentation, stream reassembly and congestion storms as in the TCP of the early 80’s.

Doing UDP traffic to a fixed remote host is easily mapped to using URP with no error correction and no flow control. Doing UDP where the remote host is different all the time is not practical on a Datakit network (i.e. a virtual circuit would be set up anyway).

A secondary takeaway is that Research Datakit eventually settled on a three-level ascii namespace: “area/trunk/switch”. On each switch, the hosts would be known by name, and each connection request had a service name as parameter. In an alternate reality we would maybe have used “ca/stclara/mtnview!google!www” to do a search.

^ permalink raw reply	[flat|nested] 38+ messages in thread

end of thread, other threads:[~2022-07-02  2:57 UTC | newest]

Thread overview: 38+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-06-28 15:50 [TUHS] Re: Research Datakit notes Noel Chiappa
2022-06-28 21:32 ` Lawrence Stewart
  -- strict thread matches above, loose matches on Subject: below --
2022-06-27 21:40 Noel Chiappa
2022-06-27 22:40 ` George Michaelson
2022-06-27  0:43 Noel Chiappa
2022-06-27  3:00 ` Erik Fair
2022-06-26  2:19 Noel Chiappa
2022-06-26  9:46 ` steve jenkin
2022-06-26 20:35   ` Erik Fair
2022-06-26 21:53     ` Steve Jenkin
2022-06-26 10:16 ` Paul Ruizendaal via TUHS
2022-06-26 13:07 ` John Cowan
2022-06-26 13:35   ` Larry McVoy
2022-06-26 13:58     ` John Cowan
2022-06-25 23:01 [TUHS] " Paul Ruizendaal
2022-06-25 23:09 ` [TUHS] " Larry McVoy
2022-06-25 23:57   ` Rob Pike
2022-06-26  1:17     ` Paul Ruizendaal
2022-07-02  2:51       ` Grant Taylor via TUHS
2022-07-02  2:57         ` Larry McVoy
2022-06-28 10:38     ` Derek Fawcus
2022-06-28 12:36       ` Rob Pike
2022-06-28 12:45         ` Rob Pike
2022-06-28 13:33           ` Dan Cross
2022-06-28 21:19             ` Lawrence Stewart
2022-06-28 21:34               ` Richard Salz
2022-06-29  6:07                 ` Stuart Remphrey
2022-06-28 16:11           ` Tom Teixeira
2022-06-28 18:28           ` John Floren
2022-06-28 12:47         ` Rich Morin
2022-06-28 13:13           ` Marc Donner
2022-06-28 14:41             ` Clem Cole
2022-06-28 15:54               ` Tom Teixeira
2022-06-28 17:05             ` Adam Thornton
2022-06-28 17:43               ` John Labovitz
2022-06-26  1:41 ` Anthony Martin
2022-06-26  9:52   ` Ralph Corderoy
2022-06-26 11:04   ` Paul Ruizendaal
2022-06-29 20:21   ` Paul Ruizendaal

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).