[TUHS] Re: Research Datakit notes

[Dan Cross <crossd@gmail.com>]

On Tue, Jun 28, 2022 at 8:46 AM Rob Pike <robpike@gmail.com> wrote:
> One of the reasons I'm not a networking expert may be relevant here.
> With networks, I never found an abstraction to hang my hat on.

Hmm, this raises some design questions.

> Unlike with file systems and files, or even Unix character
> devices, which provide a level of remove from the underlying
> blocks and sectors and so on, the Unix networking interface
> always seemed too low-level and fiddly, analogous to making
> users write files by managing the blocks and sectors themselves.

I can see this. Sockets in particular require filling in abstruse
and mostly opaque data structures and then passing pointers
to them into the kernel. It's not so terribly onerous once one
gets it going, particularly for a streaming protocol like TCP,
but eronomically it also doesn't sit particularly well in the larger
paradigm. Something like plan9's dial() seems more in line with
the Unix model than what Unix actually gives us in terms of
TLI/Sockets/whatever. I'll note that 10th Edition had its IPC
library that handled some of this.

> It could be all sockets' fault, but when I hear networking people talk
> about the protocols and stacks and routing and load shedding and
> ....my ears droop. I know it's amazing engineering and all that, but
> why aren't we allowed to program the I/O without all that fuss?
> What makes networks so _different_? A telling detail is that the
> original sockets interface had send and recv, not read and write.
> From day 1 in Unix land at least, networking was special, and it
> remains so, but I fail to see why it needs to be.

Of course, the semantics of networking are a little different than
the (mostly) stream-oriented file model of Unix, in that datagram
protocols must be accommodated somehow and they have metadata
in the form of sender/receiver information that accompanies each
IO request. How does one model that neatly in the read/write case,
except by prepending a header or having another argument?

But the same is true of streaming to/from files and file-like things
as well. I can't `seek` on a pipe or a serial device, for obvious
reasons, but that implies that that model is not completely regular.
Similarly, writes to, say, a raw disk device that are not a multiple
of the sector size have weird semantics. The best we have done
is document this and add it to the oral lore.

One may argue that the disk device thing is a special case that
is so uncomment and only relevant to extremely low-level systems
programs that it doesn't count, but the semantics of seeking are
universal: programs that want to work as Unix filters have to
accommodate this somehow. In practice this doesn't matter much;
most filters just don't seek on their input.

Once again I am in awe of how Unix got it right for 90% of use
cases, and makes the last 10% possible, even if painful.

> It just seems there has to be a better way. Sockets are just so
> unpleasant, and the endless nonsense around network
> configuration doubly so.

No argument there.

> Rhetorical questions. I'm not asking or wanting an answer.
> I'm happy to remain a greenhorn, oblivious to the wonder.

As we continue forward, I wonder how much this matters.  We
talk about sockets, but how many programmers _actually_ reach
for that interface when they want to talk over a network?  I'd
wager that _most_ of the time now days it's hidden behind a
library interface that shields the consumer from the gritty details.
Sure, that library probably uses sockets internally, but most
people probably never look under that rock.

> To adapt a reference some may recognize, I just want to read 5 terabytes.

Believe it or not, I actually had Borgmon readability.

        - Dan C.


> On Tue, Jun 28, 2022 at 10:36 PM Rob Pike <robpike@gmail.com> wrote:
>>
>> I am not a networking expert. I said that already. The issue could well be a property more of sockets than TCP/IP itself, but having the switch do some of the call validation and even maybe authentication (I'm not sure...) sounds like it takes load off the host.
>>
>> -rob
>>
>>
>> On Tue, Jun 28, 2022 at 8:39 PM Derek Fawcus <dfawcus+lists-tuhs@employees.org> wrote:
>>>
>>> On Sun, Jun 26, 2022 at 09:57:17AM +1000, Rob Pike wrote:
>>> > One of the things we liked about Datakit was that the computer didn't have
>>> > to establish the connection before it could reject the call, unlike TCP/IP
>>> > where all validation happens after the connection is made.
>>>
>>> Nor does TCP, one can send a RST to a SYN, and reject the call before it is
>>> established.  That would then look to the caller just like a non listening
>>> endpoint, unless one added data with the RST.
>>>
>>> So this is really just a consequence of the sockets API, and the current implementations.
>>> I've a vague recall of folks suggesting ways to expose that facility via the sockets
>>> layer, possibly using setsockopt(), but don't know if anyone ever did it.
>>>
>>> As I recall that TCP capability was actually exposed via the TLI/XTI API,
>>> and (for some STREAMS based TCP stacks) it did function. Although I may be
>>> thinking of embedded STREAMS TCP stacks, not unix based stacks.
>>>
>>> Or by 'connection' are you referring to an end-to-end packet delivery,
>>> and that Datakit allowed a closer switch to reject a call before the packet
>>> got to the far end?
>>>
>>> DF