From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 20981 invoked from network); 28 Jun 2022 13:35:24 -0000 Received: from minnie.tuhs.org (50.116.15.146) by inbox.vuxu.org with ESMTPUTF8; 28 Jun 2022 13:35:24 -0000 Received: from minnie.tuhs.org (localhost [IPv6:::1]) by minnie.tuhs.org (Postfix) with ESMTP id 582AD40CAE; Tue, 28 Jun 2022 23:34:49 +1000 (AEST) Received: from mail-oa1-f48.google.com (mail-oa1-f48.google.com [209.85.160.48]) by minnie.tuhs.org (Postfix) with ESMTPS id A1B7340CAD for ; Tue, 28 Jun 2022 23:34:45 +1000 (AEST) Received: by mail-oa1-f48.google.com with SMTP id 586e51a60fabf-1013ecaf7e0so17021053fac.13 for ; Tue, 28 Jun 2022 06:34:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=M9ev1XbWBHgN1HClo3SnPUe5HigVBiQ4mgcp2RzDqXU=; b=VuSX4L5iIdgnckMRt7GFOfFAeDahV6XwWkVTT/XqeV7Si/FYPNOJZIfuxOBikQVAjT hmK+2PvAlzg01nsh6NucHL4omgEF5u4gjFYuP4xo1UhfiLPfG4xToIoOyEyUE5DctPM7 pfeX/XJXkHOlaqAWIND27sL61MXZbE9YDmInOvn19oNmwBJ9Rhn67USwk1+/VBcm4oov 8bOPzIQbNWuZEMefyj5YAd82ussHDLvyWA2LWxv4ABWL1cU44mbMoC/beVl1KoJzizXn u9Y4sLtExOx4PmOPFYzAW/vZfn02fGF2c3Dl/hYY0I2X0bvZ6OXWUwZryn7+i8MOI85E P66Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=M9ev1XbWBHgN1HClo3SnPUe5HigVBiQ4mgcp2RzDqXU=; b=UdhwUw2R6KzIVjDLg6k4iqCrHM/GILIFWGPZ5+w4fG8oAXuqNFurzc3ltuUrdEYc+a 0rLCt1jALXjgWAeXPEtRe4EPGxG+0wRU77Cs213+8X7scXiKlGSSnN1nrDT7EfmmuuAM 0WcHiWnjaiOo/UeAb/odIRf0kS6DK2qoyfgOgxNXVVnLtUIk9ir+Nrbw6jJtb+40rW/A Xov98gbxEJELauCfFANeSPlrZfxUmoego5wHclNG+JbLUcB4VcqmINJsvwzCJzYZTLON RbW+2K4Mw20wKpt8FZ6h4d+jkUD3Wx0HoKp/sE+eYtjMPCLXiPHzwuxNJtpY8BnKYzbn HVYg== X-Gm-Message-State: AJIora98JnPfD8AqQlO9UIJTHSPZbX+s50QkTkTpaVCYs+sJFVMs0SzX dZCmzGCZGXMYZsze2t1gOdbOZbFTItkZMDGHV08Xahdr X-Google-Smtp-Source: AGRyM1uW98pLFwRDG7/wMBo65RHxTqKaV3sPa/n2tG6C9EmwApC+A2AoGsNjowlUZDDF5kT7Dp31pkT6UDfmTdoGr84= X-Received: by 2002:a05:6870:4625:b0:fe:4636:cf73 with SMTP id z37-20020a056870462500b000fe4636cf73mr10423420oao.278.1656423224511; Tue, 28 Jun 2022 06:33:44 -0700 (PDT) MIME-Version: 1.0 References: <2803DC51-6CBC-4257-B40C-8A559C27CAE3@planet.nl> <20220625230939.GG19404@mcvoy.com> In-Reply-To: From: Dan Cross Date: Tue, 28 Jun 2022 09:33:08 -0400 Message-ID: To: Rob Pike Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Message-ID-Hash: WVXVTQGLEM3ZDGRIYBVKCVPUBI45PQ6I X-Message-ID-Hash: WVXVTQGLEM3ZDGRIYBVKCVPUBI45PQ6I X-MailFrom: crossd@gmail.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-tuhs.tuhs.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: The Eunuchs Hysterical Society X-Mailman-Version: 3.3.6b1 Precedence: list Subject: [TUHS] Re: Research Datakit notes List-Id: The Unix Heritage Society mailing list Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Tue, Jun 28, 2022 at 8:46 AM Rob Pike wrote: > One of the reasons I'm not a networking expert may be relevant here. > With networks, I never found an abstraction to hang my hat on. Hmm, this raises some design questions. > Unlike with file systems and files, or even Unix character > devices, which provide a level of remove from the underlying > blocks and sectors and so on, the Unix networking interface > always seemed too low-level and fiddly, analogous to making > users write files by managing the blocks and sectors themselves. I can see this. Sockets in particular require filling in abstruse and mostly opaque data structures and then passing pointers to them into the kernel. It's not so terribly onerous once one gets it going, particularly for a streaming protocol like TCP, but eronomically it also doesn't sit particularly well in the larger paradigm. Something like plan9's dial() seems more in line with the Unix model than what Unix actually gives us in terms of TLI/Sockets/whatever. I'll note that 10th Edition had its IPC library that handled some of this. > It could be all sockets' fault, but when I hear networking people talk > about the protocols and stacks and routing and load shedding and > ....my ears droop. I know it's amazing engineering and all that, but > why aren't we allowed to program the I/O without all that fuss? > What makes networks so _different_? A telling detail is that the > original sockets interface had send and recv, not read and write. > From day 1 in Unix land at least, networking was special, and it > remains so, but I fail to see why it needs to be. Of course, the semantics of networking are a little different than the (mostly) stream-oriented file model of Unix, in that datagram protocols must be accommodated somehow and they have metadata in the form of sender/receiver information that accompanies each IO request. How does one model that neatly in the read/write case, except by prepending a header or having another argument? But the same is true of streaming to/from files and file-like things as well. I can't `seek` on a pipe or a serial device, for obvious reasons, but that implies that that model is not completely regular. Similarly, writes to, say, a raw disk device that are not a multiple of the sector size have weird semantics. The best we have done is document this and add it to the oral lore. One may argue that the disk device thing is a special case that is so uncomment and only relevant to extremely low-level systems programs that it doesn't count, but the semantics of seeking are universal: programs that want to work as Unix filters have to accommodate this somehow. In practice this doesn't matter much; most filters just don't seek on their input. Once again I am in awe of how Unix got it right for 90% of use cases, and makes the last 10% possible, even if painful. > It just seems there has to be a better way. Sockets are just so > unpleasant, and the endless nonsense around network > configuration doubly so. No argument there. > Rhetorical questions. I'm not asking or wanting an answer. > I'm happy to remain a greenhorn, oblivious to the wonder. As we continue forward, I wonder how much this matters. We talk about sockets, but how many programmers _actually_ reach for that interface when they want to talk over a network? I'd wager that _most_ of the time now days it's hidden behind a library interface that shields the consumer from the gritty details. Sure, that library probably uses sockets internally, but most people probably never look under that rock. > To adapt a reference some may recognize, I just want to read 5 terabytes. Believe it or not, I actually had Borgmon readability. - Dan C. > On Tue, Jun 28, 2022 at 10:36 PM Rob Pike wrote: >> >> I am not a networking expert. I said that already. The issue could well = be a property more of sockets than TCP/IP itself, but having the switch do = some of the call validation and even maybe authentication (I'm not sure...)= sounds like it takes load off the host. >> >> -rob >> >> >> On Tue, Jun 28, 2022 at 8:39 PM Derek Fawcus wrote: >>> >>> On Sun, Jun 26, 2022 at 09:57:17AM +1000, Rob Pike wrote: >>> > One of the things we liked about Datakit was that the computer didn't= have >>> > to establish the connection before it could reject the call, unlike T= CP/IP >>> > where all validation happens after the connection is made. >>> >>> Nor does TCP, one can send a RST to a SYN, and reject the call before i= t is >>> established. That would then look to the caller just like a non listen= ing >>> endpoint, unless one added data with the RST. >>> >>> So this is really just a consequence of the sockets API, and the curren= t implementations. >>> I've a vague recall of folks suggesting ways to expose that facility vi= a the sockets >>> layer, possibly using setsockopt(), but don't know if anyone ever did i= t. >>> >>> As I recall that TCP capability was actually exposed via the TLI/XTI AP= I, >>> and (for some STREAMS based TCP stacks) it did function. Although I may= be >>> thinking of embedded STREAMS TCP stacks, not unix based stacks. >>> >>> Or by 'connection' are you referring to an end-to-end packet delivery, >>> and that Datakit allowed a closer switch to reject a call before the pa= cket >>> got to the far end? >>> >>> DF