[TUHS] is networking different?

[TUHS] is networking different?

[Marc Donner]

[-- Attachment #1: Type: text/plain, Size: 1467 bytes --]

On June 28 Rob Pike wrote:

"One of the reasons I'm not a networking expert may be relevant here. With
networks, I never found an abstraction to hang my hat on. Unlike with file
systems and files, or even Unix character devices, which provide a level of
remove from the underlying blocks and sectors and so on, the Unix
networking interface always seemed too low-level and fiddly, analogous to
making users write files by managing the blocks and sectors themselves."

I've been ruminating on the question of whether networks are different from
disks (and other devices).  Here are a couple of observations:

1 - Two different packets may take two different paths from the sender to
the receiver.

1a - The transit time for one packet may vary widely from that of the other.

1b - The two packets may arrive in an order different from the order in
which they were transmitted.

(Note - recently I have been reading Bob Gezelter's monograph [and PhD
dissertation] and I've learned that modern high-performance disk systems
behave more like networks in 1a and 1b.)

2 - A packet may never arrive.

3 - Behavior 2 not a sign of hard failure for networks, whereas it is
generally considered so for other I/O devices.

There is probably more to why networks are weird, but these are some of the
big dissonances that seem to me to make Rob's comment resonate so loudly to
me.

Best,

Marc
=====
nygeek.net
mindthegapdialogs.com/home <https://www.mindthegapdialogs.com/home>

[-- Attachment #2: Type: text/html, Size: 3779 bytes --]

[TUHS] Re: is networking different?

[Marc Donner]

[-- Attachment #1: Type: text/plain, Size: 1827 bytes --]

Let's add:

0 - The two endpoints of a network connection may be (and usually are)
under independent control from one another.
=====
nygeek.net
mindthegapdialogs.com/home <https://www.mindthegapdialogs.com/home>


On Sun, Jul 3, 2022 at 4:32 PM Marc Donner <marc.donner@gmail.com> wrote:

> On June 28 Rob Pike wrote:
>
> "One of the reasons I'm not a networking expert may be relevant here. With
> networks, I never found an abstraction to hang my hat on. Unlike with file
> systems and files, or even Unix character devices, which provide a level of
> remove from the underlying blocks and sectors and so on, the Unix
> networking interface always seemed too low-level and fiddly, analogous to
> making users write files by managing the blocks and sectors themselves."
>
> I've been ruminating on the question of whether networks are different
> from disks (and other devices).  Here are a couple of observations:
>
> 1 - Two different packets may take two different paths from the sender to
> the receiver.
>
> 1a - The transit time for one packet may vary widely from that of the
> other.
>
> 1b - The two packets may arrive in an order different from the order in
> which they were transmitted.
>
> (Note - recently I have been reading Bob Gezelter's monograph [and PhD
> dissertation] and I've learned that modern high-performance disk systems
> behave more like networks in 1a and 1b.)
>
> 2 - A packet may never arrive.
>
> 3 - Behavior 2 not a sign of hard failure for networks, whereas it is
> generally considered so for other I/O devices.
>
> There is probably more to why networks are weird, but these are some of
> the big dissonances that seem to me to make Rob's comment resonate so
> loudly to me.
>
> Best,
>
> Marc
> =====
> nygeek.net
> mindthegapdialogs.com/home <https://www.mindthegapdialogs.com/home>
>

[-- Attachment #2: Type: text/html, Size: 4778 bytes --]

[TUHS] Re: is networking different?

[Dan Stromberg]

[-- Attachment #1: Type: text/plain, Size: 2031 bytes --]

On Sun, Jul 3, 2022 at 1:33 PM Marc Donner <marc.donner@gmail.com> wrote:

> On June 28 Rob Pike wrote:
>
> "One of the reasons I'm not a networking expert may be relevant here. With
> networks, I never found an abstraction to hang my hat on. Unlike with file
> systems and files, or even Unix character devices, which provide a level of
> remove from the underlying blocks and sectors and so on, the Unix
> networking interface always seemed too low-level and fiddly, analogous to
> making users write files by managing the blocks and sectors themselves."
>
> I've been ruminating on the question of whether networks are different
> from disks (and other devices).  Here are a couple of observations:
>
> 1 - Two different packets may take two different paths from the sender to
> the receiver.
>
Yes, but it doesn't much matter.  It's a solved problem.

1a - The transit time for one packet may vary widely from that of the other.
>
Yes, but it doesn't much matter.  This too is a solved problem.

1b - The two packets may arrive in an order different from the order in
> which they were transmitted.
>
Yes, but TCP will present the data they describe in order, or an error.

(Note - recently I have been reading Bob Gezelter's monograph [and PhD
> dissertation] and I've learned that modern high-performance disk systems
> behave more like networks in 1a and 1b.)
>
> 2 - A packet may never arrive.
>
TCP will retry lost data, and if transmission continues failing, TCP'll
give an error.

3 - Behavior 2 not a sign of hard failure for networks, whereas it is
> generally considered so for other I/O devices.
>
Network API's and protocols aren't all the same.  You seem to be thinking
of UDP or IP or something.

There is probably more to why networks are weird, but these are some of the
> big dissonances that seem to me to make Rob's comment resonate so loudly to
> me.
>
TCP really isn't that bad, and REST is simpler still.

The chief problem with TCP that people get wrong is:
https://stromberg.dnsalias.org/~strombrg/TCP/

[-- Attachment #2: Type: text/html, Size: 4331 bytes --]

[TUHS] Re: is networking different?

[Richard Salz]

[-- Attachment #1: Type: text/plain, Size: 54 bytes --]

I wanna live in Dan's world, or at least his network.

[-- Attachment #2: Type: text/html, Size: 105 bytes --]

[TUHS] Re.: is networking different?

[Paul Ruizendaal via TUHS]

> On Sun, Jul 3, 2022 at 1:33 PM Marc Donner wrote:
> 
> I've been ruminating on the question of whether networks are different from
> disks (and other devices).  Here are a couple of observations:

[...]

From my perspective most of these things are not unique to networks, they happen with disks and/or terminals. Only out-of-order delivery seems new. However, in many early networking contexts (Spider/Arpanet/Datakit/UUCP) this aspect was not visible to the host (and the same holds for a single segment ethernet).

To me, in some ways networks are like tty’s (e.g. completing i/o can take arbitrarily long, doing a seek() does not make sense), in other ways they are like disks (raw devices are organised into byte streams, they have a name space). Uniquely, they have two end-points, only one of which is local (but pipes come close).

Conceptually, a file system does two things: (i) it organises raw blocks into multiple files; these are the i-nodes and (ii) it provides a name space; these are directories and the namei routine. A network stack certainly does the first: a raw network device is organised into multiple pipe-like connections; depending on the network, it optionally offers a naming service.

With the first aspect one could refer to any file by “major device number, minor device number, i-node number”. This is not very different from referring to a network stream by “network number, host number, port number” in tcp/ip (and in fact this is what bind() and connect() in the sockets API do), or “switch / host / channel” in Datakit. For disks, Unix offers a clean way to organise the name spaces of multiple devices into a unified whole. How to do this with networks is not so easy, prior to the invention of the file system switch.

Early on (Arpanet Unix), it was tried to incorporate host names into a net directory by name (RFC 681) but this is not scalable. Another way would be to have a virtual directory and include only names for active connections. The simple way would be to use a text version of the numeric name as described above - but that is not much of an improvement. Better to have a network variant of namei that looks up symbolic names in a hosts file or in a network naming service. The latter does not look very performant on the hardware of 40 years ago, but it appears to have worked well on the Alto / PuPs network at Xerox PARC.

With the above one could do

open(“/net/inet/org.tuhs.www:80”, O_RDWR | O_STREAM)

to connect to the TUHS web server, and do

open(“/net/inet/any:80”, O_RDWR | O_STREAM | O_CREAT, 0600)

to create a ‘listening’ (rendez-vous) socket.

Paul

[TUHS] Re: Re.: is networking different?

[ron minnich]

Something like what you mention: open(“/net/inet/any:80”, O_RDWR |
O_STREAM | O_CREAT, 0600), is actually to be found in an RFC from
around that time.

You can do that, but the 1980s tried it and it did not end well.
What's it mean, for example, when you rename("/net/harv") to
("/net/google") -- close and reopen socket? (there's a Lost Talk from,
I think, Rob, that addressed this very question)

While it has its flaws, https://9p.io/sys/doc/net/net.html in my view
is the best example to date of how to get it right. Addresses are
strings, just like paths -- well, they *are* paths in fact. Some of
this path-like nature can be seen in the Go net package today.

Oh, and, as regards how the synthetic file system looks: you never,
never, ever, put an address in the pathname. That's important.

Also consider that if you get it right, you can do all the network IO
you want with cat and echo -- people have written telnet in Plan 9
with those two commands. And, if you get it wrong, well -- you get
socat.

On Mon, Jul 4, 2022 at 1:10 PM Paul Ruizendaal via TUHS <tuhs@tuhs.org> wrote:
>
> > On Sun, Jul 3, 2022 at 1:33 PM Marc Donner wrote:
> >
> > I've been ruminating on the question of whether networks are different from
> > disks (and other devices).  Here are a couple of observations:
>
> [...]
>
> From my perspective most of these things are not unique to networks, they happen with disks and/or terminals. Only out-of-order delivery seems new. However, in many early networking contexts (Spider/Arpanet/Datakit/UUCP) this aspect was not visible to the host (and the same holds for a single segment ethernet).
>
> To me, in some ways networks are like tty’s (e.g. completing i/o can take arbitrarily long, doing a seek() does not make sense), in other ways they are like disks (raw devices are organised into byte streams, they have a name space). Uniquely, they have two end-points, only one of which is local (but pipes come close).
>
> Conceptually, a file system does two things: (i) it organises raw blocks into multiple files; these are the i-nodes and (ii) it provides a name space; these are directories and the namei routine. A network stack certainly does the first: a raw network device is organised into multiple pipe-like connections; depending on the network, it optionally offers a naming service.
>
> With the first aspect one could refer to any file by “major device number, minor device number, i-node number”. This is not very different from referring to a network stream by “network number, host number, port number” in tcp/ip (and in fact this is what bind() and connect() in the sockets API do), or “switch / host / channel” in Datakit. For disks, Unix offers a clean way to organise the name spaces of multiple devices into a unified whole. How to do this with networks is not so easy, prior to the invention of the file system switch.
>
> Early on (Arpanet Unix), it was tried to incorporate host names into a net directory by name (RFC 681) but this is not scalable. Another way would be to have a virtual directory and include only names for active connections. The simple way would be to use a text version of the numeric name as described above - but that is not much of an improvement. Better to have a network variant of namei that looks up symbolic names in a hosts file or in a network naming service. The latter does not look very performant on the hardware of 40 years ago, but it appears to have worked well on the Alto / PuPs network at Xerox PARC.
>
> With the above one could do
>
> open(“/net/inet/org.tuhs.www:80”, O_RDWR | O_STREAM)
>
> to connect to the TUHS web server, and do
>
> open(“/net/inet/any:80”, O_RDWR | O_STREAM | O_CREAT, 0600)
>
> to create a ‘listening’ (rendez-vous) socket.
>
> Paul
>
>
>
>
>
>
>
>
>
>
>
>

[TUHS] Re: is networking different?

[Paul Ruizendaal]

Thank you for sharing experiences on what worked and what did not. Much appreciated!

As I’m now too much on a tangent of what might have been versus what once was, I’ll follow up with a PM.

> On 4 Jul 2022, at 23:17, ron minnich <rminnich@gmail.com> wrote:
> 
> Something like what you mention: open(“/net/inet/any:80”, O_RDWR |
> O_STREAM | O_CREAT, 0600), is actually to be found in an RFC from
> around that time.
> 
> You can do that, but the 1980s tried it and it did not end well.
> What's it mean, for example, when you rename("/net/harv") to
> ("/net/google") -- close and reopen socket? (there's a Lost Talk from,
> I think, Rob, that addressed this very question)
> 
> While it has its flaws, https://9p.io/sys/doc/net/net.html in my view
> is the best example to date of how to get it right. Addresses are
> strings, just like paths -- well, they *are* paths in fact. Some of
> this path-like nature can be seen in the Go net package today.
> 
> Oh, and, as regards how the synthetic file system looks: you never,
> never, ever, put an address in the pathname. That's important.
> 
> Also consider that if you get it right, you can do all the network IO
> you want with cat and echo -- people have written telnet in Plan 9
> with those two commands. And, if you get it wrong, well -- you get
> socat.

[TUHS] Re: is networking different?

[John Cowan]

[-- Attachment #1: Type: text/plain, Size: 730 bytes --]

> On 4 Jul 2022, at 23:17, ron minnich <rminnich@gmail.com> wrote:
>
> > You can do that, but the 1980s tried it and it did not end well.
> > What's it mean, for example, when you rename("/net/harv") to
> > ("/net/google") -- close and reopen socket? (there's a Lost Talk from,
> > I think, Rob, that addressed this very question)
>

Why shouldn't it just fail with EXDEV "No cross-device links"?

> Also consider that if you get it right, you can do all the network IO
> > you want with cat and echo -- people have written telnet in Plan 9
> > with those two commands.


Well, not really.  You need at least sed to convert 0xFF to 0xFF 0xFF on
the wire and back again, at least if you are talking to a conformant
implementation.

[-- Attachment #2: Type: text/html, Size: 1658 bytes --]