[TUHS] Ethernet in /dev (was Re: Were all of you.. Hippies?)

[TUHS] Ethernet in /dev (was Re: Were all of you.. Hippies?)

[Norman Wilson]

Josh Good:

  Which brings up a question I have: why didn't UNIX implement ethernet
  network interfaces as file names in the filesystem? Was that "novelty" a
  BDS development straying away from AT&T UNIX?

======

Remember that UNIX has long been a family of systems;
it's risky to make blanket statements!

The following is from memory; I haven't looked at this
stuff for a while and am a few kilometers from my
manuals as I type this.  I can dig out the complete
story later if anyone wants it; for now, this is just
the flavour and the existence proof.

Research UNIX, once it supported Ethernet at all, did
so using devices in /dev; e.g. /dev/qe0[0-7] were the
conventional names for the first DEQNA/DELQA device on a
MicroVAX.  There were eight subdevices per physical
device, each a channel that might receive datagrams
of a particular 16-bit type, programmed by an ioctl.

To set up what we now call an IP interface, one did
the following:

a.  Open an unused channel for the proper Ethernet
physical device.  (I think the devices were exclusive-
open to make this easier: open /dev/qe00, then qe01,
and so on until one succeeds.)

b.  Issue the ioctl to set the desired datagram type,
usually 0x800.

c.  Push the IP stream line discipline onto the open file.

d.  Issue an ioctl to inform that IP instance of its
address and netmask.

Now datagrams of the specified type arriving on that
device are fed to the IP subsystem, and the IP
subsystem uses the IP address and mask and possibly
other information to decide which datagrams to route
to that IP instance, which sends them out that physical
device.

I forget how ARP and Ethernet encapsulation fit in.
I know that they were someone more naive early on,
and that in the 10/e systems I can now admit I have
running at home I made things a bit smarter and less
brittle.  But that's the basic architecture.

So how does one actually make, say, a TCP connection?
Another layer of the same sort:

There are devices /dev/ipX, served by an IP device
driver that is part of the IP subsystem.  Originally
minor device X was hard-connected to IP protocol X;
I later changed that to be ioctl-configured as well.

To make TCP usable:

a.  Open /dev/ip6 (old school), or find an unused
/dev/ipX (again they are exclusive-open) and configure
it to accept protocol 6.

b.  Push the TCP stream line discipline onto the
open file.

c.  There are probably things one could then configure,
but I don't remember them offhand.

To make a TCP call, open an unused /dev/tcpNN device;
write something to it (or maybe it was an ioctl) with
the desired destination address; wait for an error or
success.  On success, writes to the file descriptor send
to the network, encapsulated as a TCP stream; reads
receive.

To receive a TCP call, open an unused /dev/tcpNN device,
and write something (or ioctl) to say `I want to listen
on this local port.'  Then read the file.  When a call
arrives, you will read a message saying who's calling,
and what /dev/tcpXX device you should open to accept the
call.

Notice the general scheme: for both TCP and IP (and there
was a primitive UDP at one point, but it has fallen out
of use on my systems), the protocol implementation
comprises:

1.  A line discipline: push it onto devices that will
transport your data.

2.  A device driver: use those devices to send and
receive calls.

The two are inextricably coupled inside the operating
system, of course.

There are all sorts of unfortunate details surrounding
communications; e.g. the TCP code knows it is talking
to IP, and constructs datagrams with partly-filled-in
IP headers.  (It is not clear one can do better than
that in practice, because the protocols themselves really
are linked, but I still think it's unfortunate.)

On the other hand, that the junctions between plumbing
are accessible makes some things very simple.  I wrote
a PPP implementation in user mode, with no kernel
changes: to plug itself into IP, it just pushed the
IP line discipline onto one end of a pipe, and read
and wrote datagrams on the other.  I later extended it
to PPPoE by having it open an Ethernet device set to
the proper datagram types (there is one for data and
another for connection setup).

On the other other hand, there are no permissions on
stream line disciplines, so an untrustworthy person
(if I allowed such on my systems) could push the IP
line discipline onto his own pipe and send whatever
datagrams he liked.  This is decidedly a flaw.

Those familiar with the original stream-system
implementation have already spotted a lesser flaw:
the file descriptor with IP pushed on (or TCP, or
whatever) must remain open; when it is closed,
everything shuts down.  In practice it is usually
useful to have a daemon listening to that file anyway;
that's a good way for the system to report errors or
confusion.  In practice, TCP incalls and outcalls
all went through a special daemon anyway, so that
programs didn't have to be full of TCP-specific crap;
that's what Dave Presotto's `connection server' is
all about.

Norman Wilson
Toronto ON

[TUHS] Ethernet in /dev (was Re: Were all of you.. Hippies?)

[Ron Natalie]

It's a violation of the network layering concept to require or even allow
the user to bind the application data streams to a physical device.


-----Original Message-----
From: TUHS [mailto:tuhs-bounces@minnie.tuhs.org] On Behalf Of Norman Wilson
Sent: Thursday, March 30, 2017 11:55 AM
To: tuhs at tuhs.org
Subject: [TUHS] Ethernet in /dev (was Re: Were all of you.. Hippies?)

Josh Good:

  Which brings up a question I have: why didn't UNIX implement ethernet
  network interfaces as file names in the filesystem? Was that "novelty" a
  BDS development straying away from AT&T UNIX?

[TUHS] Ethernet in /dev (was Re: Were all of you.. Hippies?)

[ron minnich]

I have a slightly different take on all this, namely, that Unix as it was
worked, and that innovations in interfaces moved to another platform.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170330/f3afe5fb/attachment.html>

[TUHS] Ethernet in /dev (was Re: Were all of you.. Hippies?)

[Joerg Schilling]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1438 bytes --]

Norman Wilson <norman at oclsc.org> wrote:

> Josh Good:
>
>   Which brings up a question I have: why didn't UNIX implement ethernet
>   network interfaces as file names in the filesystem? Was that "novelty" a
>   BDS development straying away from AT&T UNIX?
>
> ======
>
> Remember that UNIX has long been a family of systems;
> it's risky to make blanket statements!
>
> The following is from memory; I haven't looked at this
> stuff for a while and am a few kilometers from my
> manuals as I type this.  I can dig out the complete
> story later if anyone wants it; for now, this is just
> the flavour and the existence proof.
>
> Research UNIX, once it supported Ethernet at all, did
> so using devices in /dev; e.g. /dev/qe0[0-7] were the

This is done on a UNIX implementation that uses STREAMS.

SVr4 is such a UNIX.

lrwxrwxrwx   1 root     root          29 Mrz  3  2009 /dev/bge -> ../devices/pseudo/clone at 0:bge
lrwxrwxrwx   1 root     root          38 Mrz  3  2009 /dev/bge0 -> ../devices/pci at 0,0/pci1462,6710 at b:bge0

/dev/bge is a clone device and allows to access and plumb further devices.
/dev/bge0 is the first Broadcom Gigabit Ethernet device.

Jörg

-- 
 EMail:joerg at schily.net                  (home) Jörg Schilling D-13353 Berlin
       joerg.schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.org/private/ http://sourceforge.net/projects/schilytools/files/

[TUHS] Ethernet in /dev (was Re: Were all of you.. Hippies?)

[arnold]

More off topic ...

Norman Wilson <norman at oclsc.org> wrote:

> I did an implementation of the late lamented Coraid's AoE protocol that
> took advantage of that.

Coraid and AoE are alive and well, reincarnated by Brantley Coile under
a slightly different umbrella. But he owns the IP and the name
(once again) and is supporting any Coraid customer who needs it.

FWIW,

Arnold

[TUHS] Ethernet in /dev (was Re: Were all of you.. Hippies?)

[Joerg Schilling]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1132 bytes --]

Norman Wilson <norman at oclsc.org> wrote:

> But is IP done that way in SVr4 (or at least in Solaris, its
> most-visible descendant)?  I had the impression that the
> IP stack was more like the BSD one, with everything coupled
> together within the kernel and a fundamentally socket interface.

The code has been implemented in a way that mainly differs in that "ifconfig" 
needs to be called with "ifconfig <ifname> plumb" before you can use the 
interface. 

In former times, sockets have been emulated in userland, but this prevents a 
socket filedescriptor returned from a dup(sockfd) to work as expected. As a 
result, POSIX later required sockets to be an integral part of the 
implementation and sockets went into the kernel again.

BTW: there was a paper from Sun that explains that sockets are faster then 
using the STREAMS based t_open() and friends.

Jörg

-- 
 EMail:joerg at schily.net                  (home) Jörg Schilling D-13353 Berlin
       joerg.schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.org/private/ http://sourceforge.net/projects/schilytools/files/

[TUHS] Ethernet in /dev (was Re: Were all of you.. Hippies?)

[Norman Wilson]

Joerg Schilling:

  This is done on a UNIX implementation that uses STREAMS.

  SVr4 is such a UNIX.

======

I know that SVr4 has STREAMS (somewhat more elaborate than
the original stream I/O system, but the same principles),
and knew (though I'd forgotten) that Ethernet devices are
stream-capable.  I did an implementation of the late
lamented Coraid's AoE protocol that took advantage of that.
Somewhat like the Research IP implementation, in fact:
there was an AoE line discipline to be pushed onto an
Ethernet device, coupled to devices in /dev/dsk and /dev/rdsk.

But is IP done that way in SVr4 (or at least in Solaris, its
most-visible descendant)?  I had the impression that the
IP stack was more like the BSD one, with everything coupled
together within the kernel and a fundamentally socket interface.

I've never actually looked at the code, though.

Norman Wilson
Toronto ON

[TUHS] Ethernet in /dev (was Re: Were all of you.. Hippies?)

[Norman Wilson]

Ron Natalie:

  It's a violation of the network layering concept to require or even allow
  the user to bind the application data streams to a physical device.

=====

Oh, come on.

If you mean an applications programmer shouldn't have to wallow
in low-level network details, I agree.  That's one of the reasons
I think the socket interface now standard in nearly every UNIX
is an abomination.  It reminds me of the binary data structures
you had to assemble just to open a file in TOPS-10, only
ten times worse.

But if you mean it's a violation of layering for the kernel to
expose the pieces and let user-mode code to the work, I strongly
disagree.  By that argument, the very idea of inetd is an abomination.
Possibly even the ifconfig command.  And don't even get the
hypothetical you started on microkernels.  User-mode code for device
drivers or file systems?  Outrageous violation of layering!  Send
in the New Jersey Inquisition!!

Or perhaps you misunderstand how it all works.  Device files
for Ethernet devices, /dev/ip*, and so on are like those for
disk devices: you could allow anyone to read and write them,
but in practice you probably wouldn't: you'd restrict access
to the super-user and perhaps a special group to admit some
sort of privilege reduction (like the group allowed to read
disks on some systems, or to read /dev/kmem).

That parts of the stack are assembled in user mode is a feature,
not a bug.

The one glaring flaw, as I said, is that no permissions are
checked when pushing a stream line discipline onto a file.
I think that happened because when Dennis first wrote the
code, he was thinking about modules to implement canonical-tty
semantics, or to invoke the very-different Datakit networking
model.  It's a fundamental flaw, though.  I have had thoughts
about fixing it, but never enough time nor enough motivation.
(My technical mind is pretty much filled up by what I am
paid to do these days; I haven't done much hobby computing
in years.)

Norman Wilson
Toronto ON