The Unix Heritage Society mailing list
 help / Atom feed
* [TUHS] dmr streams & networking [was: Re: If not Linux, then what?]
@ 2019-08-28  9:17 Paul Ruizendaal
  2019-08-28 10:44 ` Angelo Papenhoff
  0 siblings, 1 reply; 49+ messages in thread
From: Paul Ruizendaal @ 2019-08-28  9:17 UTC (permalink / raw)
  To: TUHS main list


> I find it hard to believe what you remember Dennis saying. The point of
> dmr's streams was to support networking research in the lab and avoid the
> myriad bugs of the mpx interface by stepping around them completely.
> 
> Perhaps it's out of context.
> 
> -rob

> I could be wrong but that's my memory.  What he told me was streams was
> for line disciplines for tty drivers.  That's what I know but you were
> there, I was not.  I'm pretty confused because what Dennis said to me
> was that he did not think streams would work for networking, he thought
> they made sense for a stream but not for a networking connection because
> that had multiple connections coming up through a stream.


There is some contemporary material that gives a bit of context. The quotes are a bit contradictory and perhaps reflect evolving views.

[1]

The original dmr paper (1984) on streams (http://cm.bell-labs.co/who/dmr/st.html) seems to support the no networking view, focussing on terminal handling in its discussion. Also, near the end it says: "Streams are linear connections; by themselves, they support no notion of multiplexing, fan-in or fan-out. [...] It seems likely that a general multiplexing mechanism could help in both cases, but again, I do not yet know how to design it.” This seems to exclude usage for networking, which is typically multiplexed.

[2]

However, now that the V8 sources are available it is clear that the streams mechanism was used (by dmr?) to implement TCP/IP networking. He explains how that tallies with the above quote on multiplexing in a 1985 usenet post: https://groups.google.com/forum/#!topicsearchin/net.unix-wizards/subject$3Astreams/net.unix-wizards/b7W_j_0qASU
The config files in the surviving TUHS V8 source tree actually match with the setup that dmr described in the penultimate paragraph.

If the post by dmr does not immediately appear, click on the 8-10-85 post by 'd...@dutoit.uucp’ to make it fold out. For ease of reference, I’m including the message text below.

<quote>

Steven J. Langdon claimed that without multiplexing one couldn't
do a proper kernel-resident version of TCP/IP in the V8 stream context.
Here's how it's done.
It is still true in our system that stream multiplexing does not occur,
in the sense that every stream connection has (from the point of view
of the formal data structures) exactly two ends, one at a user process,
and the other at a device or another process.  However, this has, in
practice, not turned out to be a problem.  Say you have a hardware
device that hands you packets with a channel (or socket) number buried
inside in some complicated format.  The general scheme to handle the
situation uses both a line discipline (stream filter module) and
associated code that, to the system, looks like a stream device driver
with several minor devices; these have entries in /dev.

A watchdog process opens the underlying real device, and pushes
the stream module.   Arriving packets from the real device
are passed to this module, where they are analyzed,
and then given to the appropriate associated pseudo-device.
Likewise, messages written on the pseudo-device are shunted over to
the line discipline, where they are encoded appropriately and sent
to the real device.  This is where the multiplexing-demultiplexing
occurs; formally, it is outside of the stream structure, because
the data-passing doesn't follow the forward and backward links
of the stream modules.  However, the interfaces of the combined
larger module obey stream rules.

For example, IP works this way:  The IP line discipline looks at the
type field of data arriving from the device, and determines whether the
packet is TCP or UDP or ARP or whatever, and shunts it off to the
stream associated with /dev/ip6 or /dev/ip17 or whatever the numbers
are.

TCP, of course, is multiplexed as well.  So there is a TCP line
discipline, and a bunch of TCP devices; a watchdog process opens
/dev/ip6, and pushes the TCP line discipline; then the TCP packets it
gets are parcelled out to the appropriate /dev/tcpXX device.  Each TCP
device looks like the end of a stream, and may, of course, have other
modules (e.g.  tty processor) inserted in this stream.

UDP sits on top of IP in the same way.

This example is complicated, because (TCP,UDP)/IP is.  However, it
works well.  In particular, the underlying real device can be either an
ethernet or our own Datakit network; the software doesn't care.  For
example, from my machine, I can type "rlogin purdy" and connect to a
Sequent machine running 4.2; the TCP connection goes over Datakit to
machine "research" where it is gatewayed to a local ethernet that purdy
is connected to.

A further generalization (that we haven't made) is in principle easy:
there can be protocol suites other than IP on an Ethernet cable.  So
there could be another layer to separate IP from XNS from Chaosnet, etc.

        Dennis Ritchie

</quote>

Maybe the subtle notion expressed as "formally, it is outside of the stream structure, because the data-passing doesn't follow the forward and backward links of the stream modules.  However, the interfaces of the combined larger module obey stream rules” explains how dmr could talk about streams as being just suitable for line disciplines without meaning to say that they did not have good use in networking.

Paul
 


^ permalink raw reply	[flat|nested] 49+ messages in thread
* Re: [TUHS] dmr streams & networking [was: Re: If not Linux, then what?]
@ 2019-08-28 17:57 Doug McIlroy
  2019-08-28 18:05 ` Adam Thornton
                   ` (4 more replies)
  0 siblings, 5 replies; 49+ messages in thread
From: Doug McIlroy @ 2019-08-28 17:57 UTC (permalink / raw)
  To: tuhs


> How long was research running on a PDP-11 and when did they move to a VAX?

London and Reiser had ported Unix to the VAX, replete with virtual memory, in 1978. By the time v7 was released (1979), Vaxen had become the workhorse machines in Research.

Doug

^ permalink raw reply	[flat|nested] 49+ messages in thread
* Re: [TUHS] dmr streams & networking [was: Re: If not Linux, then what?]
@ 2019-08-28 18:41 Doug McIlroy
  2019-08-28 18:49 ` arnold
  2019-08-28 21:55 ` Rob Pike
  0 siblings, 2 replies; 49+ messages in thread
From: Doug McIlroy @ 2019-08-28 18:41 UTC (permalink / raw)
  To: tuhs

> Doug McIlroy <doug@cs.dartmouth.edu> wrote:
> 
>>
>>> How long was research running on a PDP-11 and when did they move to a VAX?
>>
>> London and Reiser had ported Unix to the VAX, replete with virtual memory, in 1978. By the time v7 was released (1979), Vaxen had become the workhorse machines in Research.
>> 
>> Doug
> 
> So, what's the story on why the London/Reiser port didn't get adapted
> back by Research, and they ended up starting from 4.1 BSD?
> 
> Thanks,
> 
> Arnold


Sorry, what I said about London/Reiser is true, but not the whole story. L/R didn't have demand paging; BSD did.

Doug


^ permalink raw reply	[flat|nested] 49+ messages in thread
* Re: [TUHS] dmr streams & networking [was: Re: If not Linux, then what?]
@ 2019-08-30 20:21 Norman Wilson
  2019-08-30 20:28 ` Larry McVoy
  2019-08-30 20:39 ` Clem Cole
  0 siblings, 2 replies; 49+ messages in thread
From: Norman Wilson @ 2019-08-30 20:21 UTC (permalink / raw)
  To: tuhs

John Reiser did do his own paging system for UNIX 32/V.
I heard about it from friends at Bell Labs ca. 1982-83,
when I was still running systems for physicists at Caltech.
It sounded very interesting, and I would love to have had
my hands on it--page cache unified with buffer cache,
copy-on-write from the start.

The trouble is that Reiser worked in a different group
from the original UNIX crowd, and his management didn't
think his time well spent on that work, so it never got
released.

I remember asking, either during my interview at the Labs
or after I started work there, why the 4.1 kernel had been
chosen instead of Reiser's.  It had to do with maintainability:
there were already people who could come in and hack on the
Berkeley system, as well as more using it and maintaining it,
whereas Reiser's system had become a unicorn.  Nobody in
1127 wanted to maintain a VM system or anything much close
to the VAX hardware.  So the decision was to stick with a
kernel for which someone else would do those things.

Once I'd been there for a year or so and settled in, I found
that I was actually looking after all that stuff, because I
was really interested in it.  (Which seemed to delight a lot
of people.)  Would we have ended up using Reiser's kernel had
I been there a couple of years earlier?  I don't know.

It is in any case a shame that jfr's code never saw the light
of day.  I really hope someone can find it on an old tape
somewhere and we can get it into the archive, if only because
I'd love to look at it.

Norman Wilson
Toronto ON

^ permalink raw reply	[flat|nested] 49+ messages in thread
* Re: [TUHS] dmr streams & networking [was: Re: If not Linux, then what?]
@ 2019-08-31  9:43 Rudi Blom
  2019-08-31 15:41 ` Clem Cole
  0 siblings, 1 reply; 49+ messages in thread
From: Rudi Blom @ 2019-08-31  9:43 UTC (permalink / raw)
  To: tuhs

[-- Attachment #1: Type: text/plain, Size: 161 bytes --]

Whenever I hear UNIX, networking and streams I have to think about this scheme.

Still using this, even on HP-UX 11.31 on Itanium rx-servers

Cheers,
uncle rubl

[-- Attachment #2: Transport Interface.gif --]
[-- Type: image/gif, Size: 8969 bytes --]

^ permalink raw reply	[flat|nested] 49+ messages in thread
* Re: [TUHS] dmr streams & networking [was: Re: If not Linux, then what?]
@ 2019-09-01  2:16 Rudi Blom
  0 siblings, 0 replies; 49+ messages in thread
From: Rudi Blom @ 2019-09-01  2:16 UTC (permalink / raw)
  To: Clem Cole; +Cc: tuhs

I don't remember from where I got the scheme, so it might be general,
DigitalUnix, or HP-UX related. Checking the "HP 9000 networking XTI
programmer's guide" from 1995 there's no diagram.

The application which was initially developed on a SystemV derived
UNIX the Computer division of Philips Electronics had bought, used
TLI. Taken over by DEC we moved to SCO UNIX still using TLI, moving to
XLI on Alpha/Digital Unix.

The nice thing of TLI/XLI is the poll(). A multi-client server can
check a list of file descriptors AND indicate a timeout value for the
poll(). Like in
   	ret_cd = poll(tep->CEPlist, tep->CEPnumb, timeout);

BTW putting in a bit of OSI, on SCO UNIX I use a DEC package which
offers a TLI interface to an OSI TP4/IP stack. Even worked using X.25
as WAN. OSI TP4 and NetBIOS originally bought from Retix.

>Date: Sat, 31 Aug 2019 11:41:40 -0400
>From: Clem Cole <clemc@ccc.com>
>To: Rudi Blom <rudi.j.blom@gmail.com>
>Cc: tuhs <tuhs@minnie.tuhs.org>
>Subject: Re: [TUHS] dmr streams & networking [was: Re: If not Linux,then what?]
>Message-ID:
>        <CAC20D2MJPFoU6r73U9GDaqG+Q7vpH3T7CiDNjgN3D2uyuAJgLQ@mail.gmail.com>
>Content-Type: text/plain; charset="utf-8"
>
>It's the Mentant implementation that HP originally bought.  At LCC we had
>to hacked on it a bit when we put Transparent Network Computing (TNC) stuff
>in HP-UX  [we had full process migration working BTW -- A real shame that
>never shipped].

>On Sat, Aug 31, 2019 at 5:44 AM Rudi Blom <rudi.j.blom@gmail.com> wrote:

>> Whenever I hear UNIX, networking and streams I have to think about this
>> scheme.
>>
>> Still using this, even on HP-UX 11.31 on Itanium rx-servers
>>
>> Cheers,
>> uncle rubl

^ permalink raw reply	[flat|nested] 49+ messages in thread
* Re: [TUHS] dmr streams & networking [was: Re: If not Linux, then what?
@ 2019-09-01  2:36 Rudi Blom
  0 siblings, 0 replies; 49+ messages in thread
From: Rudi Blom @ 2019-09-01  2:36 UTC (permalink / raw)
  To: tuhs

Just to be precise and because I dislike loose ends :-)

Found the XTI to TCP/IP diagram. It's even online!
  https://www.cs.auckland.ac.nz/references/unix/digital/APS2WDTE/TITLE.HTM

Network Programmer's Guide
© Digital Equipment Corporation 1994, 1995, 1996
All Rights Reserved.
Product Version:  Digital UNIX Version 4.0 or higher
March 1996

^ permalink raw reply	[flat|nested] 49+ messages in thread

end of thread, back to index

Thread overview: 49+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-08-28  9:17 [TUHS] dmr streams & networking [was: Re: If not Linux, then what?] Paul Ruizendaal
2019-08-28 10:44 ` Angelo Papenhoff
2019-08-28 17:57 Doug McIlroy
2019-08-28 18:05 ` Adam Thornton
2019-08-28 18:08 ` arnold
2019-08-28 18:27 ` Warner Losh
2019-08-28 18:34   ` Warner Losh
2019-08-28 21:54 ` Rob Pike
2019-08-29  6:43   ` arnold
2019-08-29  7:39     ` Rob Pike
2019-08-29 16:26       ` arnold
2019-08-29  3:29 ` Lawrence Stewart
2019-08-29  4:10   ` Larry McVoy
2019-08-28 18:41 Doug McIlroy
2019-08-28 18:49 ` arnold
2019-08-28 19:03   ` Chet Ramey
2019-08-28 19:05     ` Larry McVoy
2019-08-29 14:58       ` Jason Stevens
2019-08-29 16:25         ` arnold
2019-08-29 16:38           ` Ralph Corderoy
2019-08-29 17:35             ` arnold
2019-08-28 21:55 ` Rob Pike
2019-08-28 22:29   ` George Michaelson
2019-08-28 22:36     ` William Pechter
2019-08-28 23:02       ` Arthur Krewat
2019-08-29  0:11       ` Clem cole
2019-08-29  0:18         ` George Michaelson
2019-08-29  6:27         ` Lars Brinkhoff
2019-08-30 20:21 Norman Wilson
2019-08-30 20:28 ` Larry McVoy
2019-08-30 20:39 ` Clem Cole
2019-08-30 21:52   ` Larry McVoy
2019-08-31  0:58     ` Clem Cole
2019-08-31  1:13       ` Bakul Shah
2019-08-31  2:46         ` Clem cole
2019-08-31  2:57           ` Clem cole
2019-08-31  3:14             ` Gregg Levine
2019-08-31  3:47               ` Clem cole
2019-08-31  3:38             ` Bakul Shah
2019-08-31  5:37             ` Dave Horsfall
2019-08-31 19:03               ` Clem Cole
2019-09-05  5:11                 ` Al Kossow
2019-09-02  8:28               ` Peter Jeremy
2019-09-02 23:26                 ` Dave Horsfall
2019-08-31  3:19           ` Bakul Shah
2019-08-31  9:43 Rudi Blom
2019-08-31 15:41 ` Clem Cole
2019-09-01  2:16 Rudi Blom
2019-09-01  2:36 [TUHS] dmr streams & networking [was: Re: If not Linux, then what? Rudi Blom

The Unix Heritage Society mailing list

Archives are clonable: git clone --mirror http://inbox.vuxu.org/tuhs

Newsgroup available over NNTP:
	nntp://inbox.vuxu.org/vuxu.archive.tuhs


AGPL code for this site: git clone https://public-inbox.org/ public-inbox