The Unix Heritage Society mailing list
 help / color / Atom feed
* [TUHS] First appearance of named pipes
@ 2020-03-06 20:41 Paul Ruizendaal
  2020-03-06 21:06 ` Clem Cole
  0 siblings, 1 reply; 24+ messages in thread
From: Paul Ruizendaal @ 2020-03-06 20:41 UTC (permalink / raw)
  To: TUHS main list

The Luderer paper on distributed Unix has the following paragraph:

"A new special UNIX interprocess communication mechanism is the fifo, which provides communication between unrelated processes by associating a new special file type with a file name. Since remote fifos are legal, they can be used for interprocessor communication between S-UNIX machines or between an S-UNIX machine and an F-UNIX machine.”

The paper is from late 1981. Maybe I’m especially mud-eyed today, but I cannot see FIFO’s implemented in V7..V8 or 4.1xBSD. When did FIFO’s become a standard Unix feature?

Paul


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] First appearance of named pipes
  2020-03-06 20:41 [TUHS] First appearance of named pipes Paul Ruizendaal
@ 2020-03-06 21:06 ` Clem Cole
  2020-03-06 21:10   ` Clem Cole
  0 siblings, 1 reply; 24+ messages in thread
From: Clem Cole @ 2020-03-06 21:06 UTC (permalink / raw)
  To: Paul Ruizendaal; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 1580 bytes --]

The first version was from Rand (called "Rand Pipes").   They
certainly were available in the mid-70s on Sixth Edition, you have to ask
someone like Bruce Borden if they were on Fifth.  I think the code is on
one of the 'USENIX' tapes in Warren's archives.

At this point in time, someone would need to refresh my memory of the
details of Rand's implementation compared to what came in the USG systems
in the 1980s.   For instance, I believe the early versions used mknod(2) to
create the "named entity."   IIRC early USG did that too, and mkfifo(3)
came as part of the POSIX (I have memories of the discussion at a POSIX
meeting, but as I say, I've forgotten the details).

IIRC there were differences in buffering behavior, flushing, error path
between USG's later versions and the original Rand, but I'd have to stare
at the code again to remember.

On Fri, Mar 6, 2020 at 3:42 PM Paul Ruizendaal <pnr@planet.nl> wrote:

> The Luderer paper on distributed Unix has the following paragraph:
>
> "A new special UNIX interprocess communication mechanism is the fifo,
> which provides communication between unrelated processes by associating a
> new special file type with a file name. Since remote fifos are legal, they
> can be used for interprocessor communication between S-UNIX machines or
> between an S-UNIX machine and an F-UNIX machine.”
>
> The paper is from late 1981. Maybe I’m especially mud-eyed today, but I
> cannot see FIFO’s implemented in V7..V8 or 4.1xBSD. When did FIFO’s become
> a standard Unix feature?
>
> Paul
>
>

[-- Attachment #2: Type: text/html, Size: 2301 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] First appearance of named pipes
  2020-03-06 21:06 ` Clem Cole
@ 2020-03-06 21:10   ` Clem Cole
  2020-03-07  5:08     ` Heinz Lycklama
  0 siblings, 1 reply; 24+ messages in thread
From: Clem Cole @ 2020-03-06 21:10 UTC (permalink / raw)
  To: Paul Ruizendaal; +Cc: TUHS main list

[-- Attachment #1: Type: text/plain, Size: 2208 bytes --]

BTW:  My memory is that Joy did not like them for some reason,
probably because they were not as sexy as some of the stuff Accent could do
(but that's a guess -- I've forgotten).  So with 4.2, Joy created Unix
domain sockets.

BTW: a slow cache refresh is occurring in my brain ... I remember one of
the things that there was a lot of arguing/moaning about at the time was
the directionality of such a feature.  Bruce's hack from the mid-70s was
unidirectional and you needed two pipes to go both ways.

On Fri, Mar 6, 2020 at 4:06 PM Clem Cole <clemc@ccc.com> wrote:

> The first version was from Rand (called "Rand Pipes").   They
> certainly were available in the mid-70s on Sixth Edition, you have to ask
> someone like Bruce Borden if they were on Fifth.  I think the code is on
> one of the 'USENIX' tapes in Warren's archives.
>
> At this point in time, someone would need to refresh my memory of the
> details of Rand's implementation compared to what came in the USG systems
> in the 1980s.   For instance, I believe the early versions used mknod(2) to
> create the "named entity."   IIRC early USG did that too, and mkfifo(3)
> came as part of the POSIX (I have memories of the discussion at a POSIX
> meeting, but as I say, I've forgotten the details).
>
> IIRC there were differences in buffering behavior, flushing, error path
> between USG's later versions and the original Rand, but I'd have to stare
> at the code again to remember.
>
> On Fri, Mar 6, 2020 at 3:42 PM Paul Ruizendaal <pnr@planet.nl> wrote:
>
>> The Luderer paper on distributed Unix has the following paragraph:
>>
>> "A new special UNIX interprocess communication mechanism is the fifo,
>> which provides communication between unrelated processes by associating a
>> new special file type with a file name. Since remote fifos are legal, they
>> can be used for interprocessor communication between S-UNIX machines or
>> between an S-UNIX machine and an F-UNIX machine.”
>>
>> The paper is from late 1981. Maybe I’m especially mud-eyed today, but I
>> cannot see FIFO’s implemented in V7..V8 or 4.1xBSD. When did FIFO’s become
>> a standard Unix feature?
>>
>> Paul
>>
>>

[-- Attachment #2: Type: text/html, Size: 3417 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] First appearance of named pipes
  2020-03-06 21:10   ` Clem Cole
@ 2020-03-07  5:08     ` Heinz Lycklama
  0 siblings, 0 replies; 24+ messages in thread
From: Heinz Lycklama @ 2020-03-07  5:08 UTC (permalink / raw)
  To: tuhs

[-- Attachment #1: Type: text/plain, Size: 2651 bytes --]

Clem, you are correct. mknod() was used in the /usr/group standard in 1984.
mkfifo() and mkdir() were part of the IEEE Trial Use Standard in 1986, 
as well
as the adopted IEEE Std 1003.1-1988 (POSIX) standard.

Heinz

On 3/6/2020 1:10 PM, Clem Cole wrote:
> BTW:  My memory is that Joy did not like them for some reason, 
> probably because they were not as sexy as some of the stuff Accent 
> could do (but that's a guess -- I've forgotten).  So with 4.2, Joy 
> created Unix domain sockets.
>
> BTW: a slow cache refresh is occurring in my brain ... I remember one 
> of the things that there was a lot of arguing/moaning about at the 
> time was the directionality of such a feature.  Bruce's hack from the 
> mid-70s was unidirectional and you needed two pipes to go both ways.
>
> On Fri, Mar 6, 2020 at 4:06 PM Clem Cole <clemc@ccc.com 
> <mailto:clemc@ccc.com>> wrote:
>
>     The first version was from Rand (called "Rand Pipes").   They
>     certainly were available in the mid-70s on Sixth Edition, you have
>     to ask someone like Bruce Borden if they were on Fifth.  I think
>     the code is on one of the 'USENIX' tapes in Warren's archives.
>
>     At this point in time, someone would need to refresh my memory of
>     the details of Rand's implementation compared to what came in the
>     USG systems in the 1980s.   For instance, I believe the early
>     versions used mknod(2) to create the "named entity."   IIRC early
>     USG did that too, and mkfifo(3) came as part of the POSIX (I have
>     memories of the discussion at a POSIX meeting, but as I say, I've
>     forgotten the details).
>
>     IIRC there were differences in buffering behavior, flushing, error
>     path between USG's later versions and the original Rand, but I'd
>     have to stare at the code again to remember.
>
>     On Fri, Mar 6, 2020 at 3:42 PM Paul Ruizendaal <pnr@planet.nl
>     <mailto:pnr@planet.nl>> wrote:
>
>         The Luderer paper on distributed Unix has the following paragraph:
>
>         "A new special UNIX interprocess communication mechanism is
>         the fifo, which provides communication between unrelated
>         processes by associating a new special file type with a file
>         name. Since remote fifos are legal, they can be used for
>         interprocessor communication between S-UNIX machines or
>         between an S-UNIX machine and an F-UNIX machine.”
>
>         The paper is from late 1981. Maybe I’m especially mud-eyed
>         today, but I cannot see FIFO’s implemented in V7..V8 or
>         4.1xBSD. When did FIFO’s become a standard Unix feature?
>
>         Paul
>


[-- Attachment #2: Type: text/html, Size: 5124 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] First appearance of named pipes
  2020-03-11  2:47     ` Grant Taylor via TUHS
@ 2020-03-11  7:02       ` arnold
  0 siblings, 0 replies; 24+ messages in thread
From: arnold @ 2020-03-11  7:02 UTC (permalink / raw)
  To: tuhs, gtaylor

Hi Grant.

Grant Taylor via TUHS <tuhs@minnie.tuhs.org> wrote:

> The larger more onerous question is could I leverage exec to alter where 
> file descriptors 0 (STDIN), 1 (STDOUT), and 2 (STDERR) are set to, 
> including changing 1 to the value of a FIFO, and 0 of a subsequent 
> command to also be the value of the FIFO, thus have pipe like behavior 
> between two commands without using a pipe or redirection as in ">".

There's nothing preventing you from doing that. After the fork() and
before the exec(), just close() and dup() the relevant fds in the right
order and you're set.

> This has also gotten me to wonder about the possibility of having 
> multiple commands output to a file descriptor; 1 / 2 / other, that is 
> input to a separate command.  Sort of the opposite of tee, in a manner 
> of speaking.  I'll try to articulate:
>
> $ mkfifo test.fifo
> $ exec 3>&1
> $ exec 1> test.fifo
> $ for l in {a..z}; do echo $l; sleep 1; done &
> $ for L in {A..Z}; do echo $L; sleep 1; done &
> $ for n in {1..100}; do echo $n; sleep 1; done &
> $ exec 1>&3
> $ cat test.fifo

I don't think that this is any different from:

	(for l in {a..z}; do echo $l; sleep 1; done &
	 for L in {A..Z}; do echo $L; sleep 1; done &
	 for n in {1..100}; do echo $n; sleep 1; done &) | cat

which reduces to:

	(for l in {a..z}; do echo $l; sleep 1; done &
	 for L in {A..Z}; do echo $L; sleep 1; done &
	 for n in {1..100}; do echo $n; sleep 1; done &) > /some/file

(You might want to background that whole mess given that the final
pipeline will sleep for 100 seconds.)

> This seems special to me in that I have three processes (for loops) 
> writing into what is effectively the same pipe.

It's not any different than calling `stty -tostop' and then simply
backgrounding the three loops at the terminal.  Try it!

(This is the beauty of the Unix model, an fd is just a data sink,
we don't care where it goes.)

> I would be very eager to learn from anyone who is willing to teach me 
> pointers.  :-)

At the cost of tooting my own horn, I recommend my book "Linux Programming
by Example: The Fundamentals", which, despite the "Linux" in the title,
covers basic Unix programming, including file descriptor manipulation
of the sort under discussion here.

HTH,

Arnold

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] First appearance of named pipes
  2020-03-10 20:26     ` Dave Horsfall
  2020-03-10 20:37       ` Chet Ramey
@ 2020-03-11  2:51       ` Grant Taylor via TUHS
  1 sibling, 0 replies; 24+ messages in thread
From: Grant Taylor via TUHS @ 2020-03-11  2:51 UTC (permalink / raw)
  To: tuhs

[-- Attachment #1: Type: text/plain, Size: 445 bytes --]

On 3/10/20 2:26 PM, Dave Horsfall wrote:
> Related question: are they the same as Unix-domain sockets?

Prior to this thread, I would have said that named pipes / FIFOs are 
unidirectional and that sockets are bi-directional.  But there have been 
a number of things in this thread that call that understanding into 
question.  Now I need to mull things over and get a new working 
understanding.



-- 
Grant. . . .
unix || die


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4013 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] First appearance of named pipes
  2020-03-10  7:29   ` arnold
@ 2020-03-11  2:47     ` Grant Taylor via TUHS
  2020-03-11  7:02       ` arnold
  0 siblings, 1 reply; 24+ messages in thread
From: Grant Taylor via TUHS @ 2020-03-11  2:47 UTC (permalink / raw)
  To: tuhs

[-- Attachment #1: Type: text/plain, Size: 2361 bytes --]

On 3/10/20 1:29 AM, arnold@skeeve.com wrote:
> Absolutely:
> 
> 	$ mkfifo the_fifo
> 	$ ls -l  the_fifo
> 	prw-rw-r-- 1 arnold arnold 0 Mar 10 09:28 the_fifo
> 	$ echo foo > the_fifo & sleep 1 ; cat the_fifo
> 	[1] 3721
> 	foo
> 	[1]+  Done                    echo foo > the_fifo
> 
> As you stated, not that you'd want to do that, but you can.

Thank you for your reply Arnold.

As I was reading your reply, I realized that I did not fully convey the 
question that I was still mulling over in my head.  (More in a moment.)

This thread is one of about three things happening in my life that have 
to do with pipes, FIFOs, and file descriptors.  I managed to articulate 
the simpler of the questions while reading Noel's email.

The larger more onerous question is could I leverage exec to alter where 
file descriptors 0 (STDIN), 1 (STDOUT), and 2 (STDERR) are set to, 
including changing 1 to the value of a FIFO, and 0 of a subsequent 
command to also be the value of the FIFO, thus have pipe like behavior 
between two commands without using a pipe or redirection as in ">".

This has also gotten me to wonder about the possibility of having 
multiple commands output to a file descriptor; 1 / 2 / other, that is 
input to a separate command.  Sort of the opposite of tee, in a manner 
of speaking.  I'll try to articulate:

$ mkfifo test.fifo
$ exec 3>&1
$ exec 1> test.fifo
$ for l in {a..z}; do echo $l; sleep 1; done &
$ for L in {A..Z}; do echo $L; sleep 1; done &
$ for n in {1..100}; do echo $n; sleep 1; done &
$ exec 1>&3
$ cat test.fifo

This seems special to me in that I have three processes (for loops) 
writing into what is effectively the same pipe.

After having mulled this over for a few days and typing this out, I 
realize that the "pipe" is really just a fifo and that in this case the 
fifo is a named pipe on the file system.  I could do the same thing with 
a file.  Historically I would have done the same thing with a file.  But 
now I realize that the file is not required and that I can use a fifo 
which is in memory and never hits the disk.  (Save for creating the name 
interface to the pipe / fifo.)

At least, I think that's all accurate.

I would be very eager to learn from anyone who is willing to teach me 
pointers.  :-)



-- 
Grant. . . .
unix || die


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4013 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] First appearance of named pipes
  2020-03-10 20:26     ` Dave Horsfall
@ 2020-03-10 20:37       ` Chet Ramey
  2020-03-11  2:51       ` Grant Taylor via TUHS
  1 sibling, 0 replies; 24+ messages in thread
From: Chet Ramey @ 2020-03-10 20:37 UTC (permalink / raw)
  To: Dave Horsfall, The Eunuchs Hysterical Society

On 3/10/20 4:26 PM, Dave Horsfall wrote:
> On Tue, 10 Mar 2020, Chet Ramey wrote:
> 
>> Yes, absolutely. Just like a pipe is a FIFO, a FIFO is just a pipe with a
>> name that appears in the file system.
> 
> So they would still be subject to the same 8-block limit?  I haven't delved
> into the finer points of named pipes as I rarely use them.

I believe that on Linux, at least, they have the same capacity limits
(64K), but you can modify that. I don't know about BSD.

> Related question: are they the same as Unix-domain sockets?  Oddly enough I
> haven't used those much either.

They're similar, I guess ("I am but a shadowy reflection of you."). But I
haven't used unix domain sockets, either.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
		 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet@case.edu    http://tiswww.cwru.edu/~chet/

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] First appearance of named pipes
  2020-03-10 13:49   ` Chet Ramey
@ 2020-03-10 20:26     ` Dave Horsfall
  2020-03-10 20:37       ` Chet Ramey
  2020-03-11  2:51       ` Grant Taylor via TUHS
  0 siblings, 2 replies; 24+ messages in thread
From: Dave Horsfall @ 2020-03-10 20:26 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On Tue, 10 Mar 2020, Chet Ramey wrote:

> Yes, absolutely. Just like a pipe is a FIFO, a FIFO is just a pipe with 
> a name that appears in the file system.

So they would still be subject to the same 8-block limit?  I haven't 
delved into the finer points of named pipes as I rarely use them.

Related question: are they the same as Unix-domain sockets?  Oddly enough 
I haven't used those much either.

-- Dave

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] First appearance of named pipes
  2020-03-09 23:22 ` Grant Taylor via TUHS
  2020-03-10  7:29   ` arnold
@ 2020-03-10 13:49   ` Chet Ramey
  2020-03-10 20:26     ` Dave Horsfall
  1 sibling, 1 reply; 24+ messages in thread
From: Chet Ramey @ 2020-03-10 13:49 UTC (permalink / raw)
  To: Grant Taylor, tuhs

On 3/9/20 7:22 PM, Grant Taylor via TUHS wrote:
> On 3/6/20 3:44 PM, Noel Chiappa wrote:
>> a pipe is a FIFO.
> 
> Hum....
> 
> Does that mean that I should be able to replace pipes in my shell commands
> with a collections of FIFOs?  

Yes, absolutely. Just like a pipe is a FIFO, a FIFO is just a pipe with
a name that appears in the file system.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
		 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet@case.edu    http://tiswww.cwru.edu/~chet/

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] First appearance of named pipes
  2020-03-09 23:22 ` Grant Taylor via TUHS
@ 2020-03-10  7:29   ` arnold
  2020-03-11  2:47     ` Grant Taylor via TUHS
  2020-03-10 13:49   ` Chet Ramey
  1 sibling, 1 reply; 24+ messages in thread
From: arnold @ 2020-03-10  7:29 UTC (permalink / raw)
  To: tuhs, gtaylor

Grant Taylor via TUHS <tuhs@minnie.tuhs.org> wrote:

> On 3/6/20 3:44 PM, Noel Chiappa wrote:
> > a pipe is a FIFO.
>
> Hum....
>
> Does that mean that I should be able to replace pipes in my shell 
> commands with a collections of FIFOs?

Absolutely:

	$ mkfifo the_fifo
	$ ls -l  the_fifo
	prw-rw-r-- 1 arnold arnold 0 Mar 10 09:28 the_fifo
	$ echo foo > the_fifo & sleep 1 ; cat the_fifo 
	[1] 3721
	foo
	[1]+  Done                    echo foo > the_fifo

As you stated, not that you'd want to do that, but you can.

Arnold

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] First appearance of named pipes
  2020-03-06 22:44 Noel Chiappa
  2020-03-07 12:17 ` Paul Ruizendaal
@ 2020-03-09 23:22 ` Grant Taylor via TUHS
  2020-03-10  7:29   ` arnold
  2020-03-10 13:49   ` Chet Ramey
  1 sibling, 2 replies; 24+ messages in thread
From: Grant Taylor via TUHS @ 2020-03-09 23:22 UTC (permalink / raw)
  To: tuhs

[-- Attachment #1: Type: text/plain, Size: 1584 bytes --]

On 3/6/20 3:44 PM, Noel Chiappa wrote:
> a pipe is a FIFO.

Hum....

Does that mean that I should be able to replace pipes in my shell 
commands with a collections of FIFOs?  I'm not saying that I would want 
to.  Well, at least not beyond an academic exercise.

> RAND ports just allowed (effectively) a pipe to have a name in the 
> file system.

I've got to admit that this statement, particularly the first two words, 
caused me a lot of trouble.  The first few times I read it I kept 
thinking that RAND ported something from another platform / 
architecture.  I now think it effectively means "(the) port (solution) 
from RAND".  Suffice it to say that many of the emails in this thread 
didn't make sense with my first, seemingly incorrect, understanding, and 
made a LOT more sense with my second, seemingly correct, understanding.

> The implementation of both is pretty straight-forward. A pipe is 
> just a file which has a maximum length, after which the writer is 
> blocked. A port is just a pipe (it uses the pipe code) whose inode 
> appears in the file system.

Intriguing.

It is interesting learning the history of things that I've taken for 
granted for 25 years.

This thread aligns quite well with some external reading that I've done 
trying to learn more about file descriptors and redirection.  TL;DR: 
The redirection symbols (syntax) weren't doing what I thought they were 
doing.  I now believe that they change values of variables used for I/O. 
  Much like DD statements in JCL.



-- 
Grant. . . .
unix || die


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4013 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] First appearance of named pipes
  2020-03-08 21:42 Paul Ruizendaal
@ 2020-03-08 22:04 ` Jon Steinhart
  0 siblings, 0 replies; 24+ messages in thread
From: Jon Steinhart @ 2020-03-08 22:04 UTC (permalink / raw)
  To: TUHS main list

Rob Pike writes:
> Always bemused me that to get a named local I/O connection one ended up with "Unix domain (what does that even mean?) sockets" 
> rather than named pipes, especially since sockets are about as natural a Unix concept as lawn mowers.

I love my Unix lawn mower.  The great thing about it is that I don't need a separate
lawn mower for each type of grass, and that I can hook things like mulchers up using
a pipe.

Jon

^ permalink raw reply	[flat|nested] 24+ messages in thread

* [TUHS] First appearance of named pipes
@ 2020-03-08 21:42 Paul Ruizendaal
  2020-03-08 22:04 ` Jon Steinhart
  0 siblings, 1 reply; 24+ messages in thread
From: Paul Ruizendaal @ 2020-03-08 21:42 UTC (permalink / raw)
  To: TUHS main list

> Always bemused me that to get a named local I/O connection one ended up with "Unix domain (what does that even mean?) sockets" rather than named pipes, especially since sockets are about as natural a Unix concept as lawn mowers. I've been told, but haven't confirmed, that early sockets didn't even support read and write. They still don't support open and close, and never will.

My interest in Unix networking 1975-1985 originally came from wondering how we came up with this alien feeling socket API as the dominant model. The original ideas for this API are in the recently found CSRG tech reports #4 and #3 - which I hope to discuss on this list in Spring.

I think we have to distinguish the API and the underlying paradigms.

When it comes to the “Arpa” lineage of Unix networking, the original API model was fully within the open-read-write-close framework. See for instance RFC681 and this document: https://minnie.tuhs.org/cgi-bin/utree.pl?file=BBN-Vax-TCP/doc/net.5.P; the entire BBN network API model fits on a few pages of ‘man’ text.

In 1975 Arpa Unix, the network name space was integrated with the file name space, by creating a character special file for each network host. This was possible because at that time an Arpa network address was 8 bits, and this fitted in the minor number; when Arpa addresses were expanded to 24 bits soon after this approach was abandoned (but one could think of a mechanism akin to symbolic links that could have continued the practice). One could have an entry for the local host, e.g. “/n/local” or something like that.

In my mind, “socket” does not only refer to the sockets API, but also to the concept of a bi-directional, possibly remote, named pipe; ‘named' as in “discoverable by a possibly unrelated process”, i.e. the in file name space, the network name space if different, etc. [aside: I realise (now) that this is a confusing use of the word socket, but I don’t have a better phrase at hand.] In my opinion, it is this concept that has proven strong and durable, much more so than the socket API itself.

When viewed in this definition a ‘fifo’ is a limited form of socket: it is unidirectional, local only (although in the 1981 S/F-Unix it wasn’t) and a server process cannot easily distinguish or delegate individual client connections. The Rand Port was better in the sense that it prefixed each client’s data with a header block.

> Networks are not intrinsically more special than any other I/O peripheral, but they have become gilded unicorns mounted on rotating hovercrafts compared to the I/O devices Unix supported before them. -rob

"Networks are not intrinsically more special than any other I/O peripheral”: that indeed is the paradigm that underlies Spider-Datakit-streams-STREAMS-Plan9, networks are just an I/O peripheral. There is nothing wrong with that paradigm, excellent systems can be built on top of it.

The other paradigm is that the network is a (mostly hidden) substrate that carries bidirectional pipes between processes. It would seem to me that there is nothing wrong with that paradigm either and it can be implemented in a “natural Unix” way as well.

> On Sun, Mar 8, 2020 at 3:48 AM Derek Fawcus <dfawcus+lists-tuhs at employees.org> wrote:
> 
>> > On Sat, Mar 07, 2020 at 01:17:09PM +0100, Paul Ruizendaal wrote: 
>> > > 
>> > > Interestingly, Luderer also refers to a 1978 paper by Steve Holmgren (one of the Arpa Unix authors), suggesting ’sockets’ (in today’s parlance) for interproces communication. 
>> >
>> > Could that simply be bleed over of terminology from the ARPAnet / Internet 
>> > usage, in that "socket" is used to refer to protocol end points?

I meant ’socket’ in the sense that I described above.

“Socket” must be one of the most overloaded words in networking. My understanding is that on the Arpanet the “socket number” was what we now would call a "port number”, although I think it was initially meant to identify a user on a host, rather than a service on a host. In the 1980 BBN TCP implementation “socket” is used to mean “ip address”. A year later, Bill Joy uses “socket” as an API call name.

>> >
>> > i.e. see these from 1970: 
>> >
>> >  https://tools.ietf.org/html/rfc54
>> >  https://tools.ietf.org/html/rfc55
>> >  https://tools.ietf.org/html/rfc60
>> >
>> > DF

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] First appearance of named pipes
  2020-03-08 13:07         ` Ralph Corderoy
@ 2020-03-08 13:25           ` arnold
  0 siblings, 0 replies; 24+ messages in thread
From: arnold @ 2020-03-08 13:25 UTC (permalink / raw)
  To: ralph, lm; +Cc: tuhs

Ralph Corderoy <ralph@inputplus.co.uk> wrote:

> Hi Larry,
>
> > Did Plan 9 make it sane?  If so, care to say how or point me at something
> > like Masscomp's introduction to network programming?
>
> There's Plan 9's dial(2) and friends.
> http://man.cat-v.org/plan_9/2/dial
> The ctl file of a connection takes commands like ‘hangup’.
> http://man.cat-v.org/plan_9/3/ip
>
> The idea of dial() just doing the right thing lives on with Go's
> https://golang.org/pkg/net/#pkg-overview
>
> -- 
> Cheers, Ralph.

But more than that is that all files are served via the 9P protocol,
even local ones.

This is a great intro to Plan 9: https://www.youtube.com/watch?v=6m3GuoaxRNM

Just over an hour and a quarter, but well worth the time.

Arnold

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] First appearance of named pipes
  2020-03-08  2:47       ` Larry McVoy
@ 2020-03-08 13:07         ` Ralph Corderoy
  2020-03-08 13:25           ` arnold
  0 siblings, 1 reply; 24+ messages in thread
From: Ralph Corderoy @ 2020-03-08 13:07 UTC (permalink / raw)
  To: Larry McVoy; +Cc: TUHS main list

Hi Larry,

> Did Plan 9 make it sane?  If so, care to say how or point me at something
> like Masscomp's introduction to network programming?

There's Plan 9's dial(2) and friends.
http://man.cat-v.org/plan_9/2/dial
The ctl file of a connection takes commands like ‘hangup’.
http://man.cat-v.org/plan_9/3/ip

The idea of dial() just doing the right thing lives on with Go's
https://golang.org/pkg/net/#pkg-overview

-- 
Cheers, Ralph.

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] First appearance of named pipes
  2020-03-08  2:36     ` Rob Pike
  2020-03-08  2:47       ` Larry McVoy
  2020-03-08  3:06       ` Dave Horsfall
@ 2020-03-08  7:16       ` arnold
  2 siblings, 0 replies; 24+ messages in thread
From: arnold @ 2020-03-08  7:16 UTC (permalink / raw)
  To: robpike, dfawcus+lists-tuhs; +Cc: tuhs

Rob Pike <robpike@gmail.com> wrote:

> Networks are not intrinsically more special than any other I/O
> peripheral, but they have become gilded unicorns mounted on rotating
> hovercrafts compared to the I/O devices Unix supported before them.

And another one for the fortune file ... :-)

Arnold

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] First appearance of named pipes
  2020-03-08  2:36     ` Rob Pike
  2020-03-08  2:47       ` Larry McVoy
@ 2020-03-08  3:06       ` Dave Horsfall
  2020-03-08  7:16       ` arnold
  2 siblings, 0 replies; 24+ messages in thread
From: Dave Horsfall @ 2020-03-08  3:06 UTC (permalink / raw)
  To: The Eunuchs Hysterical Society

On Sun, 8 Mar 2020, Rob Pike wrote:

> Always bemused me that to get a named local I/O connection one ended up 
> with "Unix domain (what does that even mean?) sockets" rather than named 
> pipes, especially since sockets are about as natural a Unix concept as 
> lawn mowers.

Indeed...

> I've been told, but haven't confirmed, that early sockets didn't even 
> support read and write.

They had their own I/O calls such as send()/recv() (and still do).

> They still don't support open and close, and never will.

Huh; imagine my surprise when I named a function "shutdown()" because
it was called at SIGTERM to clean up...

> Networks are not intrinsically more special than any other I/O 
> peripheral, but they have become gilded unicorns mounted on rotating 
> hovercrafts compared to the I/O devices Unix supported before them.

And that's being polite...  They are the worst interface that I have ever 
seen (including OS/360).  At a previous $ORKPLACE there was a library that 
said simply "I am server on port N" and "I want to contact a service on 
a.b.c.d on port N"; I wish I'd stole^Wborrowed it when we eventually 
parted company.

-- Dave

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] First appearance of named pipes
  2020-03-08  2:36     ` Rob Pike
@ 2020-03-08  2:47       ` Larry McVoy
  2020-03-08 13:07         ` Ralph Corderoy
  2020-03-08  3:06       ` Dave Horsfall
  2020-03-08  7:16       ` arnold
  2 siblings, 1 reply; 24+ messages in thread
From: Larry McVoy @ 2020-03-08  2:47 UTC (permalink / raw)
  To: Rob Pike; +Cc: TUHS main list

I hear you but sockets are here to stay.  Sun tried to get rid of them
by going to a STREAMS networking stack (not saying that was in any way
a better answer, just different).  Didn't work, they had to put sockets
back, there was just way too much software written around the socket API.

I tried to make a more sane interface and never got to something that
handled all the edge cases.

Did Plan 9 make it sane?  If so, care to say how or point me at something
like Masscomp's introduction to network programming?

On Sun, Mar 08, 2020 at 01:36:14PM +1100, Rob Pike wrote:
> Always bemused me that to get a named local I/O connection one ended
> up with "Unix domain (what does that even mean?) sockets" rather than
> named pipes, especially since sockets are about as natural a Unix
> concept as lawn mowers. I've been told, but haven't confirmed, that
> early sockets didn't even support read and write. They still don't
> support open and close, and never will.
> 
> Networks are not intrinsically more special than any other I/O
> peripheral, but they have become gilded unicorns mounted on rotating
> hovercrafts compared to the I/O devices Unix supported before them.
> 
> -rob
> 
> On Sun, Mar 8, 2020 at 3:48 AM Derek Fawcus
> <dfawcus+lists-tuhs@employees.org> wrote:
> >
> > On Sat, Mar 07, 2020 at 01:17:09PM +0100, Paul Ruizendaal wrote:
> > >
> > > Interestingly, Luderer also refers to a 1978 paper by Steve Holmgren (one of the Arpa Unix authors), suggesting ???sockets??? (in today???s parlance) for interproces communication.
> >
> > Could that simply be bleed over of terminology from the ARPAnet / Internet
> > usage, in that "socket" is used to refer to protocol end points?
> >
> > i.e. see these from 1970:
> >
> >   https://tools.ietf.org/html/rfc54
> >   https://tools.ietf.org/html/rfc55
> >   https://tools.ietf.org/html/rfc60
> >
> > DF

-- 
---
Larry McVoy            	     lm at mcvoy.com             http://www.mcvoy.com/lm 

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] First appearance of named pipes
  2020-03-07 16:39   ` Derek Fawcus
@ 2020-03-08  2:36     ` Rob Pike
  2020-03-08  2:47       ` Larry McVoy
                         ` (2 more replies)
  0 siblings, 3 replies; 24+ messages in thread
From: Rob Pike @ 2020-03-08  2:36 UTC (permalink / raw)
  To: Derek Fawcus; +Cc: TUHS main list

Always bemused me that to get a named local I/O connection one ended
up with "Unix domain (what does that even mean?) sockets" rather than
named pipes, especially since sockets are about as natural a Unix
concept as lawn mowers. I've been told, but haven't confirmed, that
early sockets didn't even support read and write. They still don't
support open and close, and never will.

Networks are not intrinsically more special than any other I/O
peripheral, but they have become gilded unicorns mounted on rotating
hovercrafts compared to the I/O devices Unix supported before them.

-rob

On Sun, Mar 8, 2020 at 3:48 AM Derek Fawcus
<dfawcus+lists-tuhs@employees.org> wrote:
>
> On Sat, Mar 07, 2020 at 01:17:09PM +0100, Paul Ruizendaal wrote:
> >
> > Interestingly, Luderer also refers to a 1978 paper by Steve Holmgren (one of the Arpa Unix authors), suggesting ’sockets’ (in today’s parlance) for interproces communication.
>
> Could that simply be bleed over of terminology from the ARPAnet / Internet
> usage, in that "socket" is used to refer to protocol end points?
>
> i.e. see these from 1970:
>
>   https://tools.ietf.org/html/rfc54
>   https://tools.ietf.org/html/rfc55
>   https://tools.ietf.org/html/rfc60
>
> DF

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] First appearance of named pipes
  2020-03-07 12:17 ` Paul Ruizendaal
  2020-03-07 13:29   ` Clem Cole
@ 2020-03-07 16:39   ` Derek Fawcus
  2020-03-08  2:36     ` Rob Pike
  1 sibling, 1 reply; 24+ messages in thread
From: Derek Fawcus @ 2020-03-07 16:39 UTC (permalink / raw)
  To: TUHS main list

On Sat, Mar 07, 2020 at 01:17:09PM +0100, Paul Ruizendaal wrote:
> 
> Interestingly, Luderer also refers to a 1978 paper by Steve Holmgren (one of the Arpa Unix authors), suggesting ’sockets’ (in today’s parlance) for interproces communication.

Could that simply be bleed over of terminology from the ARPAnet / Internet
usage, in that "socket" is used to refer to protocol end points?

i.e. see these from 1970:

  https://tools.ietf.org/html/rfc54
  https://tools.ietf.org/html/rfc55
  https://tools.ietf.org/html/rfc60

DF

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] First appearance of named pipes
  2020-03-07 12:17 ` Paul Ruizendaal
@ 2020-03-07 13:29   ` Clem Cole
  2020-03-07 16:39   ` Derek Fawcus
  1 sibling, 0 replies; 24+ messages in thread
From: Clem Cole @ 2020-03-07 13:29 UTC (permalink / raw)
  To: Paul Ruizendaal; +Cc: TUHS main list, Bruce Borden, Dave Yost

[-- Attachment #1: Type: text/plain, Size: 2843 bytes --]

below.. in-line

On Sat, Mar 7, 2020 at 7:18 AM Paul Ruizendaal <pnr@planet.nl> wrote:

> Thanks for the feedback, all.
>
> Rand ports were done in 1977 by Sunshine/Zucker. I’ve only come across
> Rand Ports in the context of V6 and the Arpa crowd 1977-1981. I’ve never
> seen a reference to Rand Ports on V7 or later. This of course does not mean
> that it did not exist.
>
Steve Glaser hacked was playing with them at Tektronix in 1979, as he had
put them into their V6 system before I got there IIRC.   I switched the
user code to use Chesson's MPX in V7, which is why I think they never were
used much in V7.

I've forgotten what Bruce used for UNET - I'm CC'ing him here, hoping to
jog his memory.,




>
> I’ve dug further, and it would seem that named pipes under the name ‘fifo’
> appeared first in SysIII (1980). That matches with Luderer’s remark. It
> does not seem to exist in the Research editions. It only appears in BSD in
> the Reno release, 1990. All in all, it would seem that ‘fifo’s were a SysV
> thing for most of the 80’s, with the BSD lineage using domain sockets
> instead (as Clem mentioned).
>

Yes, that's right.  And if you were someone like Masscomp or Pyramid trying
to thread both systems, we had both in our kernels.



>
> Interestingly, Luderer also refers to a 1978 paper by Steve Holmgren (one
> of the Arpa Unix authors), suggesting ’sockets’ (in today’s parlance) for
> interproces communication.
>
> Paul
>
> PS really nobody on the list recalls Luderer's (et al.) distributed Unix
> and how it related to other work ??
>
>
> > On 6 Mar 2020, at 23:44, Noel Chiappa <jnc@mercury.lcs.mit.edu> wrote:
> >
> >
> >> From: Paul Ruizendaal
> >
> >> The paper is from late 1981. ... When did FIFO's become a
> >> standard Unix feature?
> >
> > Err, V4? :-) At least, that's when pipes arrived (I think - we don't
> have V4
> > sources, but there are indications that's when they appeared), and a
> pipe is a
> > FIFO. RAND ports just allowed (effectively) a pipe to have a name in the
> file
> > system.
> >
> > The implementation of both is pretty straight-forward. A pipe is just a
> file
> > which has a maximum length, after which the writer is blocked. A port is
> > just a pipe (it uses the pipe code) whose inode appears in the file
> system.
> >
> >> From: Clem Cole
> >
> >> I think the code is on one of the 'USENIX' tapes in Warren's archives.
> >
> > Doc is here:
> >
> >  https://minnie.tuhs.org//cgi-bin/utree.pl?file=BBN-V6/doc/ipc
> >
> > and sources for all that are here:
> >
> > https://minnie.tuhs.org//cgi-bin/utree.pl?file=BBN-V6/dmr
> > https://minnie.tuhs.org//cgi-bin/utree.pl?file=BBN-V6/ken
> >
> > (port.c is in 'dmr', not 'ken'where it should be).
> >
> >       Noel
>
>

[-- Attachment #2: Type: text/html, Size: 4566 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] First appearance of named pipes
  2020-03-06 22:44 Noel Chiappa
@ 2020-03-07 12:17 ` Paul Ruizendaal
  2020-03-07 13:29   ` Clem Cole
  2020-03-07 16:39   ` Derek Fawcus
  2020-03-09 23:22 ` Grant Taylor via TUHS
  1 sibling, 2 replies; 24+ messages in thread
From: Paul Ruizendaal @ 2020-03-07 12:17 UTC (permalink / raw)
  To: TUHS main list

Thanks for the feedback, all.

Rand ports were done in 1977 by Sunshine/Zucker. I’ve only come across Rand Ports in the context of V6 and the Arpa crowd 1977-1981. I’ve never seen a reference to Rand Ports on V7 or later. This of course does not mean that it did not exist.

I’ve dug further, and it would seem that named pipes under the name ‘fifo’ appeared first in SysIII (1980). That matches with Luderer’s remark. It does not seem to exist in the Research editions. It only appears in BSD in the Reno release, 1990. All in all, it would seem that ‘fifo’s were a SysV thing for most of the 80’s, with the BSD lineage using domain sockets instead (as Clem mentioned).

Interestingly, Luderer also refers to a 1978 paper by Steve Holmgren (one of the Arpa Unix authors), suggesting ’sockets’ (in today’s parlance) for interproces communication.

Paul

PS really nobody on the list recalls Luderer's (et al.) distributed Unix and how it related to other work ??


> On 6 Mar 2020, at 23:44, Noel Chiappa <jnc@mercury.lcs.mit.edu> wrote:
> 
> 
>> From: Paul Ruizendaal
> 
>> The paper is from late 1981. ... When did FIFO's become a
>> standard Unix feature?
> 
> Err, V4? :-) At least, that's when pipes arrived (I think - we don't have V4
> sources, but there are indications that's when they appeared), and a pipe is a
> FIFO. RAND ports just allowed (effectively) a pipe to have a name in the file
> system.
> 
> The implementation of both is pretty straight-forward. A pipe is just a file
> which has a maximum length, after which the writer is blocked. A port is
> just a pipe (it uses the pipe code) whose inode appears in the file system.
> 
>> From: Clem Cole
> 
>> I think the code is on one of the 'USENIX' tapes in Warren's archives.
> 
> Doc is here:
> 
>  https://minnie.tuhs.org//cgi-bin/utree.pl?file=BBN-V6/doc/ipc
> 
> and sources for all that are here:
> 
> https://minnie.tuhs.org//cgi-bin/utree.pl?file=BBN-V6/dmr
> https://minnie.tuhs.org//cgi-bin/utree.pl?file=BBN-V6/ken
> 
> (port.c is in 'dmr', not 'ken'where it should be).
> 
> 	Noel


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [TUHS] First appearance of named pipes
@ 2020-03-06 22:44 Noel Chiappa
  2020-03-07 12:17 ` Paul Ruizendaal
  2020-03-09 23:22 ` Grant Taylor via TUHS
  0 siblings, 2 replies; 24+ messages in thread
From: Noel Chiappa @ 2020-03-06 22:44 UTC (permalink / raw)
  To: tuhs; +Cc: jnc

    > From: Paul Ruizendaal

    > The paper is from late 1981. ... When did FIFO's become a
    > standard Unix feature?

Err, V4? :-) At least, that's when pipes arrived (I think - we don't have V4
sources, but there are indications that's when they appeared), and a pipe is a
FIFO. RAND ports just allowed (effectively) a pipe to have a name in the file
system.

The implementation of both is pretty straight-forward. A pipe is just a file
which has a maximum length, after which the writer is blocked. A port is
just a pipe (it uses the pipe code) whose inode appears in the file system.

    > From: Clem Cole

    > I think the code is on one of the 'USENIX' tapes in Warren's archives.

Doc is here:

  https://minnie.tuhs.org//cgi-bin/utree.pl?file=BBN-V6/doc/ipc

and sources for all that are here:

 https://minnie.tuhs.org//cgi-bin/utree.pl?file=BBN-V6/dmr
 https://minnie.tuhs.org//cgi-bin/utree.pl?file=BBN-V6/ken

(port.c is in 'dmr', not 'ken'where it should be).

	Noel

^ permalink raw reply	[flat|nested] 24+ messages in thread

end of thread, back to index

Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-03-06 20:41 [TUHS] First appearance of named pipes Paul Ruizendaal
2020-03-06 21:06 ` Clem Cole
2020-03-06 21:10   ` Clem Cole
2020-03-07  5:08     ` Heinz Lycklama
2020-03-06 22:44 Noel Chiappa
2020-03-07 12:17 ` Paul Ruizendaal
2020-03-07 13:29   ` Clem Cole
2020-03-07 16:39   ` Derek Fawcus
2020-03-08  2:36     ` Rob Pike
2020-03-08  2:47       ` Larry McVoy
2020-03-08 13:07         ` Ralph Corderoy
2020-03-08 13:25           ` arnold
2020-03-08  3:06       ` Dave Horsfall
2020-03-08  7:16       ` arnold
2020-03-09 23:22 ` Grant Taylor via TUHS
2020-03-10  7:29   ` arnold
2020-03-11  2:47     ` Grant Taylor via TUHS
2020-03-11  7:02       ` arnold
2020-03-10 13:49   ` Chet Ramey
2020-03-10 20:26     ` Dave Horsfall
2020-03-10 20:37       ` Chet Ramey
2020-03-11  2:51       ` Grant Taylor via TUHS
2020-03-08 21:42 Paul Ruizendaal
2020-03-08 22:04 ` Jon Steinhart

The Unix Heritage Society mailing list

Archives are clonable: git clone --mirror http://inbox.vuxu.org/tuhs

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://inbox.vuxu.org/vuxu.archive.tuhs


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git