The Luderer paper on distributed Unix has the following paragraph: "A new special UNIX interprocess communication mechanism is the fifo, which provides communication between unrelated processes by associating a new special file type with a file name. Since remote fifos are legal, they can be used for interprocessor communication between S-UNIX machines or between an S-UNIX machine and an F-UNIX machine.” The paper is from late 1981. Maybe I’m especially mud-eyed today, but I cannot see FIFO’s implemented in V7..V8 or 4.1xBSD. When did FIFO’s become a standard Unix feature? Paul
[-- Attachment #1: Type: text/plain, Size: 1580 bytes --] The first version was from Rand (called "Rand Pipes"). They certainly were available in the mid-70s on Sixth Edition, you have to ask someone like Bruce Borden if they were on Fifth. I think the code is on one of the 'USENIX' tapes in Warren's archives. At this point in time, someone would need to refresh my memory of the details of Rand's implementation compared to what came in the USG systems in the 1980s. For instance, I believe the early versions used mknod(2) to create the "named entity." IIRC early USG did that too, and mkfifo(3) came as part of the POSIX (I have memories of the discussion at a POSIX meeting, but as I say, I've forgotten the details). IIRC there were differences in buffering behavior, flushing, error path between USG's later versions and the original Rand, but I'd have to stare at the code again to remember. On Fri, Mar 6, 2020 at 3:42 PM Paul Ruizendaal <pnr@planet.nl> wrote: > The Luderer paper on distributed Unix has the following paragraph: > > "A new special UNIX interprocess communication mechanism is the fifo, > which provides communication between unrelated processes by associating a > new special file type with a file name. Since remote fifos are legal, they > can be used for interprocessor communication between S-UNIX machines or > between an S-UNIX machine and an F-UNIX machine.” > > The paper is from late 1981. Maybe I’m especially mud-eyed today, but I > cannot see FIFO’s implemented in V7..V8 or 4.1xBSD. When did FIFO’s become > a standard Unix feature? > > Paul > > [-- Attachment #2: Type: text/html, Size: 2301 bytes --]
[-- Attachment #1: Type: text/plain, Size: 2208 bytes --] BTW: My memory is that Joy did not like them for some reason, probably because they were not as sexy as some of the stuff Accent could do (but that's a guess -- I've forgotten). So with 4.2, Joy created Unix domain sockets. BTW: a slow cache refresh is occurring in my brain ... I remember one of the things that there was a lot of arguing/moaning about at the time was the directionality of such a feature. Bruce's hack from the mid-70s was unidirectional and you needed two pipes to go both ways. On Fri, Mar 6, 2020 at 4:06 PM Clem Cole <clemc@ccc.com> wrote: > The first version was from Rand (called "Rand Pipes"). They > certainly were available in the mid-70s on Sixth Edition, you have to ask > someone like Bruce Borden if they were on Fifth. I think the code is on > one of the 'USENIX' tapes in Warren's archives. > > At this point in time, someone would need to refresh my memory of the > details of Rand's implementation compared to what came in the USG systems > in the 1980s. For instance, I believe the early versions used mknod(2) to > create the "named entity." IIRC early USG did that too, and mkfifo(3) > came as part of the POSIX (I have memories of the discussion at a POSIX > meeting, but as I say, I've forgotten the details). > > IIRC there were differences in buffering behavior, flushing, error path > between USG's later versions and the original Rand, but I'd have to stare > at the code again to remember. > > On Fri, Mar 6, 2020 at 3:42 PM Paul Ruizendaal <pnr@planet.nl> wrote: > >> The Luderer paper on distributed Unix has the following paragraph: >> >> "A new special UNIX interprocess communication mechanism is the fifo, >> which provides communication between unrelated processes by associating a >> new special file type with a file name. Since remote fifos are legal, they >> can be used for interprocessor communication between S-UNIX machines or >> between an S-UNIX machine and an F-UNIX machine.” >> >> The paper is from late 1981. Maybe I’m especially mud-eyed today, but I >> cannot see FIFO’s implemented in V7..V8 or 4.1xBSD. When did FIFO’s become >> a standard Unix feature? >> >> Paul >> >> [-- Attachment #2: Type: text/html, Size: 3417 bytes --]
> From: Paul Ruizendaal > The paper is from late 1981. ... When did FIFO's become a > standard Unix feature? Err, V4? :-) At least, that's when pipes arrived (I think - we don't have V4 sources, but there are indications that's when they appeared), and a pipe is a FIFO. RAND ports just allowed (effectively) a pipe to have a name in the file system. The implementation of both is pretty straight-forward. A pipe is just a file which has a maximum length, after which the writer is blocked. A port is just a pipe (it uses the pipe code) whose inode appears in the file system. > From: Clem Cole > I think the code is on one of the 'USENIX' tapes in Warren's archives. Doc is here: https://minnie.tuhs.org//cgi-bin/utree.pl?file=BBN-V6/doc/ipc and sources for all that are here: https://minnie.tuhs.org//cgi-bin/utree.pl?file=BBN-V6/dmr https://minnie.tuhs.org//cgi-bin/utree.pl?file=BBN-V6/ken (port.c is in 'dmr', not 'ken'where it should be). Noel
[-- Attachment #1: Type: text/plain, Size: 2651 bytes --] Clem, you are correct. mknod() was used in the /usr/group standard in 1984. mkfifo() and mkdir() were part of the IEEE Trial Use Standard in 1986, as well as the adopted IEEE Std 1003.1-1988 (POSIX) standard. Heinz On 3/6/2020 1:10 PM, Clem Cole wrote: > BTW: My memory is that Joy did not like them for some reason, > probably because they were not as sexy as some of the stuff Accent > could do (but that's a guess -- I've forgotten). So with 4.2, Joy > created Unix domain sockets. > > BTW: a slow cache refresh is occurring in my brain ... I remember one > of the things that there was a lot of arguing/moaning about at the > time was the directionality of such a feature. Bruce's hack from the > mid-70s was unidirectional and you needed two pipes to go both ways. > > On Fri, Mar 6, 2020 at 4:06 PM Clem Cole <clemc@ccc.com > <mailto:clemc@ccc.com>> wrote: > > The first version was from Rand (called "Rand Pipes"). They > certainly were available in the mid-70s on Sixth Edition, you have > to ask someone like Bruce Borden if they were on Fifth. I think > the code is on one of the 'USENIX' tapes in Warren's archives. > > At this point in time, someone would need to refresh my memory of > the details of Rand's implementation compared to what came in the > USG systems in the 1980s. For instance, I believe the early > versions used mknod(2) to create the "named entity." IIRC early > USG did that too, and mkfifo(3) came as part of the POSIX (I have > memories of the discussion at a POSIX meeting, but as I say, I've > forgotten the details). > > IIRC there were differences in buffering behavior, flushing, error > path between USG's later versions and the original Rand, but I'd > have to stare at the code again to remember. > > On Fri, Mar 6, 2020 at 3:42 PM Paul Ruizendaal <pnr@planet.nl > <mailto:pnr@planet.nl>> wrote: > > The Luderer paper on distributed Unix has the following paragraph: > > "A new special UNIX interprocess communication mechanism is > the fifo, which provides communication between unrelated > processes by associating a new special file type with a file > name. Since remote fifos are legal, they can be used for > interprocessor communication between S-UNIX machines or > between an S-UNIX machine and an F-UNIX machine.” > > The paper is from late 1981. Maybe I’m especially mud-eyed > today, but I cannot see FIFO’s implemented in V7..V8 or > 4.1xBSD. When did FIFO’s become a standard Unix feature? > > Paul > [-- Attachment #2: Type: text/html, Size: 5124 bytes --]
Thanks for the feedback, all.
Rand ports were done in 1977 by Sunshine/Zucker. I’ve only come across Rand Ports in the context of V6 and the Arpa crowd 1977-1981. I’ve never seen a reference to Rand Ports on V7 or later. This of course does not mean that it did not exist.
I’ve dug further, and it would seem that named pipes under the name ‘fifo’ appeared first in SysIII (1980). That matches with Luderer’s remark. It does not seem to exist in the Research editions. It only appears in BSD in the Reno release, 1990. All in all, it would seem that ‘fifo’s were a SysV thing for most of the 80’s, with the BSD lineage using domain sockets instead (as Clem mentioned).
Interestingly, Luderer also refers to a 1978 paper by Steve Holmgren (one of the Arpa Unix authors), suggesting ’sockets’ (in today’s parlance) for interproces communication.
Paul
PS really nobody on the list recalls Luderer's (et al.) distributed Unix and how it related to other work ??
> On 6 Mar 2020, at 23:44, Noel Chiappa <jnc@mercury.lcs.mit.edu> wrote:
>
>
>> From: Paul Ruizendaal
>
>> The paper is from late 1981. ... When did FIFO's become a
>> standard Unix feature?
>
> Err, V4? :-) At least, that's when pipes arrived (I think - we don't have V4
> sources, but there are indications that's when they appeared), and a pipe is a
> FIFO. RAND ports just allowed (effectively) a pipe to have a name in the file
> system.
>
> The implementation of both is pretty straight-forward. A pipe is just a file
> which has a maximum length, after which the writer is blocked. A port is
> just a pipe (it uses the pipe code) whose inode appears in the file system.
>
>> From: Clem Cole
>
>> I think the code is on one of the 'USENIX' tapes in Warren's archives.
>
> Doc is here:
>
> https://minnie.tuhs.org//cgi-bin/utree.pl?file=BBN-V6/doc/ipc
>
> and sources for all that are here:
>
> https://minnie.tuhs.org//cgi-bin/utree.pl?file=BBN-V6/dmr
> https://minnie.tuhs.org//cgi-bin/utree.pl?file=BBN-V6/ken
>
> (port.c is in 'dmr', not 'ken'where it should be).
>
> Noel
[-- Attachment #1: Type: text/plain, Size: 2843 bytes --] below.. in-line On Sat, Mar 7, 2020 at 7:18 AM Paul Ruizendaal <pnr@planet.nl> wrote: > Thanks for the feedback, all. > > Rand ports were done in 1977 by Sunshine/Zucker. I’ve only come across > Rand Ports in the context of V6 and the Arpa crowd 1977-1981. I’ve never > seen a reference to Rand Ports on V7 or later. This of course does not mean > that it did not exist. > Steve Glaser hacked was playing with them at Tektronix in 1979, as he had put them into their V6 system before I got there IIRC. I switched the user code to use Chesson's MPX in V7, which is why I think they never were used much in V7. I've forgotten what Bruce used for UNET - I'm CC'ing him here, hoping to jog his memory., > > I’ve dug further, and it would seem that named pipes under the name ‘fifo’ > appeared first in SysIII (1980). That matches with Luderer’s remark. It > does not seem to exist in the Research editions. It only appears in BSD in > the Reno release, 1990. All in all, it would seem that ‘fifo’s were a SysV > thing for most of the 80’s, with the BSD lineage using domain sockets > instead (as Clem mentioned). > Yes, that's right. And if you were someone like Masscomp or Pyramid trying to thread both systems, we had both in our kernels. > > Interestingly, Luderer also refers to a 1978 paper by Steve Holmgren (one > of the Arpa Unix authors), suggesting ’sockets’ (in today’s parlance) for > interproces communication. > > Paul > > PS really nobody on the list recalls Luderer's (et al.) distributed Unix > and how it related to other work ?? > > > > On 6 Mar 2020, at 23:44, Noel Chiappa <jnc@mercury.lcs.mit.edu> wrote: > > > > > >> From: Paul Ruizendaal > > > >> The paper is from late 1981. ... When did FIFO's become a > >> standard Unix feature? > > > > Err, V4? :-) At least, that's when pipes arrived (I think - we don't > have V4 > > sources, but there are indications that's when they appeared), and a > pipe is a > > FIFO. RAND ports just allowed (effectively) a pipe to have a name in the > file > > system. > > > > The implementation of both is pretty straight-forward. A pipe is just a > file > > which has a maximum length, after which the writer is blocked. A port is > > just a pipe (it uses the pipe code) whose inode appears in the file > system. > > > >> From: Clem Cole > > > >> I think the code is on one of the 'USENIX' tapes in Warren's archives. > > > > Doc is here: > > > > https://minnie.tuhs.org//cgi-bin/utree.pl?file=BBN-V6/doc/ipc > > > > and sources for all that are here: > > > > https://minnie.tuhs.org//cgi-bin/utree.pl?file=BBN-V6/dmr > > https://minnie.tuhs.org//cgi-bin/utree.pl?file=BBN-V6/ken > > > > (port.c is in 'dmr', not 'ken'where it should be). > > > > Noel > > [-- Attachment #2: Type: text/html, Size: 4566 bytes --]
On Sat, Mar 07, 2020 at 01:17:09PM +0100, Paul Ruizendaal wrote: > > Interestingly, Luderer also refers to a 1978 paper by Steve Holmgren (one of the Arpa Unix authors), suggesting ’sockets’ (in today’s parlance) for interproces communication. Could that simply be bleed over of terminology from the ARPAnet / Internet usage, in that "socket" is used to refer to protocol end points? i.e. see these from 1970: https://tools.ietf.org/html/rfc54 https://tools.ietf.org/html/rfc55 https://tools.ietf.org/html/rfc60 DF
Always bemused me that to get a named local I/O connection one ended
up with "Unix domain (what does that even mean?) sockets" rather than
named pipes, especially since sockets are about as natural a Unix
concept as lawn mowers. I've been told, but haven't confirmed, that
early sockets didn't even support read and write. They still don't
support open and close, and never will.
Networks are not intrinsically more special than any other I/O
peripheral, but they have become gilded unicorns mounted on rotating
hovercrafts compared to the I/O devices Unix supported before them.
-rob
On Sun, Mar 8, 2020 at 3:48 AM Derek Fawcus
<dfawcus+lists-tuhs@employees.org> wrote:
>
> On Sat, Mar 07, 2020 at 01:17:09PM +0100, Paul Ruizendaal wrote:
> >
> > Interestingly, Luderer also refers to a 1978 paper by Steve Holmgren (one of the Arpa Unix authors), suggesting ’sockets’ (in today’s parlance) for interproces communication.
>
> Could that simply be bleed over of terminology from the ARPAnet / Internet
> usage, in that "socket" is used to refer to protocol end points?
>
> i.e. see these from 1970:
>
> https://tools.ietf.org/html/rfc54
> https://tools.ietf.org/html/rfc55
> https://tools.ietf.org/html/rfc60
>
> DF
I hear you but sockets are here to stay. Sun tried to get rid of them by going to a STREAMS networking stack (not saying that was in any way a better answer, just different). Didn't work, they had to put sockets back, there was just way too much software written around the socket API. I tried to make a more sane interface and never got to something that handled all the edge cases. Did Plan 9 make it sane? If so, care to say how or point me at something like Masscomp's introduction to network programming? On Sun, Mar 08, 2020 at 01:36:14PM +1100, Rob Pike wrote: > Always bemused me that to get a named local I/O connection one ended > up with "Unix domain (what does that even mean?) sockets" rather than > named pipes, especially since sockets are about as natural a Unix > concept as lawn mowers. I've been told, but haven't confirmed, that > early sockets didn't even support read and write. They still don't > support open and close, and never will. > > Networks are not intrinsically more special than any other I/O > peripheral, but they have become gilded unicorns mounted on rotating > hovercrafts compared to the I/O devices Unix supported before them. > > -rob > > On Sun, Mar 8, 2020 at 3:48 AM Derek Fawcus > <dfawcus+lists-tuhs@employees.org> wrote: > > > > On Sat, Mar 07, 2020 at 01:17:09PM +0100, Paul Ruizendaal wrote: > > > > > > Interestingly, Luderer also refers to a 1978 paper by Steve Holmgren (one of the Arpa Unix authors), suggesting ???sockets??? (in today???s parlance) for interproces communication. > > > > Could that simply be bleed over of terminology from the ARPAnet / Internet > > usage, in that "socket" is used to refer to protocol end points? > > > > i.e. see these from 1970: > > > > https://tools.ietf.org/html/rfc54 > > https://tools.ietf.org/html/rfc55 > > https://tools.ietf.org/html/rfc60 > > > > DF -- --- Larry McVoy lm at mcvoy.com http://www.mcvoy.com/lm
On Sun, 8 Mar 2020, Rob Pike wrote: > Always bemused me that to get a named local I/O connection one ended up > with "Unix domain (what does that even mean?) sockets" rather than named > pipes, especially since sockets are about as natural a Unix concept as > lawn mowers. Indeed... > I've been told, but haven't confirmed, that early sockets didn't even > support read and write. They had their own I/O calls such as send()/recv() (and still do). > They still don't support open and close, and never will. Huh; imagine my surprise when I named a function "shutdown()" because it was called at SIGTERM to clean up... > Networks are not intrinsically more special than any other I/O > peripheral, but they have become gilded unicorns mounted on rotating > hovercrafts compared to the I/O devices Unix supported before them. And that's being polite... They are the worst interface that I have ever seen (including OS/360). At a previous $ORKPLACE there was a library that said simply "I am server on port N" and "I want to contact a service on a.b.c.d on port N"; I wish I'd stole^Wborrowed it when we eventually parted company. -- Dave
Rob Pike <robpike@gmail.com> wrote:
> Networks are not intrinsically more special than any other I/O
> peripheral, but they have become gilded unicorns mounted on rotating
> hovercrafts compared to the I/O devices Unix supported before them.
And another one for the fortune file ... :-)
Arnold
Hi Larry, > Did Plan 9 make it sane? If so, care to say how or point me at something > like Masscomp's introduction to network programming? There's Plan 9's dial(2) and friends. http://man.cat-v.org/plan_9/2/dial The ctl file of a connection takes commands like ‘hangup’. http://man.cat-v.org/plan_9/3/ip The idea of dial() just doing the right thing lives on with Go's https://golang.org/pkg/net/#pkg-overview -- Cheers, Ralph.
Ralph Corderoy <ralph@inputplus.co.uk> wrote: > Hi Larry, > > > Did Plan 9 make it sane? If so, care to say how or point me at something > > like Masscomp's introduction to network programming? > > There's Plan 9's dial(2) and friends. > http://man.cat-v.org/plan_9/2/dial > The ctl file of a connection takes commands like ‘hangup’. > http://man.cat-v.org/plan_9/3/ip > > The idea of dial() just doing the right thing lives on with Go's > https://golang.org/pkg/net/#pkg-overview > > -- > Cheers, Ralph. But more than that is that all files are served via the 9P protocol, even local ones. This is a great intro to Plan 9: https://www.youtube.com/watch?v=6m3GuoaxRNM Just over an hour and a quarter, but well worth the time. Arnold
> Always bemused me that to get a named local I/O connection one ended up with "Unix domain (what does that even mean?) sockets" rather than named pipes, especially since sockets are about as natural a Unix concept as lawn mowers. I've been told, but haven't confirmed, that early sockets didn't even support read and write. They still don't support open and close, and never will. My interest in Unix networking 1975-1985 originally came from wondering how we came up with this alien feeling socket API as the dominant model. The original ideas for this API are in the recently found CSRG tech reports #4 and #3 - which I hope to discuss on this list in Spring. I think we have to distinguish the API and the underlying paradigms. When it comes to the “Arpa” lineage of Unix networking, the original API model was fully within the open-read-write-close framework. See for instance RFC681 and this document: https://minnie.tuhs.org/cgi-bin/utree.pl?file=BBN-Vax-TCP/doc/net.5.P; the entire BBN network API model fits on a few pages of ‘man’ text. In 1975 Arpa Unix, the network name space was integrated with the file name space, by creating a character special file for each network host. This was possible because at that time an Arpa network address was 8 bits, and this fitted in the minor number; when Arpa addresses were expanded to 24 bits soon after this approach was abandoned (but one could think of a mechanism akin to symbolic links that could have continued the practice). One could have an entry for the local host, e.g. “/n/local” or something like that. In my mind, “socket” does not only refer to the sockets API, but also to the concept of a bi-directional, possibly remote, named pipe; ‘named' as in “discoverable by a possibly unrelated process”, i.e. the in file name space, the network name space if different, etc. [aside: I realise (now) that this is a confusing use of the word socket, but I don’t have a better phrase at hand.] In my opinion, it is this concept that has proven strong and durable, much more so than the socket API itself. When viewed in this definition a ‘fifo’ is a limited form of socket: it is unidirectional, local only (although in the 1981 S/F-Unix it wasn’t) and a server process cannot easily distinguish or delegate individual client connections. The Rand Port was better in the sense that it prefixed each client’s data with a header block. > Networks are not intrinsically more special than any other I/O peripheral, but they have become gilded unicorns mounted on rotating hovercrafts compared to the I/O devices Unix supported before them. -rob "Networks are not intrinsically more special than any other I/O peripheral”: that indeed is the paradigm that underlies Spider-Datakit-streams-STREAMS-Plan9, networks are just an I/O peripheral. There is nothing wrong with that paradigm, excellent systems can be built on top of it. The other paradigm is that the network is a (mostly hidden) substrate that carries bidirectional pipes between processes. It would seem to me that there is nothing wrong with that paradigm either and it can be implemented in a “natural Unix” way as well. > On Sun, Mar 8, 2020 at 3:48 AM Derek Fawcus <dfawcus+lists-tuhs at employees.org> wrote: > >> > On Sat, Mar 07, 2020 at 01:17:09PM +0100, Paul Ruizendaal wrote: >> > > >> > > Interestingly, Luderer also refers to a 1978 paper by Steve Holmgren (one of the Arpa Unix authors), suggesting ’sockets’ (in today’s parlance) for interproces communication. >> > >> > Could that simply be bleed over of terminology from the ARPAnet / Internet >> > usage, in that "socket" is used to refer to protocol end points? I meant ’socket’ in the sense that I described above. “Socket” must be one of the most overloaded words in networking. My understanding is that on the Arpanet the “socket number” was what we now would call a "port number”, although I think it was initially meant to identify a user on a host, rather than a service on a host. In the 1980 BBN TCP implementation “socket” is used to mean “ip address”. A year later, Bill Joy uses “socket” as an API call name. >> > >> > i.e. see these from 1970: >> > >> > https://tools.ietf.org/html/rfc54 >> > https://tools.ietf.org/html/rfc55 >> > https://tools.ietf.org/html/rfc60 >> > >> > DF
Rob Pike writes:
> Always bemused me that to get a named local I/O connection one ended up with "Unix domain (what does that even mean?) sockets"
> rather than named pipes, especially since sockets are about as natural a Unix concept as lawn mowers.
I love my Unix lawn mower. The great thing about it is that I don't need a separate
lawn mower for each type of grass, and that I can hook things like mulchers up using
a pipe.
Jon
[-- Attachment #1: Type: text/plain, Size: 1584 bytes --] On 3/6/20 3:44 PM, Noel Chiappa wrote: > a pipe is a FIFO. Hum.... Does that mean that I should be able to replace pipes in my shell commands with a collections of FIFOs? I'm not saying that I would want to. Well, at least not beyond an academic exercise. > RAND ports just allowed (effectively) a pipe to have a name in the > file system. I've got to admit that this statement, particularly the first two words, caused me a lot of trouble. The first few times I read it I kept thinking that RAND ported something from another platform / architecture. I now think it effectively means "(the) port (solution) from RAND". Suffice it to say that many of the emails in this thread didn't make sense with my first, seemingly incorrect, understanding, and made a LOT more sense with my second, seemingly correct, understanding. > The implementation of both is pretty straight-forward. A pipe is > just a file which has a maximum length, after which the writer is > blocked. A port is just a pipe (it uses the pipe code) whose inode > appears in the file system. Intriguing. It is interesting learning the history of things that I've taken for granted for 25 years. This thread aligns quite well with some external reading that I've done trying to learn more about file descriptors and redirection. TL;DR: The redirection symbols (syntax) weren't doing what I thought they were doing. I now believe that they change values of variables used for I/O. Much like DD statements in JCL. -- Grant. . . . unix || die [-- Attachment #2: S/MIME Cryptographic Signature --] [-- Type: application/pkcs7-signature, Size: 4013 bytes --]
Grant Taylor via TUHS <tuhs@minnie.tuhs.org> wrote:
> On 3/6/20 3:44 PM, Noel Chiappa wrote:
> > a pipe is a FIFO.
>
> Hum....
>
> Does that mean that I should be able to replace pipes in my shell
> commands with a collections of FIFOs?
Absolutely:
$ mkfifo the_fifo
$ ls -l the_fifo
prw-rw-r-- 1 arnold arnold 0 Mar 10 09:28 the_fifo
$ echo foo > the_fifo & sleep 1 ; cat the_fifo
[1] 3721
foo
[1]+ Done echo foo > the_fifo
As you stated, not that you'd want to do that, but you can.
Arnold
On 3/9/20 7:22 PM, Grant Taylor via TUHS wrote: > On 3/6/20 3:44 PM, Noel Chiappa wrote: >> a pipe is a FIFO. > > Hum.... > > Does that mean that I should be able to replace pipes in my shell commands > with a collections of FIFOs? Yes, absolutely. Just like a pipe is a FIFO, a FIFO is just a pipe with a name that appears in the file system. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRU chet@case.edu http://tiswww.cwru.edu/~chet/
On Tue, 10 Mar 2020, Chet Ramey wrote:
> Yes, absolutely. Just like a pipe is a FIFO, a FIFO is just a pipe with
> a name that appears in the file system.
So they would still be subject to the same 8-block limit? I haven't
delved into the finer points of named pipes as I rarely use them.
Related question: are they the same as Unix-domain sockets? Oddly enough
I haven't used those much either.
-- Dave
On 3/10/20 4:26 PM, Dave Horsfall wrote: > On Tue, 10 Mar 2020, Chet Ramey wrote: > >> Yes, absolutely. Just like a pipe is a FIFO, a FIFO is just a pipe with a >> name that appears in the file system. > > So they would still be subject to the same 8-block limit? I haven't delved > into the finer points of named pipes as I rarely use them. I believe that on Linux, at least, they have the same capacity limits (64K), but you can modify that. I don't know about BSD. > Related question: are they the same as Unix-domain sockets? Oddly enough I > haven't used those much either. They're similar, I guess ("I am but a shadowy reflection of you."). But I haven't used unix domain sockets, either. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRU chet@case.edu http://tiswww.cwru.edu/~chet/
[-- Attachment #1: Type: text/plain, Size: 2361 bytes --] On 3/10/20 1:29 AM, arnold@skeeve.com wrote: > Absolutely: > > $ mkfifo the_fifo > $ ls -l the_fifo > prw-rw-r-- 1 arnold arnold 0 Mar 10 09:28 the_fifo > $ echo foo > the_fifo & sleep 1 ; cat the_fifo > [1] 3721 > foo > [1]+ Done echo foo > the_fifo > > As you stated, not that you'd want to do that, but you can. Thank you for your reply Arnold. As I was reading your reply, I realized that I did not fully convey the question that I was still mulling over in my head. (More in a moment.) This thread is one of about three things happening in my life that have to do with pipes, FIFOs, and file descriptors. I managed to articulate the simpler of the questions while reading Noel's email. The larger more onerous question is could I leverage exec to alter where file descriptors 0 (STDIN), 1 (STDOUT), and 2 (STDERR) are set to, including changing 1 to the value of a FIFO, and 0 of a subsequent command to also be the value of the FIFO, thus have pipe like behavior between two commands without using a pipe or redirection as in ">". This has also gotten me to wonder about the possibility of having multiple commands output to a file descriptor; 1 / 2 / other, that is input to a separate command. Sort of the opposite of tee, in a manner of speaking. I'll try to articulate: $ mkfifo test.fifo $ exec 3>&1 $ exec 1> test.fifo $ for l in {a..z}; do echo $l; sleep 1; done & $ for L in {A..Z}; do echo $L; sleep 1; done & $ for n in {1..100}; do echo $n; sleep 1; done & $ exec 1>&3 $ cat test.fifo This seems special to me in that I have three processes (for loops) writing into what is effectively the same pipe. After having mulled this over for a few days and typing this out, I realize that the "pipe" is really just a fifo and that in this case the fifo is a named pipe on the file system. I could do the same thing with a file. Historically I would have done the same thing with a file. But now I realize that the file is not required and that I can use a fifo which is in memory and never hits the disk. (Save for creating the name interface to the pipe / fifo.) At least, I think that's all accurate. I would be very eager to learn from anyone who is willing to teach me pointers. :-) -- Grant. . . . unix || die [-- Attachment #2: S/MIME Cryptographic Signature --] [-- Type: application/pkcs7-signature, Size: 4013 bytes --]
[-- Attachment #1: Type: text/plain, Size: 445 bytes --] On 3/10/20 2:26 PM, Dave Horsfall wrote: > Related question: are they the same as Unix-domain sockets? Prior to this thread, I would have said that named pipes / FIFOs are unidirectional and that sockets are bi-directional. But there have been a number of things in this thread that call that understanding into question. Now I need to mull things over and get a new working understanding. -- Grant. . . . unix || die [-- Attachment #2: S/MIME Cryptographic Signature --] [-- Type: application/pkcs7-signature, Size: 4013 bytes --]
Hi Grant. Grant Taylor via TUHS <tuhs@minnie.tuhs.org> wrote: > The larger more onerous question is could I leverage exec to alter where > file descriptors 0 (STDIN), 1 (STDOUT), and 2 (STDERR) are set to, > including changing 1 to the value of a FIFO, and 0 of a subsequent > command to also be the value of the FIFO, thus have pipe like behavior > between two commands without using a pipe or redirection as in ">". There's nothing preventing you from doing that. After the fork() and before the exec(), just close() and dup() the relevant fds in the right order and you're set. > This has also gotten me to wonder about the possibility of having > multiple commands output to a file descriptor; 1 / 2 / other, that is > input to a separate command. Sort of the opposite of tee, in a manner > of speaking. I'll try to articulate: > > $ mkfifo test.fifo > $ exec 3>&1 > $ exec 1> test.fifo > $ for l in {a..z}; do echo $l; sleep 1; done & > $ for L in {A..Z}; do echo $L; sleep 1; done & > $ for n in {1..100}; do echo $n; sleep 1; done & > $ exec 1>&3 > $ cat test.fifo I don't think that this is any different from: (for l in {a..z}; do echo $l; sleep 1; done & for L in {A..Z}; do echo $L; sleep 1; done & for n in {1..100}; do echo $n; sleep 1; done &) | cat which reduces to: (for l in {a..z}; do echo $l; sleep 1; done & for L in {A..Z}; do echo $L; sleep 1; done & for n in {1..100}; do echo $n; sleep 1; done &) > /some/file (You might want to background that whole mess given that the final pipeline will sleep for 100 seconds.) > This seems special to me in that I have three processes (for loops) > writing into what is effectively the same pipe. It's not any different than calling `stty -tostop' and then simply backgrounding the three loops at the terminal. Try it! (This is the beauty of the Unix model, an fd is just a data sink, we don't care where it goes.) > I would be very eager to learn from anyone who is willing to teach me > pointers. :-) At the cost of tooting my own horn, I recommend my book "Linux Programming by Example: The Fundamentals", which, despite the "Linux" in the title, covers basic Unix programming, including file descriptor manipulation of the sort under discussion here. HTH, Arnold