From: Marc Rochkind <mrochkind@gmail.com>
Cc: tuhs@tuhs.org
Subject: [TUHS] Re: Pipes (was Re: After 50 years, what has the Impact of Unix been?)
Date: Thu, 5 Dec 2024 14:50:56 -0700 [thread overview]
Message-ID: <CAOkr1zUTp0zc1kjEXYUOQB+QF0QcwkunvY69HQ0bAwX2wtPXKw@mail.gmail.com> (raw)
In-Reply-To: <CAEoi9W4WAb9JO8VAPpHH9eB0=-EnJr0MySt_YH_eTUdTERqoUw@mail.gmail.com>
[-- Attachment #1.1: Type: text/plain, Size: 3408 bytes --]
Some of the posts in this thread confuse pipes in UNIX shells (the |
symbol) with the pipe system call.
In the shell, how two processes can be connected with a pipe is very
constrained (only one unidirectional pipe). But the pipe system call can be
used to build much more elaborate connections. Back in 1980, when I was
still at Bell Labs, I wrote a shell called 2dsh ("two dimensional shell)
that had a more complex syntax. The memo I wrote, "2DSH—An experimental
shell for connecting processes with multiple data streams", wasn't
published externally, but exists as a Bell Labs memo. I found a reference
here:
https://scholar.google.com/scholar_lookup?title=2DSH%E2%80%94An+experimental+shell+for+connecting+processes+with+multiple+data+streams&author=M.+J.+Rochkind&publication_year=1980
.
Here are two examples from that memo:
[image: image.png]
I stumbled across another paper from 2017 titled "Extending Unix Pipelines
to DAGs," which references my un-published Bell Labs memo. I haven't read
it since I don't subscribe to IEEE Transactions on Computers.
A while ago Doug McIlroy was kind enough to send me a scan of my memo, but
I don't think I'm allowed to publish it here. In that memo, I credit Doug
for coming with a very similar idea around the same time ("A Notation for
Arboreal Plumbing").
Marc Rochkind
On Thu, Dec 5, 2024 at 2:13 PM Dan Cross <crossd@gmail.com> wrote:
> On Thu, Dec 5, 2024 at 3:56 PM <arnold@skeeve.com> wrote:
> > Chet Ramey via TUHS <tuhs@tuhs.org> wrote:
> > > On 12/5/24 10:19 AM, Dan Cross wrote:
> > >
> > > > Unix pipelines, on the other hand, tend to be used in a manner that
> is
> > > > strictly linear, without the fan-out and fan-in capabilities
> described
> > > > by Morrison. Of course, nothing prevents one from building a
> > > > Morrison-style "network" from Unix processes and pipes, though it's
> > > > hard to see how that would work without something like `select`,
> which
> > > > didn't yet exist in 1978. Regardless, Unix still doesn't expose a
> > > > particularly convenient syntax for expressing these sorts of
> > > > constructions at the shell.
> > >
> > > Process substitution is about as close as we can get, but most programs
> > > still process their filename arguments one at a time, beginning to end.
> > >
> > > The canonical process substitution example is
> > >
> > > diff <(old-program-version) <(new-program-version)
> > >
> > > to do simple regression testing.
> >
> > And fanout is simply
> >
> > ... | tee >(pipeline1) >(pipeline2)
>
> And indeed these things are pretty nifty, but don't they generate
> trees, and not arbitrary dags? They don't quite capture the full
> generality of Morrison-style networks since it doesn't seem like
> there's a way to connect process substitution fan-out with fan-in; at
> least, not conveniently.
>
> It is, perhaps, notable that Go allows me to do this sort of thing
> with channels and goroutines, but it has `select` built into the
> language.
>
> Funny, despite using Unix almost daily for over 30 years now, I don't
> think I've ever felt limited by the power of pipelines. On the
> contrary, I've lost count of the times I've felt limited on systems
> that do Not support pipes.
>
> - Dan C.
>
--
*My new email address is mrochkind@gmail.com <mrochkind@gmail.com>*
[-- Attachment #1.2: Type: text/html, Size: 4757 bytes --]
[-- Attachment #2: image.png --]
[-- Type: image/png, Size: 368482 bytes --]
next prev parent reply other threads:[~2024-12-05 21:51 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-12-04 3:17 [TUHS] After 50 years, what has the Impact of Unix been? sjenkin
2024-12-04 13:05 ` [TUHS] " Marc Donner
2024-12-04 13:40 ` William Cheswick
2024-12-04 15:02 ` Rich Salz
2024-12-05 3:08 ` John Levine
2024-12-05 15:19 ` [TUHS] Re: Pipes (was Re: After 50 years, what has the Impact of Unix been?) Dan Cross
2024-12-05 16:00 ` John R Levine
2024-12-05 16:17 ` Heinz Lycklama
2024-12-05 17:06 ` Marc Rochkind
2024-12-05 17:53 ` John Cowan
2024-12-05 18:05 ` John Levine
2024-12-05 17:22 ` Paul Winalski
2024-12-05 18:19 ` Ron Natalie
2024-12-06 2:29 ` Adam Thornton
2024-12-07 20:38 ` Ron Natalie
2024-12-05 16:55 ` Adam Thornton
2024-12-05 17:35 ` Chet Ramey via TUHS
2024-12-05 20:55 ` arnold
2024-12-05 21:12 ` Dan Cross
2024-12-05 21:50 ` Marc Rochkind [this message]
2024-12-05 22:03 ` Warner Losh
2024-12-05 22:19 ` Chet Ramey via TUHS
2024-12-05 23:07 ` Marc Rochkind
2024-12-06 8:16 ` Diomidis Spinellis
2024-12-06 0:46 ` Alexis
2024-12-06 21:46 ` Chet Ramey via TUHS
2024-12-05 23:07 ` arnold
2024-12-06 1:09 ` G. Branden Robinson
2024-12-06 1:31 ` Greg A. Woods
2024-12-06 2:05 ` Steve Nickolas
2024-12-06 16:44 ` arnold
2024-12-05 22:05 ` Bakul Shah via TUHS
2024-12-06 2:02 ` John Levine
2024-12-06 2:21 ` Dan Cross
2024-12-06 16:46 ` arnold
2024-12-06 2:26 Douglas McIlroy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAOkr1zUTp0zc1kjEXYUOQB+QF0QcwkunvY69HQ0bAwX2wtPXKw@mail.gmail.com \
--to=mrochkind@gmail.com \
--cc=tuhs@tuhs.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).