From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI autolearn=ham autolearn_force=no version=3.4.4 Received: from minnie.tuhs.org (minnie.tuhs.org [50.116.15.146]) by inbox.vuxu.org (Postfix) with ESMTP id DEB4D22D71 for ; Fri, 6 Dec 2024 03:22:34 +0100 (CET) Received: from minnie.tuhs.org (localhost [IPv6:::1]) by minnie.tuhs.org (Postfix) with ESMTP id 4F42A4274C; Fri, 6 Dec 2024 12:22:30 +1000 (AEST) Received: from mail-lj1-x230.google.com (mail-lj1-x230.google.com [IPv6:2a00:1450:4864:20::230]) by minnie.tuhs.org (Postfix) with ESMTPS id 73D7D4274B for ; Fri, 6 Dec 2024 12:22:27 +1000 (AEST) Received: by mail-lj1-x230.google.com with SMTP id 38308e7fff4ca-2ffc016f301so13944321fa.1 for ; Thu, 05 Dec 2024 18:22:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733451746; x=1734056546; darn=tuhs.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=H3tfMViTKpkP40cVnLQ1KzFJEdTR5uALr+0TaKavBas=; b=h/e+bmExKxRbyaQlLHwUC8QqcFvSb4tYr/7QxM/4Fabd6bOGPWahY3W4uzV8PQiGnb 6fFQC4mfDbdpY9RXbqQAc4wMwDHAAR4ZuIlKkyDjc6cTbw33wd+vrmdWrv5yOVGRuDZF bU6nT1VVhB5ydXHktYAYd9I6+bHobLLQlyB0ShPENmHDENoJ6/nWSPeK/wOAw9V7wJc/ a0nOOZATngovFPz6cudL3rYRhSlKiOHSHRVJsjrc3t2LdiLvRbMhIRW4uL1jT3ooZ5m/ x7NeG7nqcNtzzfLh+lAwpOl3HItT4CosTPgzuRT1vFkH5vE7HI5l8OX7sY3Gf+JSTlkN NZWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733451746; x=1734056546; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=H3tfMViTKpkP40cVnLQ1KzFJEdTR5uALr+0TaKavBas=; b=weoyihUY5jXa2EX7jIp2+6OAgeXmKdEt0bllekIIZTJoynOOsWd4lrqkwgy/KVe1UN lqBylIUaMMwBcqg7N4i/+t6FgwOsQFblYbyVbugFtzya09us3QKB7iGffupvG7cC1owo OaAX4JDJUs7Emk/9rMAVpbDDzTX7Bt4/BWpoZENX6KHdUUWvdNhh+DVEUg0rw6npgHv0 Tnn6JLSh82x00twvIN1HzmB/he5Js24cEKI7AyoOWvPnWh+4RddRoIm/Jk3aKxkUHjbh cU2HaxPhKYybyUskS8mWM8hhN+QDGHTCt4Nw7UhO7BRaPoQYaJGye8iYYGqNCh86ADtH jUPw== X-Gm-Message-State: AOJu0Yx7Vq3BrzJ7kGz5Cs3JtPZNudjr1EsfaLEHphV5d5l5T5z/CCRC 4y9ikAJhAyUCDcQjuZ9sRh+TaXFcDZ0Gl3uN2SDaL9licAFL2tBzPCPyfdDgEimy7XaKUq9CC2O QHcXAn8sPwQWWnzWWbXiHG2f6jDYbTGB6 X-Gm-Gg: ASbGncvTaMU+WdfaRUTfkyRqpN+52/TUiR2oE9WU6IBQV9m46rvKuwlj+qWvjLZCTZu Pn97sw3kww9H/zFOZ06q5kuONASnwZ2QAFpfEh/tMh+tbrn4BNVSurCECzU+UCux6 X-Google-Smtp-Source: AGHT+IFgIuXwvh8V39ov7vGBfEQ/BtkFfCmcbWwI4NnlK3vYmaxmoNawyznrR/wDW80j4lMe4wfKnw7R6TZsYiCJ294= X-Received: by 2002:a2e:be9d:0:b0:2f7:543a:3b1a with SMTP id 38308e7fff4ca-3002f8b27e2mr2657781fa.7.1733451745353; Thu, 05 Dec 2024 18:22:25 -0800 (PST) MIME-Version: 1.0 References: <20241205030843.8552FAB1EDA5@ary.qy> <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <202412052055.4B5KtYf1781856@freefriends.org> <2fd6a361-b26f-4ca4-9c65-b6ba0559644e@case.edu> <20241206020224.E59B3AB6EF07@ary.qy> In-Reply-To: <20241206020224.E59B3AB6EF07@ary.qy> From: Dan Cross Date: Thu, 5 Dec 2024 21:21:49 -0500 Message-ID: To: John Levine Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Message-ID-Hash: ELOBK5GJDT74HDY4C2IT7LA7KYO3VNJF X-Message-ID-Hash: ELOBK5GJDT74HDY4C2IT7LA7KYO3VNJF X-MailFrom: crossd@gmail.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-tuhs.tuhs.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: tuhs@tuhs.org X-Mailman-Version: 3.3.6b1 Precedence: list Subject: [TUHS] Re: Pipes (was Re: After 50 years, what has the Impact of Unix been?) List-Id: The Unix Heritage Society mailing list Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Thu, Dec 5, 2024 at 9:02=E2=80=AFPM John Levine wrote: > According to Dan Cross : > >> > diff <(old-program-version) <(new-program-version) > >> > > >> > to do simple regression testing. > >> > >> And fanout is simply > >> > >> ... | tee >(pipeline1) >(pipeline2) > > > >And indeed these things are pretty nifty, but don't they generate > >trees, and not arbitrary dags? They don't quite capture the full > >generality of Morrison-style networks since it doesn't seem like > >there's a way to connect process substitution fan-out with fan-in; at > >least, not conveniently. > > You can use mkfifo to make named pipes and then plumb them as wanted. It= 's > not particularly beautiful but it's as general as you could want. > > That's worked since 4.4BSD which was quite a while ago. Sure, but that's orthogonal since it's not really using shell syntax to build the thing up in a canonical way. I mean, perhaps in some literal sense it is (one types the commands into the interpreter...) but nowhere near the expressiveness of `ls | grep foo | whatever`. Fundamentally, the questions here are all about syntax. As I wrote in my first email, the system calls are very general, and could be used to build arbitrary directed graphs (not even limited to DAGs) of processes connected by pipes in all sorts of configurations (presumably those processes would be written to handle multiple input and output streams), without having to resort to named pipes. One can imagine some sort of DSL that describes such graphs of pipes and processes (Bakul alluded to this), and an interpreter for that DSL that creates the described set of pipes and processes, all appropriately connected and running. One could even imagine such a thing being used as part of a more traditional pipeline. My claim is simply that extant shell syntax isn't really amenable to this in a natural way. As Chet and Arnold pointed out, process substitution as pioneered in ksh gets us a little closer, but it's not quite as general (I believe any such graphs would have to be acyclic); it's certainly not as syntactically pleasant. There has been work along these lines; I was sent a reference off-list to a paper by Spinellis and Fragkoulis about a DAG-oriented shell: https://www.spinellis.gr/sw/dgsh/, which seems relevant. - Dan C.