From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 2618 invoked from network); 6 Jun 2020 04:34:50 -0000 Received: from ns1.primenet.com.au (HELO primenet.com.au) (203.24.36.2) by inbox.vuxu.org with ESMTPUTF8; 6 Jun 2020 04:34:50 -0000 Received: (qmail 16842 invoked by alias); 6 Jun 2020 04:34:37 -0000 Mailing-List: contact zsh-workers-help@zsh.org; run by ezmlm Precedence: bulk X-No-Archive: yes List-Id: Zsh Workers List List-Post: List-Help: List-Unsubscribe: X-Seq: 45993 Received: (qmail 23688 invoked by uid 1010); 6 Jun 2020 04:34:37 -0000 X-Qmail-Scanner-Diagnostics: from wout4-smtp.messagingengine.com by f.primenet.com.au (envelope-from , uid 7791) with qmail-scanner-2.11 (clamdscan: 0.102.3/25828. spamassassin: 3.4.4. Clear:RC:0(64.147.123.20):SA:0(-2.6/5.0):. Processed in 4.807681 secs); 06 Jun 2020 04:34:37 -0000 X-Envelope-From: d.s@daniel.shahaf.name X-Qmail-Scanner-Mime-Attachments: | X-Qmail-Scanner-Zip-Files: | Received-SPF: none (ns1.primenet.com.au: domain at daniel.shahaf.name does not designate permitted sender hosts) X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedrudeggedgjeejucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkjghfofggtgfgsehtqh dttdertdejnecuhfhrohhmpeffrghnihgvlhcuufhhrghhrghfuceougdrshesuggrnhhi vghlrdhshhgrhhgrfhdrnhgrmhgvqeenucggtffrrghtthgvrhhnpeegveegveeukedvte ekffffvedvvdeiieevvddvveelleekkeejhfejfffhfedtteenucffohhmrghinhepohhp vghnghhrohhuphdrohhrghdpuddqvddtudejrdhimhdplhhipheirdhfrhenucfkphepje elrddujeeirdefledrieelnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehm rghilhhfrhhomhepugdrshesuggrnhhivghlrdhshhgrhhgrfhdrnhgrmhgv X-ME-Proxy: Date: Sat, 6 Jun 2020 04:33:50 +0000 From: Daniel Shahaf To: "brian m. carlson" Cc: Mikael Magnusson , zsh-workers@zsh.org Subject: Re: [PATCH v2] exec: run final pipeline command in a subshell in sh modeZZ Message-ID: <20200606043350.6b7fb334@tarpaulin.shahaf.local2> In-Reply-To: <20200605204144.GD6569@camp.crustytoothpaste.net> References: <20200605015338.1347787-1-sandals@crustytoothpaste.net> <20200605015338.1347787-2-sandals@crustytoothpaste.net> <20200605204144.GD6569@camp.crustytoothpaste.net> X-Mailer: Claws Mail 3.17.3 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable brian m. carlson wrote on Fri, 05 Jun 2020 20:41 +0000: > On 2020-06-05 at 10:21:41, Mikael Magnusson wrote: > > On 6/5/20, brian m. carlson wrote: =20 > > > zsh typically runs the final command in a pipeline in the main shell > > > instead of a subshell. However, POSIX requires that all commands in a > > > pipeline run in a subshell, but permits zsh's behavior as an extensio= n. =20 > >=20 > > What POSIX actually says is: > > "each command of a multi-command pipeline is in a subshell > > environment; as an extension, however, any or all commands in a > > pipeline may be executed in the current environment" That's quoted from https://pubs.opengroup.org/onlinepubs/9699919799/utiliti= es/V3_chap02.html#tag_18_12. The part Brian quotes below is from https://pubs.opengroup.org/onlinepubs/9= 699919799/basedefs/V1_chap02.html#tag_02_01_01. > > Ie, it does not say "shall", so it doesn't require a subshell all, in > > fact it explicitly does permit not using one as you also say. The This interpretation is analogous to how conforming C programs must assume neither that =C2=ABchar=C2=BB is signed nor that it is unsigned. > > patch is possibly useful (seems unlikely to me), but to say it is > > required by POSIX is not true. If someone depends on every command in > > a pipeline being a subshell, they should fix their code, for example > > by adding ( ) around it (the command(s) or the whole pipeline). =20 >=20 > POSIX makes a declarative statement about the behavior of a pipeline. > It is true that it doesn't explicitly use the word "shall" in this case, > since such a statement would explicitly prohibit the inclusion of an > extension at all and make it explicitly non-conforming. >=20 The sentence preceding the one you quoted reads: . Non-standard extensions, when used, may change the behavior of utilities, functions, or facilities defined by POSIX.1-2017. I take this to mean non-standard extensions aren't bound by "shall"s. As to why the passage Mikael quoted doesn't use the word "shall"=E2=80=A6 w= ell, presumably it doesn't use the word "shall" because it doesn't describe "a feature or behavior that is mandatory"=C2=B9. > What POSIX does say is that one =E2=80=9Cshall define an environment in w= hich an > application can be run with the behavior specified by POSIX.1-2017.=E2=80= =9D > I'm proposing that "zsh --emulate sh" implement the POSIX behavior for > that reason. What Mikael's saying is that zsh's incumbent behaviour is already POSIX-conforming, but POSIX-conforming implementations have some leeway: have a range of possible behaviours to choose from, just like conforming C compilers can choose what signedness to give to =C2=ABchar=C2=BB. The passage Mikael quoted specifies that running the last command in a pipeline in a subshell by default is permitted in certain cases, outlined by the phrases "as an extension" and "may". The definition of "may"=C2=B9 says it's used to describe "optional" behavio= urs, and that conforming applications should tolerate both presence and absence of that behaviour. As to that behaviour's being an extension, the sentence you quoted, once put back in context (link given above), requires conforming implementations to document how to disable non-standard extensions where they change POSIX-documented behaviour. I don't think the passage Mikael quoted is covered by that: it _is_ an extension, of course, but I question whether it's a "non-standard extension"=C2=B2. After all, it is specified in the standard. To summarize, I don't see why behaviour specified with the phrases "as an extension" and "may" should be off by default in a POSIX-conforming mode. Would you elaborate on this? (On the other hand, I'm not sure why they bothered to write the words "as an extension" there. They don't seem to change the meaning one way or the other.) > I will tell you that as a practical matter, nobody writing code for sh > expects the last command not to be run in a subshell and consequently > lots of code is practically broken in this case with zsh as /bin/sh. > The Git Project is very fastidious about writing portable shell, as is > Debian, and I can tell you from experience that both are broken with zsh > as sh with the current behavior, even if they should not have made that > assumption. >=20 > I would also like to see macOS users who decide to use zsh as /bin/sh > have a good experience with existing code that overwhelmingly does > make this assumption. Well, perhaps there is something we can do to make their lives easier. Continuing the analogy to C, gcc(1) has -fsigned-char/-funsigned-char flags to help unportable programs. However, I hesitate to propose adding an option just for this: adding options is always easy to suggest, but not always a good idea. Since zsh already incorporates a parser for sh scripts, perhaps we could write a tool that automatically adds parentheses to the last element in every pipeline. That's not such a crazy idea: it already exists (in a much more general form) for C: http://coccinelle.lip6.fr/ [Between these two, an extra option is definitely the lower-hanging fruit, of course.] > zsh is a very popular interactive shell, and allowing it to be used as a > portable sh on systems where the system sh is less capable would be > really beneficial. How would it be beneficial? > If your objection is to the wording, I'm happy to revise it to remove > the word "requires", but I do think this provides a lot of benefits for > the sh scripting case while not impacting users who are expecting > different behavior for the zsh case. The patch would constitute a backwards-incompatible change to anyone who uses zsh as sh today and relies on the current behaviour of pipelines. This might have been acceptable if it were a question of changing a non-conforming behaviour to a conforming behaviour. However, the current behaviour does appear to be conforming. Furthermore, if the patch is accepted, those who rely on the incumbent behaviour won't have an easy workaround to get it behaviour back, something comparable to the "add parentheses around every element of every pipeline" strategy that can be used given the incumbent implementation to get the patch's semantics. Cheers, Daniel =C2=B9 https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap01.= html =C2=B2 If POSIX didn't want to make this distinction, it would have written just "extension", unqualified.