From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-3.3 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 30935 invoked from network); 4 Nov 2022 06:09:50 -0000 Received: from zero.zsh.org (2a02:898:31:0:48:4558:7a:7368) by inbox.vuxu.org with ESMTPUTF8; 4 Nov 2022 06:09:50 -0000 ARC-Seal: i=1; cv=none; a=rsa-sha256; d=zsh.org; s=rsa-20210803; t=1667542190; b=f4j6oA8Axn53kYk6a5kIB27qRsJEzKxWj+iYIYRn8ccT1Zjl5E34WnnOnHURKJYJ+rLdb9Hmgf YXrsS0dgRMYCDb70FptkcAJ7mZ2kt/7cK+dPaG82Kw2/ktwb49pgLuPDopehwmcTtxN7b0/BPf erHOsAp7ousfjpFjYCa5tNAaWsLi/lYUczx/FQ4SKyyizYXhakTF9wNcldJ0LnOeP8JR4DBKrH 6EVaayW3PbAVXPr+6qIUo4jF8dVgi064mFZmi6QXb5fPH8CLbe/0t2sgFzk/54/p1nQ7MC3ZN5 D9N+/KA0xZPclYXJs7IgDkr6EQhp6FAJ16NHnfB5RYTGcA==; ARC-Authentication-Results: i=1; zsh.org; iprev=pass (mail-ed1-f49.google.com) smtp.remote-ip=209.85.208.49; dkim=pass header.d=brasslantern-com.20210112.gappssmtp.com header.s=20210112 header.a=rsa-sha256; dmarc=none header.from=brasslantern.com; arc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed; d=zsh.org; s=rsa-20210803; t=1667542190; bh=g6wiTyOfUD/hwaCbpocBZRIc+CDJV6UcT6AdDQ5SQM8=; h=List-Archive:List-Owner:List-Post:List-Unsubscribe:List-Subscribe:List-Help: List-Id:Sender:Content-Type:To:Subject:Message-ID:Date:From:In-Reply-To: References:MIME-Version:DKIM-Signature:DKIM-Signature; b=nY1ozes4n4HoZj5wZxIV8Kevep4E8pqarf9/zltwNyYFQw/g4epHS8CSR9Ydfet5tTh8UyS6WI R0OXCiW4/evbcDSvjlYrdE45gor7zzoevOqXj2k7/zkfMlRJe5zJnPQEJCYTQSZoCVaY64+ylQ yDkevUB7Kipy/bU9a4naH8PPHdc2RGZkUQmycaZmFlgJDIWKlq41NWXK+BENajUhaVyxzlyeOz Ke+Z1DY5vFKopXgDCHK6oSSi2bw8B+IWYqIZEW+LPATvWIqRJSIDLcj8XPm6nSQFwYgC/eUzqX F0TUoZEdce6uir7ukoLHYQu9OB4e9InCFvd43K2FQY/ezg==; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=zsh.org; s=rsa-20210803; h=List-Archive:List-Owner:List-Post:List-Unsubscribe: List-Subscribe:List-Help:List-Id:Sender:Content-Type:To:Subject:Message-ID: Date:From:In-Reply-To:References:MIME-Version:Reply-To:Cc: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID; bh=JCTEMXmK13Eq8RUnOjRwAl+qBCC47o2h5nbyjjVhMp0=; b=K9k1DCoXIGqKcdxA5uf1osGEFP 7urJAxO/2cBiQMxOaO3oxV5TFC33y2IoVFackI4wHDpgKfj54UIStSS1ayXVkhmfmzpRHBJqJSAY7 Bo9aTdT3P/iPZ9qELvrz1MwRvNerL4eQF8gPQB4ObyhbYIJRKmeNRduGSxaculQhliMu6M7viEsTs CJM0W2XYVvbPRG/7FUoxUDGNfrP1S4F7GUPzsIG+aQEB+qwsv0Uay77BPhwT9mpgA2Vc2fmOvHLx0 /eDi19RRqOkHXKTtljFFqaKe/TEj4Y/ImkigzGQAzEPhaMDsOgs1/hUmRu2M9W3YLebEOOa2koHUd Wbm/oXcg==; Received: by zero.zsh.org with local id 1oqptm-0008Sc-Az; Fri, 04 Nov 2022 06:09:50 +0000 Authentication-Results: zsh.org; iprev=pass (mail-ed1-f49.google.com) smtp.remote-ip=209.85.208.49; dkim=pass header.d=brasslantern-com.20210112.gappssmtp.com header.s=20210112 header.a=rsa-sha256; dmarc=none header.from=brasslantern.com; arc=none Received: from mail-ed1-f49.google.com ([209.85.208.49]:45892) by zero.zsh.org with esmtps (TLS1.3:TLS_AES_128_GCM_SHA256:128) id 1oqptU-00088l-9Y; Fri, 04 Nov 2022 06:09:33 +0000 Received: by mail-ed1-f49.google.com with SMTP id a67so6137857edf.12 for ; Thu, 03 Nov 2022 23:09:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=brasslantern-com.20210112.gappssmtp.com; s=20210112; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :from:to:cc:subject:date:message-id:reply-to; bh=JCTEMXmK13Eq8RUnOjRwAl+qBCC47o2h5nbyjjVhMp0=; b=sjg2lEznfm06FET78aloszuR0Tg3rtsQyG5I6i54CWISVEBTjc4sLQrIJAelPAoonT c66MKQF+NB84srOci+KibOepIJzLM3iNX8AX9NyXbPzWMB6oy96BdIfpKlffd2bQoboU 5eOFsRWQey3FLR2e/0Z7vE6vI0i2ZdJYBz1mZ4T+YWxdc+CUfYYiOTPW0ONgZw3desAK pIHE16vHcumAQjERHl2/cdE21OEKbCU2Lfgn+/NZEVODCDv5ioQfdSWGuSKSAQSBz38u LTeXuXO2UPeEBgSjQcwCUK5zFxdLqKAnHu7wYjXkLMmucSZPIqzMevH+vmeW/j4GkJyH XuEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=JCTEMXmK13Eq8RUnOjRwAl+qBCC47o2h5nbyjjVhMp0=; b=nE1X/Rcd7rLcbAuAX4x/2yzcTcFMbfz70U6A9ZeBcxg+jnU3sJd48+vpWbQh7qcT1Z SIopIiTOfZS5GtZeNXtsJm39BViW+dOkpnpNwhu3Whpv+czSB93ZhbdgvJbrxvMXjiav n53asYnKa1RjpmWrNy/gMkZ545FqKDp6KT5VeLK9p0fJEYJV2cXjBob7RcIhGOKtOC4d 2qv6LCfS/2dElnfIpV2Dvmi1faTsCHfEpwL1f+7WBJ3FTUvvJ95N5P2HSeY7UhYqvmnH d3ulgREl0bpRGSiOqZBTqxZhMjpwAYMXvzLm2nARUqJRmQTMXzbG5s+sBLUCLP00qSSz GvWw== X-Gm-Message-State: ACrzQf3kpUizAIzG5tw7V2TP0sN8rk8MtgGxddOnBV0sxpU/L0dBf66D wteh2lSgEIuhxDiWNPCVptqYV8M4+SW7Y7/oRm7OLFiyCigjEw== X-Google-Smtp-Source: AMsMyM4ze6Dkj4ygjVB8qUzO+aBbn40AhYKYdY8jz0l8lNuO+5RDoSod8DaaIk6cFde/6thsSlgpP1iVTsLQyXXeXD4= X-Received: by 2002:a50:d602:0:b0:463:a83b:6f89 with SMTP id x2-20020a50d602000000b00463a83b6f89mr19240999edi.366.1667542171308; Thu, 03 Nov 2022 23:09:31 -0700 (PDT) MIME-Version: 1.0 References: <2FE96386-2D04-434F-AD2F-F08356B1AE04@kba.biglobe.ne.jp> In-Reply-To: From: Bart Schaefer Date: Thu, 3 Nov 2022 23:09:19 -0700 Message-ID: Subject: [PATCH] Re: problem with 'ls | less' shell function To: Zsh hackers list Content-Type: multipart/mixed; boundary="000000000000c504b205ec9eeade" X-Seq: 50862 Archived-At: X-Loop: zsh-workers@zsh.org Errors-To: zsh-workers-owner@zsh.org Precedence: list Precedence: bulk Sender: zsh-workers-request@zsh.org X-no-archive: yes List-Id: List-Help: , List-Subscribe: , List-Unsubscribe: , List-Post: List-Owner: List-Archive: --000000000000c504b205ec9eeade Content-Type: text/plain; charset="UTF-8" On Thu, Nov 3, 2022 at 4:10 PM Bart Schaefer wrote: > > We send a SIGCONT to the process group for the forked job, > which wakes up both the subjob and the superjob, but when we get the > SIGCHLD for the subjob we decide that its superjob is running not > because we just restarted it, but because it is still waiting to be > stopped from the previous SIGTSTP. I find we have to take this all the way back to workers/50105 and the "unkillable pipeline". The earlier example given was this: { /bin/sleep 10 ; /bin/sleep 20; } | { /bin/sleep 30 ; /bin/sleep 40; } I "fixed" this with workers/50134 by avoiding a reset of the process group leader when the job is not in the current shell, but it turns out that's not actually the problem. In the above example, two subshells are forked for either side of the pipe and "/bin/sleep 30" becomes the foreground job. Upon interrupting this, "/bin/sleep 40" begins to execute and the right-hand subshell immediately exits, because only the exit value of that sleep matters to the parent. The patch in 50134 makes "/bin/sleep 40" its own group leader in that case, allowing further keyboard-generated signals to reach it. Without 50134, the sleep effectively becomes an orphan; the parent is waiting, but for the wrong job, and the job that is running is immune to signals. There are two problems with this. First, arguably the "sleep 40" should not start at all, because the pipeline was interrupted. (This is in fact what happens in "sh" emulation thanks to workers/47794.) Second, in the "ls | less" function example from this thread, upon ^Z it's wrong to set the group leader to the newly-forked subshell because that results in the aforementioned race condition -- upon "fg", "ls" keeps getting stopped before "less" can get started, which causes the "oops, subjob is still stopped, better stop the superjob" confusion. I believe the reason interrupting "sleep 30" doesn't abort the whole pipeline is because the { ... } expression is in an interactive shell, so the two sleeps get interpreted as separately "jobbable". Running the whole thing in a non-interactive shell (zsh -fs) works fine. In fact % { /bin/sleep 10 ; /bin/sleep 20; } | { /bin/sleep 30 ; /bin/sleep 40; } ^Z % fg ^C % also works fine and does end the pipeline, but if you "fg" without immediate ^C, once "sleep 40" starts running we're right back in the invulnerable orphan state. There are two fixes needed here. One, never set the group leader to a process that's about to exit. Sadly, as far as I can tell, the parent doesn't know whether that's going to happen, and attempting to check with kill(gleader, 0) or similar just leads to a different race condition. Two, stop the whole { ... } construct if any job within it gets interrupted. In both cases 50134 should then be reverted. The attached patch seems to satisfactorily implement the second fix. There's an anecdote I may have mentioned before wherein a mechanic is called upon to fix a piece of machinery. When the job is finished he presents a bill for $1000. "What was the problem?" "Oh, a bolt had come out, I just replaced it." "What? $1000 for a single bolt?" "Sorry, let me correct that." The mechanic takes back his invoice, scribbles a few words, and hands it back. It now reads: "New bolt: $1. Finding out where to put it: $999" This patch is a $1000 bolt. Some no-op whitespace and comments included. Unfortunately, that leaves us with another example in 50105 still unfixed: ( zsh -fc 'print a few words; /bin/sleep 10' ) | { head -n 1 } Here we end up with the current shell as the foreground job because "head" has exited, and as an interactive shell it is ignoring signals. It needs to bring the left-hand-side into the foreground so it can be killed. I haven't tracked that down (nor figured out why 50134 works around it, except to believe it's two bugs canceling each other). Also, yes, I'm on vacation, and it's been raining all day here. --000000000000c504b205ec9eeade Content-Type: text/plain; charset="US-ASCII"; name="pipejobs.txt" Content-Disposition: attachment; filename="pipejobs.txt" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_la20155i0 ZGlmZiAtLWdpdCBhL1NyYy9qb2JzLmMgYi9TcmMvam9icy5jCmluZGV4IDcwNzM3NDI5Ny4uMmM0 ZjQxNTQ3IDEwMDY0NAotLS0gYS9TcmMvam9icy5jCisrKyBiL1NyYy9qb2JzLmMKQEAgLTU2Niw2 ICs1NjYsMTIgQEAgdXBkYXRlX2pvYihKb2Igam4pCiAJCSAgICAgKiB3aGVuIHRoZSBqb2IgaXMg ZmluYWxseSBkZWxldGVkLgogCQkgICAgICovCiAJCSAgICBqbi0+c3RhdCB8PSBTVEFUX0FUVEFD SDsKKwkJICAgIC8qCisJCSAgICAgKiBJZiB3ZSdyZSBpbiBzaGVsbCBqb2JzIG9uIHRoZSByaWdo dCBzaWRlIG9mIGEgcGlwZWxpbmUKKwkJICAgICAqIHdlIHNob3VsZCB0cmVhdCBpdCBsaWtlIGEg am9iIGluIHRoZSBjdXJyZW50IHNoZWxsLgorCQkgICAgICovCisJCSAgICBpZiAoaW5mb3JlZ3Jv dW5kID09IDIpCisJCQlpbmZvcmVncm91bmQgPSAxOwogCQl9CiAJCS8qIElmIHdlIGhhdmUgYGZv b3x3aGlsZSB0cnVlOyAoKCB4KysgKSk7IGRvbmUnLCBhbmQgaGl0CiAJCSAqIF5DLCB3ZSBoYXZl IHRvIHN0b3AgdGhlIGxvb3AsIHRvby4gKi8KQEAgLTE0ODgsMTAgKzE0OTQsNyBAQCBhZGRwcm9j KHBpZF90IHBpZCwgY2hhciAqdGV4dCwgaW50IGF1eCwgc3RydWN0IHRpbWV2YWwgKmJndGltZSwK IAkgKiBzZXQgaXQgZm9yIHRoYXQsIHRvby4KIAkgKi8KIAlpZiAoZ2xlYWRlciAhPSAtMSkgewot CSAgICBpZiAoam9idGFiW3RoaXNqb2JdLnN0YXQgJiBTVEFUX0NVUlNIKQotCQlqb2J0YWJbdGhp c2pvYl0uZ2xlYWRlciA9IGdsZWFkZXI7Ci0JICAgIGVsc2UKLQkJam9idGFiW3RoaXNqb2JdLmds ZWFkZXIgPSBwaWQ7CisJICAgIGpvYnRhYlt0aGlzam9iXS5nbGVhZGVyID0gZ2xlYWRlcjsKIAkg ICAgaWYgKGxpc3RfcGlwZV9qb2JfdXNlZCAhPSAtMSkKIAkJam9idGFiW2xpc3RfcGlwZV9qb2Jf dXNlZF0uZ2xlYWRlciA9IGdsZWFkZXI7CiAJICAgIC8qCkBAIC0xNTAwLDcgKzE1MDMsNyBAQCBh ZGRwcm9jKHBpZF90IHBpZCwgY2hhciAqdGV4dCwgaW50IGF1eCwgc3RydWN0IHRpbWV2YWwgKmJn dGltZSwKIAkgICAgICovCiAJICAgIGxhc3RfYXR0YWNoZWRfcGdycCA9IGdsZWFkZXI7CiAJfSBl bHNlIGlmICgham9idGFiW3RoaXNqb2JdLmdsZWFkZXIpCi0JCWpvYnRhYlt0aGlzam9iXS5nbGVh ZGVyID0gcGlkOworCSAgICBqb2J0YWJbdGhpc2pvYl0uZ2xlYWRlciA9IHBpZDsKIAkvKiBhdHRh Y2ggdGhpcyBwcm9jZXNzIHRvIGVuZCBvZiBwcm9jZXNzIGxpc3Qgb2YgY3VycmVudCBqb2IgKi8K IAlwbmxpc3QgPSAmam9idGFiW3RoaXNqb2JdLnByb2NzOwogICAgIH0KQEAgLTI1MDYsNiArMjUw OSw3IEBAIGJpbl9mZyhjaGFyICpuYW1lLCBjaGFyICoqYXJndiwgT3B0aW9ucyBvcHMsIGludCBm dW5jKQogCQlqb2J0YWJbam9iXS5zdGF0ICY9IH5TVEFUX0NVUlNIOwogCSAgICB9CiAJICAgIGlm ICgoc3RvcHBlZCA9IChqb2J0YWJbam9iXS5zdGF0ICYgU1RBVF9TVE9QUEVEKSkpIHsKKwkJLyog V0lGQ09OVElOVUVEIHdpbGwgbWFrZXJ1bm5pbmcoKSBhZ2FpbiBhdCBraWxsamIoKSAqLwogCQlt YWtlcnVubmluZyhqb2J0YWIgKyBqb2IpOwogCQlpZiAoZnVuYyA9PSBCSU5fQkcpIHsKIAkJICAg IC8qIFNldCAkISB0byBpbmRpY2F0ZSB0aGlzIHdhcyBiYWNrZ3JvdW5kZWQgKi8K --000000000000c504b205ec9eeade--