From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 13041 invoked by alias); 8 Oct 2013 20:45:05 -0000 Mailing-List: contact zsh-workers-help@zsh.org; run by ezmlm Precedence: bulk X-No-Archive: yes List-Id: Zsh Workers List List-Post: List-Help: X-Seq: 31803 Received: (qmail 28151 invoked from network); 8 Oct 2013 20:44:49 -0000 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on f.primenet.com.au X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.2 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:subject:message-id:in-reply-to :references:mime-version:content-type:content-transfer-encoding; bh=Qn1wb/JaT3A4miSE+dleODGO0SWiCo/rKyqN/edSCCI=; b=R16tMs+8IBXrg8GXDe903LcY88K9WTC4jOw5WqS8GMkFTSMw5FTOFaTCmOMPe6QCEK AOQq7dQ1coorghDEDwV8+hQimePJcSdCW9JPKl/DbkBk8TCGzxSijMWQiX9do7gr0wju bBd3j3Gau2c3K6xXlRsrcOQjQ80aOu4/itBXw10DYfeVDGBIFRK3AXyUOL1/VeFYcHXs 7qaA+N7oNZZeoKtGhEWDTjfxYze9q9J7KDwqa34XFuYFkwk2rbQ7KGQZDOF3bo/XGWUA FXoobECe2eHWV1uxmn3rL122eCgZqn+ogHCm0N39FvUnJpsXE5NymG01z3jis9lksICk OSDg== X-Gm-Message-State: ALoCoQkm10jPwrWrrHSXrO95pMKnc+d7pTMrVZyKFTh/zC5nIdIUg4gMn9aA51m8wOtpn4p1KuOm X-Received: by 10.180.187.101 with SMTP id fr5mr3416128wic.26.1381265082118; Tue, 08 Oct 2013 13:44:42 -0700 (PDT) X-ProxyUser-IP: 86.6.157.246 Date: Tue, 8 Oct 2013 21:44:37 +0100 From: Peter Stephenson To: zsh-workers@zsh.org Subject: Shell job structure Message-ID: <20131008214437.41dc396a@pws-pc.ntlworld.com> In-Reply-To: <131007074049.ZM32707@torch.brasslantern.com> References: <20131005223159.25fea6a0@pws-pc.ntlworld.com> <131006173621.ZM31831@torch.brasslantern.com> <20131007102529.5354f342@pwslap01u.europe.root.pri> <131007074049.ZM32707@torch.brasslantern.com> X-Mailer: Claws Mail 3.8.0 (GTK+ 2.24.7; x86_64-redhat-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit On Mon, 07 Oct 2013 07:40:49 -0700 Bart Schaefer wrote: > - because external jobs can exit and be reaped in arbitrary order, even > in a pipeline, the job table is used to keep track of which position > in the array to update > > - jobs that run in the current shell don't have a complete job table entry > [cf. all the gyrations to suspend/background a loop] and aren't "reaped" > in the same code path as external jobs, so the wrong array position (or > none at all) may get updated > > - the incorrect update depends on whether the external job exited AND got > reaped before the current-shell job has completed, because of the way > reaping updates the job table, so correctness is unpredictable > > - complex commands "in the current shell" may have external subjobs that > need a separate pipestatus (this applies only at end of a pipeline) This may be pie in the sky, but probably what we need is for jobs to be hierarchical. A job at the top level would always be treated as separate from another job at the top level for the original purpose of job control but might include nested jobs, representing objects such as functions and chunks of pipelines, that might themselves have jobs representing external commands. Furthermore, a job would become a first class representation of shell state, so anything just involving shell builtins would have a job, removing the obfuscation in the current relationship between jobs and code execution. It would thus in principle allow better encapsulation of other shell state. The job table would become a list of top-level jobs, while the job structure would have pointers to nested jobs. We might get rid of the table all together and simply have a single pointer that got searched for top-level jobs the same way those jobs got searched for subjobs. If this was inefficient for searching we could hash PIDs and the like; if it was inefficient for memory management (but I don't see why it should be, particularly, compared with memory management for anything else) we could still have a pool of job structures. I think this means we'd have a status associated with a job, not just a process. How you got the job status would depend on the nature of the job. I think a job would be one of: - An external command with one main process and possibly auxiliary processes; the status of the job is just that of the process. This is sort of a degenerate case of a pipeline but I'm wondering it might be neater to make it just one part of a pipeline so auxiliary processes get tied to the right main process. This also makes an external command in a pipeline more similar to the case of a shell construct in a pipeline, which we know needs to look like a single job (in this new sense); that's where we came in. - A shell structure of code being executed together (details such as where we need new subjobs TBD) such as a function or a complex command typed at the command line or in a particular part of a pipeline --- this is roughly what's referred to as a "list" in the code. This would have arbitrarily complicated subjobs which would vary depending on what was being executed within the structure. There would be no external process associated with the top level job unless we explicitly forked to make this happen; putting the current shell job into the background should now be as natural as putting a chunk of shell code into a pipeline not as the last element. The status of the job is the status of the last command to execute (which may be a "return" if it's a function). - A pipeline, a set of jobs (in this new sense --- maybe we need a better name to separate this from job control) each of which was one of the above two types. (I don't think pipelines nest directly in pipelines, you need some intervening shell structure otherwise they degenerate to the same pipeline.) The status of the job is either the status of the last subjob in the pipeline (NO_PIPE_FAIL) or uses the PIPE_FAIL rules. If it worked to the extent of allowing the removal of things like list_pipe and STAT_SUPERJOB it would have served its purpose well. We might even be able to move over to this gradually. As long as we can still search all jobs, we could introduce the new structures with the existing logic. For example, the pipestatus code could for now search every process associated with the current job, but would gradually get rewritten to pick out subjobs at the appropriate level. You might hope that as this process went on it would become easier to ask the question "when has this job finished"? Possibly the most unpleasant bit of this is making the hierarchy of calls in exec.c agree with this new structural hierarchy. It might be messy enough to put the kibosh on the whole thing --- I don't understand execpline() and execpline2() and I think it's necessary to do so. Whether this is ever going to come about... pws -- Peter Stephenson Web page now at http://homepage.ntlworld.com/p.w.stephenson/