From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-3.3 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 8684 invoked from network); 20 Nov 2022 15:09:40 -0000 Received: from zero.zsh.org (2a02:898:31:0:48:4558:7a:7368) by inbox.vuxu.org with ESMTPUTF8; 20 Nov 2022 15:09:40 -0000 ARC-Seal: i=1; cv=none; a=rsa-sha256; d=zsh.org; s=rsa-20210803; t=1668956980; b=Zv/qzbv95Hx+NUWBW9QlX6Agq0QqjgFBrsCuzkb/InhDpWyPfuqVRsWvhrSLdh7yPvqipgxo4s BolR6RfIiz/c4BGil/WnXjYFHiYCPKl5cgGzYeWKZRyvX+T9mBcXUbFVUfNibxhtS2weJJV8fe BbXYasDKgqcanbh2z1vRH7hKgwTtaQzUOYYdhJe9xR+56gLPNYNAGPewnX7gQmVJowbIfYC5MN vnb1CkNR5N9BkQhV2SjI83ihQXF3tQ5bXr6zot0821lxKQq1AhR2upSGFtxMKsN+dSkBUACiJf z6JG1J9uvvI78R0zDCfk7AkbwEzrNJQ2xMSs3sgI8h+A1g==; ARC-Authentication-Results: i=1; zsh.org; iprev=pass (relay2-d.mail.gandi.net) smtp.remote-ip=217.70.183.194; dmarc=none header.from=chazelas.org; arc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed; d=zsh.org; s=rsa-20210803; t=1668956980; bh=QYK14rCo6mVTqfFbeFxqGak0IdsuBfdgH81koUFaYJo=; h=List-Archive:List-Owner:List-Post:List-Unsubscribe:List-Subscribe:List-Help: List-Id:Sender:In-Reply-To:Content-Transfer-Encoding:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:DKIM-Signature; b=IQaTgonf3wW4+wC5WKD3G/Q7LGzMU7IiJkhB0GxTbIn5Wq5/bnfwqb6Lps3wAB0w8B765r3Thi bXZ83y8JCFr4UWlXOV2Yf1qATRZPMA8Io7Llg4sD14ysseDDzR2ywW5qFm2j+9be37nHDz4nmg vko3LZxJ+c32SwC6XOsZuCqQB2Pu6zKq0EWy2n967YV+7mMXZdm3uvnMDip6cu2hLr29nXfKcF wULAJHgZg3zPIJ6IKzkxYKKqIa11w5iL+eF2qqT2jdqOfwJmRs05llr+EvbCZDbgYpe5uRbRHY lVSNol75Iz+iFu4Kdjmm/o+t5DMen0OM49iZIRWn1dUZ2w==; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=zsh.org; s=rsa-20210803; h=List-Archive:List-Owner:List-Post:List-Unsubscribe: List-Subscribe:List-Help:List-Id:Sender:In-Reply-To:Content-Transfer-Encoding :Content-Type:MIME-Version:References:Message-ID:Subject:Cc:To:From:Date: Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender :Resent-To:Resent-Cc:Resent-Message-ID; bh=Xj5DJ+oLTGEgo3hZAykPpGWc0v6EGxZTv4PoSz59MHs=; b=pgYZJs5HzMoeJhCMC9a4byhbQa z2a3/mQOYhXj9JIfxT1/P2j5ryoIt+YTfVt50B9U2Tyar4TuyTo4H1KSVKpBpCBH6ArZBXIlWqjQK BmSTNdOMUBlk6bC5LjwhEZABVwX2E/2NyWkG1A4fwzph35kzsy9edNNY2gS0AKS3mim59ISRZcSoX xZWh/KEWCAT+5QwZJfQrMeeaDr0Z6k6XdIQDD7S77jMFuCdZe5LPsPSEOP87C1v7Ihn0wg1rHm7Jr xKwgrm04F+pVvp5xhclf9ZYvcRivrpZpRw86triQ/jTLCqG+JXpwO1+NLDc2rup4dTMrGptskDT2F S3Z7efYA==; Received: by zero.zsh.org with local id 1owlwx-000Lgn-Mw; Sun, 20 Nov 2022 15:09:39 +0000 Authentication-Results: zsh.org; iprev=pass (relay2-d.mail.gandi.net) smtp.remote-ip=217.70.183.194; dmarc=none header.from=chazelas.org; arc=none Received: from relay2-d.mail.gandi.net ([217.70.183.194]:41699) by zero.zsh.org with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) id 1owlwA-000Kzu-6d; Sun, 20 Nov 2022 15:08:50 +0000 Received: (Authenticated sender: stephane@chazelas.org) by mail.gandi.net (Postfix) with ESMTPSA id 2161D40004; Sun, 20 Nov 2022 15:08:48 +0000 (UTC) Date: Sun, 20 Nov 2022 15:08:48 +0000 From: Stephane Chazelas To: Ray Andrews Cc: zsh-users@zsh.org Subject: Re: why is eval needed? Message-ID: <20221120150848.74di3mpjwnfcul34@chazelas.org> Mail-Followup-To: Ray Andrews , zsh-users@zsh.org References: <20221119164852.hwujmufa6hn5lotr@chazelas.org> <352823cc-954a-fa79-d830-d69d593b1c02@eastlink.ca> <3fa3f7ff-1733-4730-a62f-dd0e138c3b72@app.fastmail.com> <20221120085519.szudhyg5ewrw3b4o@chazelas.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Seq: 28421 Archived-At: X-Loop: zsh-users@zsh.org Errors-To: zsh-users-owner@zsh.org Precedence: list Precedence: bulk Sender: zsh-users-request@zsh.org X-no-archive: yes List-Id: List-Help: , List-Subscribe: , List-Unsubscribe: , List-Post: List-Owner: List-Archive: 2022-11-20 05:47:33 -0800, Ray Andrews: [...] > > arguments. > Yeah I get it.  The first time I 'got it' was when discussing 'aptitude' and > the various quotings needed.  Again one is creating an invisible level of > organization imposed on a string of characters.  Where it got confusing was > when zsh imposed it's own ideas of grouping, making '-L 2' into a single > entity. zsh doesn't impose anything. "-L 2" is a string made of 4 characters in any shell or programming language. There's no programming language where "-L 2" means 2 strings -L and 2. What may be the source of your confusion is that in the Bourne shell, there was a second round of splitting applied to unquoted text in list context. Space is a special character in the syntax of the shell which is used to delimit command arguments, but it's also in the default value of the $IFS special variable which was used to *further* split text into futher arguments. In the Bourne shell (introduced in the late 70s): edit file Is like in all other shells code that's meant to run /path/to/edit with "edit" and "file" as separate arguments, but if $IFS also happens to contain "i", then those two words happen to be further split into "ed", "t" and "f", "le", so you end up running "/bin/ed" with "ed", "t", "f" and "le" as arguments. That splitting also happened on top of parameter expansion. ed $file Upon syntax evaluation yields "ed" and "$file" as two separate tokens, but both "ed" and the contents of $file further underwent $IFS-splitting. Almost worse, globbing also happened on top of $file expansion. That is a very weird feature from a language design point of view. Anybody who doesn't know what the shells looked like before the Bourne shell would think Stephen Bourne was out of his mind. The thing is in the original Unix shell (designed in the early 70s on computers than had kilobytes of RAM), there was no variable. There was a concept of script that could take arguments, and in the script, you'd refer to them as $1, $2... the positional parameters (which have survived in modern shells). Parameter expansion was really crude, it was a bit like aliases, the contents of the parameter was just expanded in place into the code being evaluated. So for instance, if you called your script as: my-script 'hi;rm -rf /' And the script did: echo $1 That would say hi and destroy the system. It had its uses though. my-script 'dir1 dir2' 'file3 file4' With a script that did: ls $1 rm $2 Would list the dir1 and dir2 directories and remove the file3 and file4 files. And: my-script '*.txt' in a script that did: cd /foo rm $1 cd /bar rm $1 Would remove the files with name ending in .txt in both /foo and /bar (or the current directory if cd failed... but that's beside the point). My understanding is that the Bourne shell's bizarre IFS handling and the fact that globbing was performed upon parameter expansion was an attempt to keep some level of backward compatibility with that Thompson shell. You see similar things hapenning in csh from the same era. The Korn shell (from the early 80s) kept most of that with the exception that it only did IFS-splitting upon expansions, not on literal text. In: file=/some/file edit $file With IFS=i, $file would be split into /some/f and le, but edit would stay edit. The POSIX specification of sh standardised that behaviour so that's the one found in bash / dash / yash... as well. That nonsense was fixed in most shells written after that though, starting with rc, the shell of plan9 / Unix v10 from the late 80s, but also zsh (early 90s) or more recently (mid 2000s) fish. The fact that in sh/ksh/bash: arg='-L 1' tree $arg Calls tree with "-L" and "1" as arguments is not that bash doesn't do some sort of magic "grouping" that zsh would be doing, but that ksh/bash contrary to zsh does that extra layer of $IFS-splitting from the Bourne shell on top of the syntax parsing as the default value of $IFS happens to contain the space character. That's why in sh/bash/ksh you almost always need to quote parameter expansions if you intend to pass the contents of a variable as an argument to a command. See https://unix.stackexchange.com/questions/171346/security-implications-of-forgetting-to-quote-a-variable-in-bash-posix-shells for the kind of thing that can happen if you forget. > Dunno if it's really a thing to be desired but naively one might > like some way of assembling a command string as if it was at CLI, that is > *just* a string of characters -- which is how I was looking at it. If you want to assemble a string, and that string to be evaluated as shell code, that's what eval is for. eval evaluate code written in the shell language. That's a way to dynamically invoke the language interpreter. But generally, that's not what you want. Writing correct shell code dynamically based on external input is very easy to get wrong. In your case, it rather looks like you want to build up a list of arguments to pass to a command for which you obviously need a shell list/array variable. > > tree parses its options. It looks like it does not use the > > standard getopt() API or the GNU getopt_long() API for that: > > > > $ nm -D =tree | grep -i getopt > > $ ltrace -e '*opt*@*' tree > /dev/null > > +++ exited (status 0) +++ > What are you doing there?  I have no 'nm' command here.  No such command in > Debian repository. nm is a standard development command (though not -D which AFAIK is a GNU extension). Part of GNU binutils on GNU systems. Here used to list the external functions that the utility claims it needs. The above shows that tree doesn't use the getopt() standard command or that if it does, it embedded a copy into the executable rather than using the one from the GNU libc. > > So it must be doing it by hand. > Yeah, so much in the GNU/Linux world is ad hoc.  Everybody did their own > thing.  No rules. Note that tree is not part of the GNU project, it's just a utility written by some guy and shared to the world. There is *some* level of consistency among utilities in the GNU toolchest. There is even such a thing as published GNU coding standards. See for instance https://www.gnu.org/prep/standards/html_node/Command_002dLine-Interfaces.html#Command_002dLine-Interfaces zsh is not part of the GNU project either. Note that zsh also predates Linux and has been used in a wide variety of GNU and non-GNU systems. I'll agree with you that the lack of consistency in API between the different tools that are available out there can be annoying (nothing to do with GNU or Linux) [...] > figure out.  But why does calling 'eval' fix it?  It seems as if eval > 'flattens' everything -- one is back to a command line string of characters > with no imposed grouping. eval evaluates shell code. I struggle to understand what you don't understand. Maybe you're thinking too much or too little of what a "command line string" is. A "command line string" like: var='value'; if blah; then echo $(cmd) | tr -d x; fi is just one line of code in the syntax of the shell programing language. It's not some magic universal language to talk to the system. You can store that code in bits in variables with: a="'val" b="ue'; if bl" c='ah; then ech' d='o $(cmd) | tr -' e='d x; fi' But surely you don't expact var=$a$b$c$d$e to have the same effect as running that command. You can however do: eval "var=$a$b$c$d$e" For that string to be passed as code to the shell language interpreter for it to interpret. [...] > Yeah, as I was saying, it does seem that 'tree' is very crabby. Why don't > the GNU people iron these things out? [...] Again, tree has nothing to do with the GNU project. There is also unfortunatly not one standard to parse options and arguments. There is POSIX getopt() but for instance, it doesn't support long options (neither a la GNU nor a la X11, or a la perl Getopt::Long nor a la zsh zparseopts...). -- Stephane