* order of processing in brace expansion @ 2001-04-27 17:31 Allan Poindexter 2001-04-27 18:09 ` Peter Stephenson 0 siblings, 1 reply; 12+ messages in thread From: Allan Poindexter @ 2001-04-27 17:31 UTC (permalink / raw) To: zsh-workers Consider the following: pts/11 10:37 208%setopt braceccl cato:/export/home/cato/apoindex pts/11 10:38 209%foo=b,c cato:/export/home/cato/apoindex pts/11 10:38 210%echo a{$foo}d cato:/export/home/cato/apoindex a,d abd acd pts/11 10:38 211%echo a{b,c}d cato:/export/home/cato/apoindex abd acd pts/11 10:38 212%unsetopt braceccl cato:/export/home/cato/apoindex pts/11 10:38 213%echo a{$foo}d cato:/export/home/cato/apoindex a{b,c}d According to info: Expansion is done in the above specified order in five steps. The first is "history expansion" which is only performed in interactive shells. The next step is "alias expansion" which is done right before the command line is parsed. They are followed by "process substitution", "parameter expansion", "command substitution", "arithmetic expansion", and "brace expansion" which are performed in one step in left-to-right fashion. After these expansions, all unquoted occurrences of the characters `\', `'', and `"' are removed and the result is subjected to "filename expansion" followed by "filename generation". Since brace expansion has syntactic markers at two points there is some ambiguity about where it would fall in the left to right processing but it seems that neither choice should cause the actual behavior seen. If brace expansion is considered to occur at the opening brace then I would expect: %setopt braceccl %echo a{$foo}d a$d afd aod %unsetopt braceccl %echo a{$foo}d ab,cd in the above example since the brace expansion would occur before $foo was parameter expanded. If brace expansion is considered to occur at the closing brace then I would expect: %echo a{$foo}d abd acd since parameter expansion would occur first. (The setting of brace_ccl would be unimportant in this case.) What actually occurs seems to be that it is decided what kind of brace expansion is needed at the opening brace but the brace expansion is actually done (consistent with the decision of type made earlier) at the closing brace. My personal preference would be for brace expansion to occur entirely at the closing brace since that seems to me to be the most useful definition. Perhaps others have a different view. Can anyone shed any light on what I might be overlooking here? /-\ |_ |_ /-\ |\| ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: order of processing in brace expansion 2001-04-27 17:31 order of processing in brace expansion Allan Poindexter @ 2001-04-27 18:09 ` Peter Stephenson 2001-04-28 6:05 ` Bart Schaefer 0 siblings, 1 reply; 12+ messages in thread From: Peter Stephenson @ 2001-04-27 18:09 UTC (permalink / raw) To: Zsh hackers list Allan Poindexter <apoindex@aoc.nrao.edu> wrote: > Consider the following: > > pts/11 10:37 208%setopt braceccl > pts/11 10:38 209%foo=b,c > pts/11 10:38 210%echo a{$foo}d > a,d abd acd > > Can anyone shed any light on what I might be overlooking here? By default, parameters in zsh are expanded without turning the contents into tokens, so the comma is just treated as a normal character, just as if you had typed a{'b,c'}d. This is the behaviour I would expect. The real bug, I would say, is this: % echo a{$~foo}d a,d abd acd The ~ force $foo to be interpreted pretty much as if the characters had been put directly on the command line (the option GLOB_SUBST does the same thing). This is done by default by a lot of other shells. In *this* case, I would expect what you expect: the expression is interpreted the same as a{b,c}d. Unfortunately it isn't. The reason seems to be that the comma doesn't get turned back into a token. If we want to fix this, the following patch will do it. Index: Src/glob.c =================================================================== RCS file: /cvsroot/zsh/zsh/Src/glob.c,v retrieving revision 1.15 diff -u -r1.15 glob.c --- Src/glob.c 2001/04/24 05:45:17 1.15 +++ Src/glob.c 2001/04/27 17:57:25 @@ -2373,6 +2373,7 @@ case '*': case '?': case '=': + case ',': for (t = ztokens; *t; t++) if (*t == *s) { if (bslash) -- Peter Stephenson <pws@csr.com> Software Engineer CSR Ltd., Unit 300, Science Park, Milton Road, Cambridge, CB4 0XL, UK Tel: +44 (0)1223 392070 ********************************************************************** The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer. ********************************************************************** ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: order of processing in brace expansion 2001-04-27 18:09 ` Peter Stephenson @ 2001-04-28 6:05 ` Bart Schaefer 2001-04-28 18:17 ` Bart Schaefer 2001-05-02 9:50 ` Peter Stephenson 0 siblings, 2 replies; 12+ messages in thread From: Bart Schaefer @ 2001-04-28 6:05 UTC (permalink / raw) To: Peter Stephenson, Zsh hackers list On Apr 27, 7:09pm, Peter Stephenson wrote: } Subject: Re: order of processing in brace expansion } } I would expect what you expect: the expression is interpreted the same as } a{b,c}d. Unfortunately it isn't. The reason seems to be that the comma } doesn't get turned back into a token. If we want to fix this, the } following patch will do it. The only issue, I think, is that comma is not a token unless it is inside braces, and we don't know at time of expanding $~foo that the surrounding expression is eligible for brace expansion. A tokenized comma does still match an "ordinary" comma during file globbing, but does that mean that it always matches an ordinary comma? If tokenizing when it's not necessary doesn't hurt anything, then I think we should fix the bug. ("make check" does pass with the patch applied.) -- Bart Schaefer Brass Lantern Enterprises http://www.well.com/user/barts http://www.brasslantern.com Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: order of processing in brace expansion 2001-04-28 6:05 ` Bart Schaefer @ 2001-04-28 18:17 ` Bart Schaefer 2001-05-02 9:50 ` Peter Stephenson 1 sibling, 0 replies; 12+ messages in thread From: Bart Schaefer @ 2001-04-28 18:17 UTC (permalink / raw) To: Peter Stephenson, Zsh hackers list On Apr 28, 6:05am, Bart Schaefer wrote: } Subject: Re: order of processing in brace expansion } } On Apr 27, 7:09pm, Peter Stephenson wrote: } } Subject: Re: order of processing in brace expansion } } } } I would expect what you expect: the expression is interpreted the same as } } a{b,c}d. Unfortunately it isn't. The reason seems to be that the comma } } doesn't get turned back into a token. If we want to fix this, the } } following patch will do it. } } The only issue, I think, is that comma is not a token unless it is inside } braces Perhaps that's not the only issue ... consider: zsh% x='*,v' zsh% echo abc{*,$~x} That is, in all versions of zsh so far, using a parameter expansion is a way to quote commas against brace expansion while still getting filename generation after the expansion. This makes me more reluctant to apply the patch; I just can't decide. -- Bart Schaefer Brass Lantern Enterprises http://www.well.com/user/barts http://www.brasslantern.com Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: order of processing in brace expansion 2001-04-28 6:05 ` Bart Schaefer 2001-04-28 18:17 ` Bart Schaefer @ 2001-05-02 9:50 ` Peter Stephenson 2001-05-02 14:59 ` Bart Schaefer 1 sibling, 1 reply; 12+ messages in thread From: Peter Stephenson @ 2001-05-02 9:50 UTC (permalink / raw) To: Zsh hackers list > On Apr 27, 7:09pm, Peter Stephenson wrote: > } Subject: Re: order of processing in brace expansion > } > } I would expect what you expect: the expression is interpreted the same as > } a{b,c}d. Unfortunately it isn't. The reason seems to be that the comma > } doesn't get turned back into a token. If we want to fix this, the > } following patch will do it. > > The only issue, I think, is that comma is not a token unless it is inside > braces, and we don't know at time of expanding $~foo that the surrounding > expression is eligible for brace expansion. A tokenized comma does still > match an "ordinary" comma during file globbing, but does that mean that it > always matches an ordinary comma? > > If tokenizing when it's not necessary doesn't hurt anything, then I think > we should fix the bug. ("make check" does pass with the patch applied.) I would be fairly confident about this, since there are plenty of other tokens which might be left hanging around, e.g. #'s or ~'s when EXTENDEDGLOB isn't set. These cases are all handled by a generic search for tokens which need turning back into normal characters. So I'll commit this and see. -- Peter Stephenson <pws@csr.com> Software Engineer CSR Ltd., Unit 300, Science Park, Milton Road, Cambridge, CB4 0XL, UK Tel: +44 (0)1223 392070 ********************************************************************** The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer. ********************************************************************** ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: order of processing in brace expansion 2001-05-02 9:50 ` Peter Stephenson @ 2001-05-02 14:59 ` Bart Schaefer 2001-05-02 15:28 ` Peter Stephenson 0 siblings, 1 reply; 12+ messages in thread From: Bart Schaefer @ 2001-05-02 14:59 UTC (permalink / raw) To: Peter Stephenson, Zsh hackers list On May 2, 10:50am, Peter Stephenson wrote: } Subject: Re: order of processing in brace expansion } } > If tokenizing when it's not necessary doesn't hurt anything, then I think } > we should fix the bug. ("make check" does pass with the patch applied.) } } I would be fairly confident about this, since there are plenty of other } tokens which might be left hanging around, e.g. #'s or ~'s when } EXTENDEDGLOB isn't set. With regard to zsh-workers/14153: > That is, in all versions of zsh so far, using a parameter expansion is a > way to quote commas against brace expansion while still getting filename > generation after the expansion. I was wondering whether there might be some sort of compromise, such as only tokenizing commas when BRACE_CCL is set. -- Bart Schaefer Brass Lantern Enterprises http://www.well.com/user/barts http://www.brasslantern.com Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: order of processing in brace expansion 2001-05-02 14:59 ` Bart Schaefer @ 2001-05-02 15:28 ` Peter Stephenson 2001-05-02 16:30 ` Bart Schaefer 0 siblings, 1 reply; 12+ messages in thread From: Peter Stephenson @ 2001-05-02 15:28 UTC (permalink / raw) To: Zsh hackers list Bart wrote: > > That is, in all versions of zsh so far, using a parameter expansion is a > > way to quote commas against brace expansion while still getting filename > > generation after the expansion. > > I was wondering whether there might be some sort of compromise, such as > only tokenizing commas when BRACE_CCL is set. I don't really like filling the code full of this sort of dependency, unless it can be proven that someone, somewhere was using this particular hack which none of us here even suspected existed until a few days ago. But it has just occurred to me that actually the fix results in some other illogical results. % foo='{a,b}' % print $~foo {a,b} % print {$~foo} {a b} This is rather hairy. The obvious fix is to tokenize braces, too. This seems to do the trick, but the change of behaviour is now much more obvious. We probably ought to make it either all or nothing. Bash doesn't expand braces that result from parameters (probably due to the ordering of expansions). Any suggestions? It doesn't seem worth an option. Index: Src/glob.c =================================================================== RCS file: /cvsroot/zsh/zsh/Src/glob.c,v retrieving revision 1.16 diff -u -r1.16 glob.c --- Src/glob.c 2001/05/02 09:53:32 1.16 +++ Src/glob.c 2001/05/02 15:22:28 @@ -2374,6 +2374,8 @@ case '?': case '=': case ',': + case '{': + case '}': for (t = ztokens; *t; t++) if (*t == *s) { if (bslash) -- Peter Stephenson <pws@csr.com> Software Engineer CSR Ltd., Unit 300, Science Park, Milton Road, Cambridge, CB4 0XL, UK Tel: +44 (0)1223 392070 ********************************************************************** The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer. ********************************************************************** ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: order of processing in brace expansion 2001-05-02 15:28 ` Peter Stephenson @ 2001-05-02 16:30 ` Bart Schaefer 2001-05-09 9:06 ` Peter Stephenson 2001-05-11 19:22 ` Allan Poindexter 0 siblings, 2 replies; 12+ messages in thread From: Bart Schaefer @ 2001-05-02 16:30 UTC (permalink / raw) To: Zsh hackers list On May 2, 4:28pm, Peter Stephenson wrote: } } This is rather hairy. The obvious fix is to tokenize braces, too. This } seems to do the trick, but the change of behaviour is now much more } obvious. We probably ought to make it either all or nothing. Bash doesn't } expand braces that result from parameters (probably due to the ordering of } expansions). Any suggestions? I'd prefer that the code and the doc agree with on another. Doc: ${~SPEC} Turn on the GLOB_SUBST option for the evaluation of SPEC; if the `~' is doubled, turn it off. When this option is set, the string resulting from the expansion will be interpreted as a pattern anywhere that is possible, such as in filename expansion and filename generation and pattern-matching contexts ... GLOB_SUBST <C> <K> <S> Treat any characters resulting from parameter expansion as being eligible for file expansion and filename generation, and any characters resulting from command substitution as being eligible for filename generation. There's no mention of brace expansion there anywhere, and braces are not expanded in "a pattern" for most senses of "pattern" elsewhere in the doc; on the other hand, for command substitutions, it's not even supposed to tokenize a leading `~' or `=', which it obviously does anyway: zsh% echo $(echo '~') ~ zsh% (setopt glob_subst; echo $(echo '~')) /home/schaefer [That latter is definitely a SH_FILE_EXPANSION interaction with GLOB_SUBST; the GLOB_SUBST doc is describing the case where SH_FILE_EXPANSION is set: zsh% (setopt glob_subst sh_file_expansion; echo $(echo '~')) ~ So we have some doc to fix even if we don't change comma/brace behavior.] As of this moment, my leaning in the "all or nothing" category is towards "nothing", particularly because of the possibility of breaking sh/bash compatibility (note that GLOB_SUBST is set by default for sh/ksh emulation). -- Bart Schaefer Brass Lantern Enterprises http://www.well.com/user/barts http://www.brasslantern.com Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: order of processing in brace expansion 2001-05-02 16:30 ` Bart Schaefer @ 2001-05-09 9:06 ` Peter Stephenson 2001-05-11 16:57 ` Allan Poindexter 2001-05-11 19:22 ` Allan Poindexter 1 sibling, 1 reply; 12+ messages in thread From: Peter Stephenson @ 2001-05-09 9:06 UTC (permalink / raw) To: Zsh hackers list I've backed off the patch that tokenizes commas coming from a parameter expansion with globsubst. Probably the following is a good idea to make it clearer. Index: Doc/Zsh/options.yo =================================================================== RCS file: /cvsroot/zsh/zsh/Doc/Zsh/options.yo,v retrieving revision 1.16 diff -u -r1.16 options.yo --- Doc/Zsh/options.yo 2001/03/30 16:20:03 1.16 +++ Doc/Zsh/options.yo 2001/05/09 09:05:02 @@ -473,7 +473,8 @@ Treat any characters resulting from parameter expansion as being eligible for file expansion and filename generation, and any characters resulting from command substitution as being eligible for -filename generation. +filename generation. Braces (and commas in between) do not become eligible +for expansion. ) pindex(HASH_CMDS) cindex(hashing, of commands) -- Peter Stephenson <pws@csr.com> Software Engineer CSR Ltd., Unit 300, Science Park, Milton Road, Cambridge, CB4 0XL, UK Tel: +44 (0)1223 392070 ********************************************************************** The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer. ********************************************************************** ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: order of processing in brace expansion 2001-05-09 9:06 ` Peter Stephenson @ 2001-05-11 16:57 ` Allan Poindexter 0 siblings, 0 replies; 12+ messages in thread From: Allan Poindexter @ 2001-05-11 16:57 UTC (permalink / raw) To: zsh-workers Peter> I've backed off the patch that tokenizes commas coming from a Peter> parameter expansion with globsubst. Probably the following is a good Peter> idea to make it clearer. You may also want to add a clarification to this description in the Expansion node about brace expansion falling in the left to right order at the closing brace: They are followed by "process substitution", "parameter expansion", "command substitution", "arithmetic expansion", and "brace expansion" which are performed in one step in left-to-right fashion. I'm not sure this is absolutely necessary but it may forestall a potential for confusion. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: order of processing in brace expansion 2001-05-02 16:30 ` Bart Schaefer 2001-05-09 9:06 ` Peter Stephenson @ 2001-05-11 19:22 ` Allan Poindexter 2001-05-11 20:09 ` Bart Schaefer 1 sibling, 1 reply; 12+ messages in thread From: Allan Poindexter @ 2001-05-11 19:22 UTC (permalink / raw) To: zsh-workers Bart> There's no mention of brace expansion there anywhere, and braces are Bart> not expanded in "a pattern" for most senses of "pattern" elsewhere in Bart> the doc; on the other hand, for command substitutions, it's not even Bart> supposed to tokenize a leading `~' or `=', which it obviously does Bart> anyway: I confess that conforming to the docs is a good thing. OTOH I don't see any way to parameterize patterns for brace expansion if this is done. (E.g. setting a parameter in a script to a pattern that will be used in brace expansion in several places and which may need to be changed from time to time. IMO is a good thing to be able to do). Am I overlooking something? Is there some other way to do this? ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: order of processing in brace expansion 2001-05-11 19:22 ` Allan Poindexter @ 2001-05-11 20:09 ` Bart Schaefer 0 siblings, 0 replies; 12+ messages in thread From: Bart Schaefer @ 2001-05-11 20:09 UTC (permalink / raw) To: apoindex, zsh-workers On May 11, 1:22pm, Allan Poindexter wrote: > > I don't see any way to parameterize patterns for brace expansion if > this is [not] done. (E.g. setting a parameter in a script to a pattern > that will be used in brace expansion in several places and which may > need to be changed from time to time. IMO is a good thing to be able > to do). Am I overlooking something? Is there some other way to do > this? Brace expansion is completely equivalent to expanding an array parameter with rc_expand_param set. So instead of alternates=x,y,z echo foo/{$alternates}/bar you do alternates=(x y z) echo foo/$^alternates/bar -- Bart Schaefer Brass Lantern Enterprises http://www.well.com/user/barts http://www.brasslantern.com Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2001-05-11 20:11 UTC | newest] Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2001-04-27 17:31 order of processing in brace expansion Allan Poindexter 2001-04-27 18:09 ` Peter Stephenson 2001-04-28 6:05 ` Bart Schaefer 2001-04-28 18:17 ` Bart Schaefer 2001-05-02 9:50 ` Peter Stephenson 2001-05-02 14:59 ` Bart Schaefer 2001-05-02 15:28 ` Peter Stephenson 2001-05-02 16:30 ` Bart Schaefer 2001-05-09 9:06 ` Peter Stephenson 2001-05-11 16:57 ` Allan Poindexter 2001-05-11 19:22 ` Allan Poindexter 2001-05-11 20:09 ` Bart Schaefer
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/zsh/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).