* Expanding quotes @ 2013-12-17 14:48 Yuri D'Elia 2013-12-17 18:26 ` Bart Schaefer 0 siblings, 1 reply; 11+ messages in thread From: Yuri D'Elia @ 2013-12-17 14:48 UTC (permalink / raw) To: zsh-users Hi everyone, mkdir test cd test touch file file\' file\" ls *\"<TAB> results in: % ls file file\" file\' although the result of the pure glob *\" is correct, and only contains file\". This confused me yesterday, where I almost got the impression that the quote was breaking the match. Another tidbit: % ls *[\']<TAB> doesn't expand entirely, although it globs correctly file\' when executed. The problem only exists with the quote/double quote (in both forms). ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Expanding quotes 2013-12-17 14:48 Expanding quotes Yuri D'Elia @ 2013-12-17 18:26 ` Bart Schaefer 2013-12-23 17:38 ` Yuri D'Elia 0 siblings, 1 reply; 11+ messages in thread From: Bart Schaefer @ 2013-12-17 18:26 UTC (permalink / raw) To: zsh-users On Dec 17, 3:48pm, Yuri D'Elia wrote: } Subject: Expanding quotes } } touch file file\' file\" } ls *\"<TAB> } } results in: } } % ls file file\" file\' Hmm, somewhere down in the completion internals the code that sets the PREFIX string is re-quoting both the backslash and the single quote. When this is re-interpolated by the expand-word code it turns into the empty string, so you get the expansion of "*". } % ls *[\']<TAB> } } doesn't expand Same problem, except you get the expansion of *[] which is an invalid pattern and hence nothing. There's a basic conflict in the completion system in that it can't "know" if you mean for the literal string backslash-quote to be completed, or if you mean for the backslash to have its usual semantic effect on the quote. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Expanding quotes 2013-12-17 18:26 ` Bart Schaefer @ 2013-12-23 17:38 ` Yuri D'Elia 2013-12-23 19:15 ` Bart Schaefer 0 siblings, 1 reply; 11+ messages in thread From: Yuri D'Elia @ 2013-12-23 17:38 UTC (permalink / raw) To: zsh-users On 12/17/2013 07:26 PM, Bart Schaefer wrote: > There's a basic conflict in the completion system in that it can't "know" > if you mean for the literal string backslash-quote to be completed, or if > you mean for the backslash to have its usual semantic effect on the quote. Hhhm, how could this happen? Surely, zsh seem to take note if the argument to be completed starts with a quote. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Expanding quotes 2013-12-23 17:38 ` Yuri D'Elia @ 2013-12-23 19:15 ` Bart Schaefer 2013-12-24 15:12 ` Ray Andrews 2013-12-24 16:11 ` Yuri D'Elia 0 siblings, 2 replies; 11+ messages in thread From: Bart Schaefer @ 2013-12-23 19:15 UTC (permalink / raw) To: zsh-users On Dec 23, 6:38pm, Yuri D'Elia wrote: } } On 12/17/2013 07:26 PM, Bart Schaefer wrote: } > There's a basic conflict in the completion system in that it can't "know" } > if you mean for the literal string backslash-quote to be completed, or if } > you mean for the backslash to have its usual semantic effect on the quote. } } Hhhm, how could this happen? Surely, zsh seem to take note if the } argument to be completed starts with a quote. It's a syntactic question as well as a semantic one. For example, you typed ls *\" and expected completion to treat backslash-dquote as a unit matching a literal double quote. But someone else might type ls *" and expect completion to *supply* the missing backslash, because there is a file name ending with a quote. It gets even more complicated if the backslash is inside another set of quotes, etc. We spent quite a lot of time (years ago) responding to complaints about this kind of thing before finally concluding that we couldn't please everybody and should stick with making the common cases work. A third complication is that there are a huge number of different ways to apply quoting, and trying to figure out which one corresponds to what the user already typed is difficult to generalize. The globbing code that generates the list of possible files returns a particular quoting scheme that would have to be converted to the user's arbitrary quoting. Consequently there are a number of cases that either never came up when this was all being developed, or that have been deemed to seldom come up in practice, where the results are imperfect. We'd be happy to find some more volunteers to work on the internals of completion to resolve more of these cases. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Expanding quotes 2013-12-23 19:15 ` Bart Schaefer @ 2013-12-24 15:12 ` Ray Andrews 2013-12-24 16:24 ` Yuri D'Elia 2013-12-24 16:11 ` Yuri D'Elia 1 sibling, 1 reply; 11+ messages in thread From: Ray Andrews @ 2013-12-24 15:12 UTC (permalink / raw) To: zsh-users On 23/12/13 11:15 AM, Bart Schaefer wrote: > and expect completion to *supply* the missing backslash, because there > is a file name ending with a quote. It gets even more complicated if > the backslash is inside another set of quotes, etc. We spent quite a > lot of time (years ago) responding to complaints about this kind of thing > before finally concluding that we couldn't please everybody and should > stick with making the common cases work. Pardon for wandering off on a philosophical rant, but it seems to me that there is a very deep error in even trying to solve 'problems' like this. It is not logically possible to permit any syntax and still parse a statement. All meaningful grammars exist--can only exist--because they impose limitations on language. It is monstrous that a file name is permitted to end with a quotation mark! Without reserved characters we invite chaos. It's the same with spaces inside a filename--it is insanity to permit it. And then the double quote is legal too? file name .................. two identifiers so ... "file name" .................. is the filename ... << file name >> or are the double quotes include in the filename so that the filename is .... << "file name" >> ? Madness! Granted I'm still a relative beginner, but it seems to me that the syntax of zsh (all sh*) is already vastly over complicated, even Byzantine. As more and more special situations are handled, the code must mushroom into an intractable mess, and actually create more problems than it solves. Better IMHO not to even try. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Expanding quotes 2013-12-24 15:12 ` Ray Andrews @ 2013-12-24 16:24 ` Yuri D'Elia 2013-12-24 17:18 ` Ray Andrews 0 siblings, 1 reply; 11+ messages in thread From: Yuri D'Elia @ 2013-12-24 16:24 UTC (permalink / raw) To: zsh-users On 12/24/2013 04:12 PM, Ray Andrews wrote: > file name .................. two identifiers so ... > "file name" .................. is the filename ... << file name >> or > are the double quotes include in the filename so that the filename is > .... << "file name" >> ? > > Madness! > > Granted I'm still a relative beginner, but it seems to me that the > syntax of zsh (all sh*) is already vastly over complicated, even > Byzantine. As more and more special situations are handled, the code > must mushroom into an intractable mess, and actually create more > problems than it solves. Better IMHO not to even try. Quoting is invariably part of any language construct. It's really important, especially as a beginner, to understand quoting right from the start as a "normal event", not as an exception. Restricting the allowed characters of a file in the file system will not remove quoting issues of a variable's value (for example). To wrap your mind against it, you might see quoting as a problem for the interpreter to *separate arguments* and not to interpret data. That is, if we could choose # as an argument separator, we could have any character in the file name except #. It just so happens that it's /usually/ more readable to type: $ command argument "a value" then $ command#argument#a value Just my 2c. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Expanding quotes 2013-12-24 16:24 ` Yuri D'Elia @ 2013-12-24 17:18 ` Ray Andrews 2013-12-26 14:50 ` Yuri D'Elia 0 siblings, 1 reply; 11+ messages in thread From: Ray Andrews @ 2013-12-24 17:18 UTC (permalink / raw) To: zsh-users On 24/12/13 08:24 AM, Yuri D'Elia wrote: > On 12/24/2013 04:12 PM, Ray Andrews wrote: >> file name .................. two identifiers so ... >> "file name" .................. is the filename ... << file name >> or >> are the double quotes include in the filename so that the filename is >> .... << "file name" >> ? >> >> Madness! >> >> Granted I'm still a relative beginner, but it seems to me that the >> syntax of zsh (all sh*) is already vastly over complicated, even >> Byzantine. As more and more special situations are handled, the code >> must mushroom into an intractable mess, and actually create more >> problems than it solves. Better IMHO not to even try. > Quoting is invariably part of any language construct. Of course. > It's really important, especially as a beginner, to understand quoting > right from the start as a "normal event", not as an exception. Of course. > Restricting the allowed characters of a file in the file system will not > remove quoting issues of a variable's value (for example). > > To wrap your mind against it, you might see quoting as a problem for the > interpreter to *separate arguments* and not to interpret data. That is, > if we could choose # as an argument separator, we could have any > character in the file name except #. It just so happens that it's > /usually/ more readable to type: That's just the point: There must be reserved characters, and it seems to me that quotation marks are first on that list. Also, as a policy, I prefer simplicity with some resulting limitations on what can be done, over complicated efforts to do 'anything'. It's just as Bart said, some issues are better left 'unsolved'. The solution could be worse than the problem. I think back to my DOS days, where almost all of the 'special' characters were reserved--it sure made things simpler. > $ command argument "a value" > > then > > $ command#argument#a value > > Just my 2c. > And mine ;-) I come at this, not as a zsh expert, but as a guy who knows something about information theory. Noam Chomsky would have something to say about this sort of issue. > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Expanding quotes 2013-12-24 17:18 ` Ray Andrews @ 2013-12-26 14:50 ` Yuri D'Elia 2013-12-26 19:54 ` Ray Andrews 0 siblings, 1 reply; 11+ messages in thread From: Yuri D'Elia @ 2013-12-26 14:50 UTC (permalink / raw) To: zsh-users On 12/24/2013 06:18 PM, Ray Andrews wrote: > I prefer simplicity with some resulting limitations on what can be done, > over complicated efforts to do 'anything'. It's just as Bart said, some > issues are better left 'unsolved'. The solution could be worse than the > problem. I think back to my DOS days, where almost all of the 'special' > characters were reserved--it sure made things simpler. I think you're conflating the text UI of a shell, the filesystem and the shell language itself as part of the same issue. Yes, taken together, I do agree that from my user point of view it's definitely inconvenient and the interaction is not ideal. However, I would gladly prefer a shell solution than "fixing" what's clearly not broken: the filesystem. If you use a visual file manager, users will curse your from not being allowed to use quotes and spaces as part of the file name. I know windows users complain all the time about the disallowed characters, because they don't see the reasons of the restricted character set anymore (this is true also for users of nautilus/gnome/kde of course). I also agree with these users. > And mine ;-) I come at this, not as a zsh expert, but as a guy who knows > something about information theory. Noam Chomsky would have something to > say about this sort of issue. The line-editing interface has definitely limitations, but I would always like to think about it just as one of the possible interfaces to the file system. Imagine a different shell, where you had to different insertion modes, like "vi". To enter a path, you would enter "pathname" mode with ESCp, and the shell would keep track internally of the beginning and end of the file name, while just showing the path maybe in a different color: % some-command ESCppath name with spacesESCc continue command Problem solved without escaping entirely? (damn, I would love such a shell). ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Expanding quotes 2013-12-26 14:50 ` Yuri D'Elia @ 2013-12-26 19:54 ` Ray Andrews 0 siblings, 0 replies; 11+ messages in thread From: Ray Andrews @ 2013-12-26 19:54 UTC (permalink / raw) To: zsh-users On 26/12/13 06:50 AM, Yuri D'Elia wrote: > On 12/24/2013 06:18 PM, Ray Andrews wrote: >> I prefer simplicity with some resulting limitations on what can be done, >> over complicated efforts to do 'anything'. It's just as Bart said, some >> issues are better left 'unsolved'. The solution could be worse than the >> problem. I think back to my DOS days, where almost all of the 'special' >> characters were reserved--it sure made things simpler. > I think you're conflating the text UI of a shell, the filesystem and the > shell language itself as part of the same issue. Yes, taken together, I > do agree that from my user point of view it's definitely inconvenient > and the interaction is not ideal. However, I would gladly prefer a shell > solution than "fixing" what's clearly not broken: the filesystem. You're right, I'm looking at this as one collective issue. But IMHO the thing that is broken is to permit things like double quotes in a filename at all. From what I understand, POSIX does not allow it, but maybe I misunderstand. Anyway, if somehow zsh can learn to live with this sort of thing, then so be it, but I think the task should not be asked. Perhaps some switch or flag of some sort could be added to indicate: "I have quotes in filenames here ... God be with you." Anyway, that's how it looks to me, maybe I'm wrong. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Expanding quotes 2013-12-23 19:15 ` Bart Schaefer 2013-12-24 15:12 ` Ray Andrews @ 2013-12-24 16:11 ` Yuri D'Elia 2013-12-25 6:59 ` Bart Schaefer 1 sibling, 1 reply; 11+ messages in thread From: Yuri D'Elia @ 2013-12-24 16:11 UTC (permalink / raw) To: zsh-users On 12/23/2013 08:15 PM, Bart Schaefer wrote: > It's a syntactic question as well as a semantic one. For example, you > typed > > ls *\" > > and expected completion to treat backslash-dquote as a unit matching a > literal double quote. But someone else might type > > ls *" > > and expect completion to *supply* the missing backslash, because there > is a file name ending with a quote. It gets even more complicated if I'm pretty sure I've experienced this behavior before, though it's definitely rare. > A third complication is that there are a huge number of different ways > to apply quoting, and trying to figure out which one corresponds to what > the user already typed is difficult to generalize. The globbing code > that generates the list of possible files returns a particular quoting > scheme that would have to be converted to the user's arbitrary quoting. I see. > We'd be happy to find some more volunteers to work on the internals of > completion to resolve more of these cases. Thanks for the clarification, the rationale is sound for me. Personally, I would actually prefer is there was an option in zsh to normalize the quoting mechanism to always "double quote" the argument instead of escaping characters (thus inserting the initial double-quote if missing). I don't know if this would actually simplify or furthermore complicate the expansion rules. With complete_in_word and all the other completion switches I definitely see the complexity of "doing the right thing" in all the cases. Though the fact that a glob might not expand in the same way as when executed makes a bit of uneasy feeling for me. I often use expansion to proof-check a glob instead of executing the command twice. Maybe there's a better way to do it that doesn't involve running "ls glob" before "real-command glob". ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Expanding quotes 2013-12-24 16:11 ` Yuri D'Elia @ 2013-12-25 6:59 ` Bart Schaefer 0 siblings, 0 replies; 11+ messages in thread From: Bart Schaefer @ 2013-12-25 6:59 UTC (permalink / raw) To: zsh-users On Dec 24, 5:11pm, Yuri D'Elia wrote: } } Personally, I would actually prefer is there was an option in zsh to } normalize the quoting mechanism to always "double quote" the argument } instead of escaping characters (thus inserting the initial double-quote } if missing). I don't know if this would actually simplify or furthermore } complicate the expansion rules. Have a look at Functions/Zle/quote-and-complete-word in the distributed sample functions. } the fact that a glob might not expand in the same way as when executed } makes a bit of uneasy feeling for me. I often use expansion to } proof-check a glob instead of executing the command twice. } } Maybe there's a better way to do it that doesn't involve running "ls } glob" before "real-command glob". You can create your own little completer that does nothing but globbing: _glob () { local -a globbed globbed=( ${(Q)~words[CURRENT]} ) [[ $globbed == $words[CURRENT]] ]] && return 1 compadd -Uf -a globbed } You can fiddle with the assignment to globbed until you get the behavior you want. What I've done with ${(Q)~words[CURRENT]} may not be to your liking; e.g. you might prefer ${~words[CURRENT]//\\/} to only remove any backslashes. Something of the kind is needed, though, because the tilde operator only makes globbing characters active, it doesn't unquote other special characters that may be quoted in the value of $words. The default completers if you don't set a style are (_complete _ignored). So you just need to add yours: zstyle ':completion:*' completer _glob _complete _ignored (Use "zstlyle -L" to see if you've already changed the completer style, and if so just add _glob in an appropriate place.) ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2013-12-26 20:24 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2013-12-17 14:48 Expanding quotes Yuri D'Elia 2013-12-17 18:26 ` Bart Schaefer 2013-12-23 17:38 ` Yuri D'Elia 2013-12-23 19:15 ` Bart Schaefer 2013-12-24 15:12 ` Ray Andrews 2013-12-24 16:24 ` Yuri D'Elia 2013-12-24 17:18 ` Ray Andrews 2013-12-26 14:50 ` Yuri D'Elia 2013-12-26 19:54 ` Ray Andrews 2013-12-24 16:11 ` Yuri D'Elia 2013-12-25 6:59 ` Bart Schaefer
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/zsh/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).