Prefixed backslash removed when completing command

Prefixed backslash removed when completing command

[Mikael Auno]

[-- Attachment #1: Type: text/plain, Size: 577 bytes --]

Hi. I don't know if prefixing a command with a backslash to suppress
alias expansion is a intentional feature or not. If it is though, tab
completing a (complete or incomplete) command prefixed with a backslash
ought not remove the backslash. Current behavior seems to be to complete
something like "\mpla" to "mplayer" and not to "\mplayer" as I would expect.

I would be happy to try to fix it myself if no one has time to do it. A
pointer on where to start looking in that case would be good though as
I'm not at all familiar with the code of Zsh.

Mikael Auno


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 835 bytes --]

Re: Prefixed backslash removed when completing command

[Peter Stephenson]

On Thu, 20 May 2010 13:28:51 +0200
Mikael Auno <auno@kth.se> wrote:
> Hi. I don't know if prefixing a command with a backslash to suppress
> alias expansion is a intentional feature or not. If it is though, tab
> completing a (complete or incomplete) command prefixed with a
> backslash ought not remove the backslash. Current behavior seems to
> be to complete something like "\mpla" to "mplayer" and not to
> "\mplayer" as I would expect.
> 
> I would be happy to try to fix it myself if no one has time to do it.
> A pointer on where to start looking in that case would be good though
> as I'm not at all familiar with the code of Zsh.

You're quite right that this should work better, but I suspect fixing it is
a real pig.  Quoting is one of the messiest parts of completion (regular
viewers will know this is rather a strong statement).  I think fixing it
properly by leaving all quoting in place isn't going to be easy (to put it
mildly).

I think you have a workaround, though: use an initial single quote.  Not
only does that get kept, the closing single quote is put in place
afterwards.  Is this good enough?  (It would be good to document this but
goodness knows where it goes.)

-- 
Peter Stephenson <pws@csr.com>            Software Engineer
Tel: +44 (0)1223 692070                   Cambridge Silicon Radio Limited
Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, UK




Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom

Re: Prefixed backslash removed when completing command

[Mikael Auno]

Peter Stephenson wrote:
> Mikael Auno wrote:
>> Hi. I don't know if prefixing a command with a backslash to suppress
>> alias expansion is a intentional feature or not. If it is though, tab
>> completing a (complete or incomplete) command prefixed with a
>> backslash ought not remove the backslash. Current behavior seems to
>> be to complete something like "\mpla" to "mplayer" and not to
>> "\mplayer" as I would expect.
>>
>> I would be happy to try to fix it myself if no one has time to do it.
>> A pointer on where to start looking in that case would be good though
>> as I'm not at all familiar with the code of Zsh.
>
> You're quite right that this should work better, but I suspect fixing it is
> a real pig.  Quoting is one of the messiest parts of completion (regular
> viewers will know this is rather a strong statement).  I think fixing it
> properly by leaving all quoting in place isn't going to be easy (to put it
> mildly).
>
> I think you have a workaround, though: use an initial single quote.  Not
> only does that get kept, the closing single quote is put in place
> afterwards.  Is this good enough?  (It would be good to document this but
> goodness knows where it goes.)

(I did not see this reply until now as I am not on the list and as such 
did not receive the reply directly. I only saw it now as I happened to 
find the online archive of the mailing list.)

If I am not completely misunderstanding your reply, I think you 
misunderstood my issue. My issue has nothing to do with either single or 
double quotes (they are only there in the original message to separate 
the examples from the rest of the text), but with the backslash.

When I try to tab complete something starting with a leading backslash, 
e.g. \mpla, then it is completed to mplayer (without a leading 
backslash) instead of \mplayer (with a leading backslash) as I would expect.

I hope this clarifies the issue.

Mikael Auno

Re: Prefixed backslash removed when completing command

[Peter Stephenson]

On Wed, 27 Oct 2010 11:06:16 +0200
Mikael Auno <auno@kth.se> wrote:
> If I am not completely misunderstanding your reply, I think you 
> misunderstood my issue. My issue has nothing to do with either single
> or double quotes (they are only there in the original message to
> separate the examples from the rest of the text), but with the
> backslash.

No, you've misunderstood my reply.

> When I try to tab complete something starting with a leading
> backslash, e.g. \mpla, then it is completed to mplayer (without a
> leading backslash) instead of \mplayer (with a leading backslash) as
> I would expect.

Yes, I realise that.  Here's my reply rephrased.

The internals of completion are complicated.  There is no prospect of it
ever being able to preserve quotations in a general way.  In particular
there is no prospect of it ever being able to retain backslashes that
have no basic syntactic effect.

Slight digression on what I mean by a "basic syntactic effect".  Aliases
are a special case as they are expanded on input, so things that are
usually ignored in the usual shell grammar become significant;
completion doesn't know anything about this.  Completion just tries to
muddle through and produce an expression that (ignoring the alias
problem) is quoted in an appropriate way for the shell to process the
word on the command line.

If quoting is done using backslashes the only backslashes left will be
those needed for special characters.  Completion makes no attempt to try
to remember how the original word looked and reconstruct it.

However, if quoting is done with a single or double quote character at
the start of the word, that form of quoting is preserved even if it's
not (apparently) needed to quote special characters.  In this case,
completion remembers the whole word is quoted.  This is much simpler
than trying to remember what happens character by character.

So if you really insist on having a backslash, you are stuck, as far as
completion is concerned.  (You still have the possibility of writing
line editor, non-completion, functions to quote the word how you want.)

However, *if* it happens to be the case that you are simply trying to
ensure the word is quoted, somehow, you can get away with starting it
with a single quote, and forget about the backslash.  The single quote
is preserved during completion, unlike the backslash, and a closing
quote appended.  So it all works, and you have completed a quoted word.

I'm not aware of anywhere in the shell that requires you to quote a word
as \mplayer rather than 'mplayer'.  They should have the same effect in
every case except the bizarre one where you have defined 'mplayer',
including the quotes, as an alias.  So if you started with a single
quote (I know that's not what you asked about) it should do the right
thing, or I'd like to hear why it doesn't.

-- 
Peter Stephenson <pws@csr.com>            Software Engineer
Tel: +44 (0)1223 692070                   Cambridge Silicon Radio Limited
Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, UK


Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom

Re: Prefixed backslash removed when completing command

[Mikael Auno]

On 2010-10-27 12:02, Peter Stephenson wrote:
> On Wed, 27 Oct 2010 11:06:16 +0200
> Mikael Auno<auno@kth.se>  wrote:
>> If I am not completely misunderstanding your reply, I think you
>> misunderstood my issue. My issue has nothing to do with either single
>> or double quotes (they are only there in the original message to
>> separate the examples from the rest of the text), but with the
>> backslash.
>
> No, you've misunderstood my reply.
>
>> When I try to tab complete something starting with a leading
>> backslash, e.g. \mpla, then it is completed to mplayer (without a
>> leading backslash) instead of \mplayer (with a leading backslash) as
>> I would expect.
>
> Yes, I realise that.  Here's my reply rephrased.
>
> The internals of completion are complicated.  There is no prospect of it
> ever being able to preserve quotations in a general way.  In particular
> there is no prospect of it ever being able to retain backslashes that
> have no basic syntactic effect.
>
> Slight digression on what I mean by a "basic syntactic effect".  Aliases
> are a special case as they are expanded on input, so things that are
> usually ignored in the usual shell grammar become significant;
> completion doesn't know anything about this.  Completion just tries to
> muddle through and produce an expression that (ignoring the alias
> problem) is quoted in an appropriate way for the shell to process the
> word on the command line.
>
> If quoting is done using backslashes the only backslashes left will be
> those needed for special characters.  Completion makes no attempt to try
> to remember how the original word looked and reconstruct it.
>
> However, if quoting is done with a single or double quote character at
> the start of the word, that form of quoting is preserved even if it's
> not (apparently) needed to quote special characters.  In this case,
> completion remembers the whole word is quoted.  This is much simpler
> than trying to remember what happens character by character.
>
> So if you really insist on having a backslash, you are stuck, as far as
> completion is concerned.  (You still have the possibility of writing
> line editor, non-completion, functions to quote the word how you want.)
>
> However, *if* it happens to be the case that you are simply trying to
> ensure the word is quoted, somehow, you can get away with starting it
> with a single quote, and forget about the backslash.  The single quote
> is preserved during completion, unlike the backslash, and a closing
> quote appended.  So it all works, and you have completed a quoted word.
>
> I'm not aware of anywhere in the shell that requires you to quote a word
> as \mplayer rather than 'mplayer'.  They should have the same effect in
> every case except the bizarre one where you have defined 'mplayer',
> including the quotes, as an alias.  So if you started with a single
> quote (I know that's not what you asked about) it should do the right
> thing, or I'd like to hear why it doesn't.

You are completely correct. I misunderstood your reply as I did not know 
that the backslash was at all related to quoting or that ordinary 
quoting had the same result as prefixing a command with a backslash with 
respect to suppressing alias expansion. Prefixing with single or double 
quotes works great, especially as the closing quote is appended 
automatically.

Thank you for this insight,
Mikael Auno

Re: Prefixed backslash removed when completing command

[Peter Stephenson]

On Wed, 27 Oct 2010 12:41:05 +0200
Mikael Auno <auno@kth.se> wrote:
> You are completely correct. I misunderstood your reply as I did not
> know that the backslash was at all related to quoting or that
> ordinary quoting had the same result as prefixing a command with a
> backslash with respect to suppressing alias expansion.

The manual currently just gives the one example which, given your
experience, is probably not very helpful.

Index: Doc/Zsh/grammar.yo
===================================================================
RCS file: /cvsroot/zsh/zsh/Doc/Zsh/grammar.yo,v
retrieving revision 1.18
diff -p -u -r1.18 grammar.yo
--- Doc/Zsh/grammar.yo	3 Feb 2010 18:36:57 -0000	1.18
+++ Doc/Zsh/grammar.yo	27 Oct 2010 10:47:09 -0000
@@ -477,12 +477,15 @@ cindex(aliases, global)
 An alias is defined using the tt(alias) builtin; global aliases
 may be defined using the tt(-g) option to that builtin.
 
-Alias expansion is done on the shell input before any
-other expansion except history expansion.  Therefore,
-if an alias is defined for the word tt(foo), alias expansion
-may be avoided by quoting part of the word, e.g. tt(\foo).
-But there is nothing to prevent an alias being defined
-for tt(\foo) as well.
+Alias expansion is done on the shell input before any other expansion
+except history expansion.  Therefore, if an alias is defined for the
+word tt(foo), alias expansion may be avoided by quoting part of the
+word, e.g. tt(\foo).  But there is nothing to prevent an alias being
+defined for tt(\foo) as well.  For use with completion, which would
+remove an initial backslash following by a character that isn't special,
+it may be more convenient to quote the word by starting with a single
+quote, i.e. tt('foo); completion will automatically add the trailing
+single quote.
 
 There is a commonly encountered problem with aliases
 illustrated by the following code:

-- 
Peter Stephenson <pws@csr.com>            Software Engineer
Tel: +44 (0)1223 692070                   Cambridge Silicon Radio Limited
Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, UK


Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom

Re: Prefixed backslash removed when completing command

[Mikael Magnusson]

On 27 October 2010 12:41, Mikael Auno <auno@kth.se> wrote:

> You are completely correct. I misunderstood your reply as I did not know
> that the backslash was at all related to quoting or that ordinary quoting
> had the same result as prefixing a command with a backslash with respect to
> suppressing alias expansion. Prefixing with single or double quotes works
> great, especially as the closing quote is appended automatically.
>
> Thank you for this insight,

It's a little bit different from that even. Alias expansion is done
literally on the input basically just split on spaces. So even things
like ''mplayer or mplaye\r or mplayer"" will "cancel" out alias
expansion. That the quotes disappear in normal shell parsing is sort
of a lucky coincidence as far as the alias expansion is concerned. :)

-- 
Mikael Magnusson

Re: Prefixed backslash removed when completing command

[Štěpán Němec]

Peter Stephenson <Peter.Stephenson@csr.com> writes:

[...]
> +remove an initial backslash following by a character that isn't special,
                               ^^^^^^^^^
followed

Thanks for that addition, I tended to think the same as Mikael (Auno)
before (i.e. routinely use backslash to prevent alias expansion, not
realising quotes work just as well).

  Štěpán

Re: Prefixed backslash removed when completing command

[Mikael Magnusson]

On 27 October 2010 14:23, Štěpán Němec <stepnem@gmail.com> wrote:
> Peter Stephenson <Peter.Stephenson@csr.com> writes:
>
> [...]
>> +remove an initial backslash following by a character that isn't special,
>                               ^^^^^^^^^
> followed
>
> Thanks for that addition, I tended to think the same as Mikael (Auno)
> before (i.e. routinely use backslash to prevent alias expansion, not
> realising quotes work just as well).

Another thing that will incidentally prevent alias expansion and still
run the command is prefixing the command with a = (this will expand
the word to the full path to the binary, anywhere on the command
line).

-- 
Mikael Magnusson

PATCH: document more alias problems

[Peter Stephenson]

While I'm at it, it's probably high time that another note on alias
problems was promoted to the manual.  It's been in the FAQ for a while
but it could do with highlighting since it's really annoying and
confusing if you come across it.

Index: Doc/Zsh/grammar.yo
===================================================================
RCS file: /cvsroot/zsh/zsh/Doc/Zsh/grammar.yo,v
retrieving revision 1.20
diff -p -u -r1.20 grammar.yo
--- Doc/Zsh/grammar.yo	27 Oct 2010 12:29:13 -0000	1.20
+++ Doc/Zsh/grammar.yo	27 Oct 2010 12:45:19 -0000
@@ -500,6 +500,27 @@ is too late to expand the newly defined 
 a problem in shell scripts, functions, and code executed with `tt(source)'
 or `tt(.)'.  Consequently, use of functions rather than aliases is
 recommended in non-interactive code.
+
+Note also the unhelpful interaction of aliases and function definitions:
+
+example(alias func='noglob func'
+func+LPAR()RPAR() {
+    echo Do something with $*
+})
+
+Because aliases are expanded in function definitions, this causes the
+following command to be executed:
+
+example(noglob func+LPAR()RPAR() {
+    echo Do something with $*
+})
+
+which defines tt(noglob) as well as tt(func) as functions with the
+body given.  To avoid this, either quote the name tt(func) or use the
+alternative function definition form `tt(function func)'.  Ensuring the
+alias is defined after the function works but is problematic if the
+code fragment might be re-executed.
+
 texinode(Quoting)()(Aliasing)(Shell Grammar)
 sect(Quoting)
 cindex(quoting)

-- 
Peter Stephenson <pws@csr.com>            Software Engineer
Tel: +44 (0)1223 692070                   Cambridge Silicon Radio Limited
Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, UK


Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom

Re: Prefixed backslash removed when completing command

[Bart Schaefer]

On Oct 27, 11:02am, Peter Stephenson wrote:
}
} The internals of completion are complicated.  There is no prospect of it
} ever being able to preserve quotations in a general way.  In particular
} there is no prospect of it ever being able to retain backslashes that
} have no basic syntactic effect.

It strikes me that it might be possible to special-case a backslash as
the very first character in $BUFFER, but it's probably not worth the
effort ...