Case-insensitive completion with partial substring matching

Case-insensitive completion with partial substring matching

[nicolas.canceill]

Hi every one,

I was looking for a way to enable case-insensitive completion with partial substring matching, and it seems many people are using this:

zstyle ':completion:*' matcher-list 'm:{a-zA-Z}={A-Za-z}' 'r:|[._-]=* r:|=*' 'l:|=* r:|=*'

However, people at Oh-My-Zsh found it triggers an interesting "bug" [1]. To be clear, it is not related to OMZ (at least, no further than the fact that OMZ also uses that line). I call it a "bug", but it is just an undesirable side-effect of the matcher-list statement, and I want to understand how to fix it.

How to reproduce?
The "bug" occurs when a completion is triggered with multiple choices such that:
* completion choices are all the same length;
* completion choices share a common suffix;
* that suffix is at least half of the length but not the full length;
* in each choice, the first and the n-th character of the suffix are both letters, where n is the length of the prefix;
* in each choice, the character just before the common suffix is a digit (maybe it works with other character types but i did not find any), and that character is different for each choice.

Under the above conditions, the "bug" occurs when completing after a substring prefixing the common suffix (i.e. completing after "a" or "ab" or "abc", when the common suffix is "abc") will insert a letter before the suffix. This letter will be the n-th character of the suffix, in inverted case.

Example: in a directory containing two files "1abc" and "2abc", completing after "less a" will expand it to "abc" and then add an extra "A", so the command line will look like "less Aabc".

I am still struggling to understand the depth of Zsh's completion system, so please, can anyone help me understand why the above behavior happens? Especially why does it specifically happen with digits?

Just pointing me in the right direction will be greatly appreciated. Best regards.

Nicolas Canceill

[1] https://github.com/robbyrussell/oh-my-zsh/issues/2030

Re: Case-insensitive completion with partial substring matching

[Bart Schaefer]

On Jun 7,  5:12pm, nicolas.canceill wrote:
}
} zstyle ':completion:*' matcher-list 'm:{a-zA-Z}={A-Za-z}' 'r:|[._-]=* r:|=*' 'l:|=* r:|=*'
} 
} Under the above conditions, the "bug" occurs when completing after a
} substring prefixing the common suffix (i.e. completing after "a" or
} "ab" or "abc", when the common suffix is "abc") will insert a letter
} before the suffix. This letter will be the n-th character of the
} suffix, in inverted case.
}
} Example: in a directory containing two files "1abc" and "2abc",
} completing after "less a" will expand it to "abc" and then add an
} extra "A", so the command line will look like "less Aabc".

This is definitely a bug, and although I don't understand why it is
happening, it appears to have to do with characters that are left out
of the m: range in the matcher list.  E.g. if you change to:

zstyle ':completion:*' matcher-list \
	'm:{a-zA-Z0-9}={A-Za-z0-9}' \
	'r:|[._-]=* r:|=*' 'l:|=* r:|=*'

then the problem with digits goes away, but you can still reproduce the
bug for files with names like "_abc" and "%abc".

Re: Case-insensitive completion with partial substring matching

[nicolas.canceill]

Hi Bart, and thanks for replying.

Are you saying I should file a bug with Zsh about that? If you think I can assist in debugging, maybe you could help me find the relevant portions of the Zsh code?

Cheers.

Nicolas Canceill



Le 7 juin 2014 à 19:54, Bart Schaefer <schaefer@brasslantern.com> a écrit :

> On Jun 7,  5:12pm, nicolas.canceill wrote:
> }
> } zstyle ':completion:*' matcher-list 'm:{a-zA-Z}={A-Za-z}' 'r:|[._-]=* r:|=*' 'l:|=* r:|=*'
> } 
> } Under the above conditions, the "bug" occurs when completing after a
> } substring prefixing the common suffix (i.e. completing after "a" or
> } "ab" or "abc", when the common suffix is "abc") will insert a letter
> } before the suffix. This letter will be the n-th character of the
> } suffix, in inverted case.
> }
> } Example: in a directory containing two files "1abc" and "2abc",
> } completing after "less a" will expand it to "abc" and then add an
> } extra "A", so the command line will look like "less Aabc".
> 
> This is definitely a bug, and although I don't understand why it is
> happening, it appears to have to do with characters that are left out
> of the m: range in the matcher list.  E.g. if you change to:
> 
> zstyle ':completion:*' matcher-list \
> 	'm:{a-zA-Z0-9}={A-Za-z0-9}' \
> 	'r:|[._-]=* r:|=*' 'l:|=* r:|=*'
> 
> then the problem with digits goes away, but you can still reproduce the
> bug for files with names like "_abc" and "%abc".

Re: Case-insensitive completion with partial substring matching

[Bart Schaefer]

On Jun 7,  8:00pm, nicolas.canceill wrote:
}
} Are you saying I should file a bug with Zsh about that?

You already did, just by sending your first message.

} If you think I can assist in debugging, maybe you could help me find
} the relevant portions of the Zsh code?

Well, it's all in Src/Zle/compmatch.c, and it probably has something to
do with the functions bld_line, pattern_match_equivalence, or join_sub,
but there'll be a pretty steep learning curve.  Dive in, if brave ...

I also recommend testing with the ":" command or anything for which there
is no complicated completion function defined; using "less" for example
generates a lot of extra irrelevant passes through the completion code.

Re: Case-insensitive completion with partial substring matching

[nicolas.canceill]

Oh yes, I had not seen the notice about the sourceforge tickets.

In order to look into the code, I want to log a bunch of stuff (my favorite way). The debug flags I see mentioned in the INSTALL file do not seem relevant. Do you know what the best way to log stuff from within the functions would be?

Thanks a bunch for your help.

Nicolas Canceill



Le 7 juin 2014 à 21:30, Bart Schaefer <schaefer@brasslantern.com> a écrit :

> On Jun 7,  8:00pm, nicolas.canceill wrote:
> }
> } Are you saying I should file a bug with Zsh about that?
> 
> You already did, just by sending your first message.
> 
> } If you think I can assist in debugging, maybe you could help me find
> } the relevant portions of the Zsh code?
> 
> Well, it's all in Src/Zle/compmatch.c, and it probably has something to
> do with the functions bld_line, pattern_match_equivalence, or join_sub,
> but there'll be a pretty steep learning curve.  Dive in, if brave ...
> 
> I also recommend testing with the ":" command or anything for which there
> is no complicated completion function defined; using "less" for example
> generates a lot of extra irrelevant passes through the completion code.

Re: Case-insensitive completion with partial substring matching

[Bart Schaefer]

This is rapidly approaching a zsh-workers discussion rather than -users.

On Jun 7, 10:39pm, nicolas.canceill wrote:
} Subject: Re: Case-insensitive completion with partial substring matching
}
} In order to look into the code, I want to log a bunch of stuff (my
} favorite way). The debug flags I see mentioned in the INSTALL file do
} not seem relevant. Do you know what the best way to log stuff from
} within the functions would be?

I'm not sure what you're asking about there.  Logging stuff from within
the C functions isn't really automated in any way.  Logging stuff from
the shell functions that make up completion can be done, but it's not
of much use in this case (I already examined that).

You want to configure --with-zsh-debug --disable-dynamic for pursposes
of using e.g. gdb, but the built-in debugging messages mostly take the
form of assertions that only produce output if an anticipated condition
turns out to be false, but that doesn't really apply.

So you're pretty much on your own, I guess.