fnmatch bug? escaping of '-' and ']' inside [...] does not work.

fnmatch bug? escaping of '-' and ']' inside [...] does not work.

[Denys Vlasenko]

Moving bbox build to musl did uncover one definite bug.
bbox has gazillion tests for its shells, and one of them
started failing. I added debug printout and with it,
the following fnmatch calls give different results now:

ash: fnmatch(pattern:'[a\-c]',str:'-',0):1
ash: fnmatch(pattern:'[\]]',str:']',0):1
ash: fnmatch(pattern:'[a\]]',str:']',0):1

glibc and uclibc threat the above as a match.

The assumption may be that one should place ']' or '-'
as a first character in the [...] and thus escaping
is not necessary, but shell does not have the luxury
of choosing the pattern, in comes from the script.

Just in case you want to know, the complete test is:

# Testing glob-escaping of every ASCII punctuation char
# Some chars have more than one test
# 21..2f
case '!' in [\!] ) echo ok;; *) echo 'WRONG!';; esac
case '"' in [\"] ) echo ok;; *) echo 'WRONG"';; esac
case '#' in [\#] ) echo ok;; *) echo 'WRONG#';; esac
case '$' in [\$] ) echo ok;; *) echo 'WRONG$';; esac
case '%' in [\%] ) echo ok;; *) echo 'WRONG%';; esac
case '&' in [\&] ) echo ok;; *) echo 'WRONG&';; esac
case "'" in [\'] ) echo ok;; *) echo "WRONG'";; esac
case '(' in [\(] ) echo ok;; *) echo 'WRONG(';; esac
case ')' in [\)] ) echo ok;; *) echo 'WRONG)';; esac
case '*' in [\*] ) echo ok;; *) echo 'WRONG*';; esac
case '+' in [\+] ) echo ok;; *) echo 'WRONG+';; esac
case ',' in [\,] ) echo ok;; *) echo 'WRONG,';; esac
case '-' in [\-] ) echo ok;; *) echo 'WRONG-';; esac
case '-' in [a\-c]) echo ok;; *) echo 'WRONGa\-c';; esac
case '.' in [\.] ) echo ok;; *) echo 'WRONG.';; esac
case '/' in [\/] ) echo ok;; *) echo 'WRONG/';; esac
# 3a..40
case ':' in [\:] ) echo ok;; *) echo 'WRONG:';; esac
case ';' in [\;] ) echo ok;; *) echo 'WRONG;';; esac
case '<' in [\<] ) echo ok;; *) echo 'WRONG<';; esac
case '=' in [\=] ) echo ok;; *) echo 'WRONG=';; esac
case '>' in [\>] ) echo ok;; *) echo 'WRONG>';; esac
case '?' in [\?] ) echo ok;; *) echo 'WRONG?';; esac
case '@' in [\@] ) echo ok;; *) echo 'WRONG@';; esac
# 5b..60
case '[' in [\[] ) echo ok;; *) echo 'WRONG[';; esac
case '\' in [\\] ) echo ok;; *) echo 'WRONG\';; esac
case '\' in \\ ) echo ok;; *) echo 'WRONG\\';; esac
case ']' in [\]] ) echo ok;; *) echo 'WRONG]';; esac
case ']' in [a\]]) echo ok;; *) echo 'WRONGa]';; esac
case '^' in [\^] ) echo ok;; *) echo 'WRONG^';; esac
case '_' in [\_] ) echo ok;; *) echo 'WRONG_';; esac
case '`' in [\`] ) echo ok;; *) echo 'WRONG`';; esac
# 7b..7e
case '{' in [\{] ) echo ok;; *) echo 'WRONG{';; esac
case '|' in [\|] ) echo ok;; *) echo 'WRONG|';; esac
case '}' in [\}] ) echo ok;; *) echo 'WRONG}';; esac
case '~' in [\~] ) echo ok;; *) echo 'WRONG~';; esac

Re: fnmatch bug? escaping of '-' and ']' inside [...] does not work.

[Szabolcs Nagy]

* Denys Vlasenko <vda.linux@googlemail.com> [2017-08-15 15:12:17 +0200]:
> Moving bbox build to musl did uncover one definite bug.
> bbox has gazillion tests for its shells, and one of them
> started failing. I added debug printout and with it,
> the following fnmatch calls give different results now:
> 
> ash: fnmatch(pattern:'[a\-c]',str:'-',0):1
> ash: fnmatch(pattern:'[\]]',str:']',0):1
> ash: fnmatch(pattern:'[a\]]',str:']',0):1
> 
> glibc and uclibc threat the above as a match.
> 
> The assumption may be that one should place ']' or '-'
> as a first character in the [...] and thus escaping
> is not necessary, but shell does not have the luxury
> of choosing the pattern, in comes from the script.
> 

it's not clear to me if \ is valid as escape in fnmatch pattern:
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_13_01

in re bracket expression \ is not an escape char,
but in the shell i think the pattern should be
unquoted first, which may or may not apply to
fnmatch bracket expression, depending on how you
read the text.

i guess if other implementations do the unquoting
musl should follow that, but the standard could
be improved.

Re: fnmatch bug? escaping of '-' and ']' inside [...] does not work.

[Rich Felker]

On Tue, Aug 15, 2017 at 03:12:17PM +0200, Denys Vlasenko wrote:
> Moving bbox build to musl did uncover one definite bug.
> bbox has gazillion tests for its shells, and one of them
> started failing. I added debug printout and with it,
> the following fnmatch calls give different results now:
> 
> ash: fnmatch(pattern:'[a\-c]',str:'-',0):1
> ash: fnmatch(pattern:'[\]]',str:']',0):1
> ash: fnmatch(pattern:'[a\]]',str:']',0):1
> 
> glibc and uclibc threat the above as a match.

I believe this is connected to a known discrepancy between what the
standard seems to say and what glibc is doing. I'll see if I can dig
up past discussions of it. If it's coming up again, it's likely that
this is an issue that needs a proper interpretation (and possibly fix
in the language of the standard, if the current text really does imply
something different than what it's supposed to; the fix might end up
being along the lines of "it's unspecified whether...").

Rich