Re: Metafication in error messages (Was: [PATCH] unmetafy Re: $var not expanded in ${x?$var})

[Bart Schaefer <schaefer@brasslantern.com>]

On Fri, Feb 23, 2024 at 11:27 AM Stephane Chazelas
<stephane@chazelas.org> wrote:
>
> 1. whether user-supplied text in messages should be output raw

The question here is whether error message is about MISTAKENLY
user-supplied text, vs. INTENTIONALLY supplied.  My assertion is that
it's only the caller of zerr() and related functions that can
interpret whether the text is intentional or not.

I would argue that your math examples are mistakes, at least as far as
the parser could possibly "know".

> 2. Whether internally in the code the data should be passed
>   "metafied" or not to zerr* functions.

That still only means we need one way to instruct the function to skip
"nice"-ing and one way to say not to.

Skipping ahead a bit, we'll come around again later ...

> > $ printf '%d\n' $'1+|a\x83 c'
> > zsh: bad math expression: operand expected at `|a^@c'
>
> Should have been:
>
> zsh: bad math expression: operand expected at `|aM-^C c'

You're missing part of my point here.

% printf '%d\n' $(( 1+|a\x83 c ))
zsh: bad math expression: operand expected at `|a\x83 c '

That is IMO more useful than either of "^@" or "M-^C " and is down to
the difference between using printf on a $'...' string (which is
interpreted before printf even gets its mitts on it, and two layers
before the math parser does) vs. using the actual math parser
directly.  This has nothing to do with how the string is passed to
zerr() and everything to do with how printf and parsing interpret the
input -- by the time the math parser actually calls zerr() it can't
know how to unwind that, and the internals of zerr() are even further
removed.

I would therefore argue that these examples are out of scope for this
discussion -- these examples are not about how zerr() et al. should
receive strings, they're about how the math parser should receive
them, and needs to be fixed upstream e.g. in bin_print().

More relevant to this discussion is that math errors are one of the
two existing callers using the %l format, so any attempt to improve
this is going to require changing those calls anyway.

> For 1, IMO, when the error message is generated by zsh, it
> should go through nicezputs(). zsh should decide of the
> formatting, have it pass escape sequences as-is would make it
> hard to understand and diagnose the error.

Agreed in concept, but there's a difference between errors actually
generated BY zsh, and errors with user input that zsh is reporting.
For example, the same literal string might be a file name generated by
globbing, or it might be something the user typed out in a
syntactically invalid command.  There's no way to put intelligence
about how to format those into the guts of zerr().

The question of whether to go through all 500 calls to zerr(),
zwarn(), etc., and "fix" them is something entirely else.  Presently
we have just the one known case of wanting something different for
${var?msg}.

> For 2, it looks like zerrmsg expects its input metafied and as
> you say, that input in most cases is likely to be metafied
> already. Not metafied would mean either we couldn't pass text
> containing NUL, or we'd need to pass it as ptr+len.

There's already a way to pass text not containing NUL (%s) and a way
to pass text as ptr+len (%l).  There are a vanishingly small number of
uses of the latter (2 callers out of the ~500 total call examples).
There's exactly one case so far of wanting output to contain NUL, and
per the "only caller can interpret" assertion, it seems worthwhile to
use %l for the NUL case and let the other 3 callers decide to "nice"
the strings they pass (or not).

This not only skips extra metafication needed to use the proposed %S,
but also simplifies the implementation of %l, and requires the
addition of only 1 or 2 lines of code to each of the two existing
callers using %l (maybe zero lines of code in the case of yyerror()).

> %S also passed metafied, but no nicezputs.

That requires metafy in the caller followed by unmetafy in zerr().
Much easier to remove code from %l than to add it to a new %S,
especially given that we're editing the solitary caller where %S would
be used.

> Now, my previous message was showing there were quite a few
> issue with the metafication and possibly with the nicezputs'ing
> and/or multibyte handling.

Fine, but not fixable in zerr() and friends.

> [...]
> > $ printf '%d\n' '1+|ÃÃÃÃÃÃ'
> > zsh: bad math expression: operand expected at `|\M-C\M-c\M-c\M-c\M-c\M-c\M-^C...'
>
> I picked à because it's a letter from the latin script, so you
> can even use it in variable names

Sure, but it's not being interpreted as any part of a variable name
here, because you put it in quotes and then passed it to printf.
Remove the "|" and you get something worse:

% printf '%d\n' '1+ÃÃÃÃÃÃ'
BUG: unexpected end of string in ztrlen()
zsh: bad math expression: operand expected at `\M-C\M-c\M-c\M-c\M-c\M-c'
0
% printf '%d\n' $((1+ÃÃÃÃÃÃ))
1

(Also a bit weird that the first \M-C is capitalized and the rest are
not?)  Still not a problem to be resolved in zerr().

> > $ ((1+|ÃÃÃÃÃÃ))
> > zsh: bad math expression: operand expected at `|ÃÃÃÃ\M-C...'
>
> In that case, metafication OK, but character cut in the middle.

Still not zerr()'s fault and needs to be addressed where the number of
bytes for %l is being calculated in checkunary().

> > % ((1+|ÃÃÃÃÃÃ))
> > zsh: bad math expression: operand expected at `|Ã?Ã?Ã?...'
>
> It seems rather worse to me.

That's because of the way I chose to lift nice-ifying up into
checkunary() for testing the approach.  It's hard to be consistent
there, given the foregoing business about different formats being sent
down from printf vs. $((...)), and it's also why I said "no patch
without feedback".