Re: Remaining zsh3.0-pre2 bugs

[Zefram <A.Main@dcs.warwick.ac.uk>]

>So I rather suspect that has something to do with the bug, but I'm lost
>at this point in terms of knowing what \230 represents, or how it got
>into tokstr and thus ultimately into clwords[1], or whether INULL() is
>supposed to be true at that point.

'\230' is Snull -- the token that the ' turns into.  INULL() *is*
supposed to be true for this token -- it indicates that that character
should be ignored when considering the line at later stages of
execution.  The four tokens it is true for are Snull (single quote as
here), Dnull (double quote), Bnull (backslash) and Nularg (an empty
argument).

Note that the ; in the middle did *not* turn into a token -- the lexer
sees that it is quoted, so leaves it literal.  If the line were
actually being executed (pretending that it really is a ; and not a
newline), the code after the lexer would see an argument a;b, ignoring
the Snull tokens.  Had the quotes not been there, the ; would have
turned into a token and been treated specially.

It's really quite an elegant system.  Unfortunately it does fall over
somewhat when the token values are appearing in the input as normal
characters, but as you know we now have a workaround for that too.

>Anyway, cs decrements down to 8 because of the two \230 in qword; but
>"echo 'a;b'" of course has 10 characters.

Bingo.  That loop has special code for handling backslashes, but not
quotes.  I think it needs to handle quotes too.  The completion code
really doesn't cope with quotes at all, so I'm not surprised this is
happening.

-zefram