Re: [bug] busy loop and memory exhaustion on {x..$'\80'} with nomultibyte

[Stephane Chazelas <stephane@chazelas.org>]

2022-09-26 14:49:08 +0900, Jun T:
> 
> > 2022/09/25 19:34, Stephane Chazelas <stephane@chazelas.org> wrote:
> > 
> > Thanks that's better, but now:
> > 
> > $ echo $options[multibyte]
> > off
> > $ printf %s {$'\x80'..$'\xff'} | hexdump -C
> > 00000000  5c 4d 2d 5e 40 5c 4d 2d  5e 41 5c 4d 2d 5e 42 5c  |\M-^@\M-^A\M-^B\|
> > 00000010  4d 2d 5e 43 5c 4d 2d 5e  44 5c 4d 2d 5e 45 5c 4d  |M-^C\M-^D\M-^E\M|
> (snip)
> > 
> > That's bytes 0x80 to 0x9f with their \M-^X representation followed by UTF-8
> > encoded (in my locale using UTF-8 as charmap) characters U+00A0 to U+00FF
> > instead of bytes 0x80 to 0xff which I'd expect with nomultibyte.
> 
> Did you try 'LANG=C; setopt print_eight_bit' ?

Thanks for the print_eight_bit clue, though it doesn't seem to make a difference:

$ zsh -o printeightbit +o multibyte -c $'printf %s {\xe8..\xea}' | hd
00000000  5e 28 5e 29 5e 2a                                 |^(^)^*|
00000006
$ LC_ALL=C zsh -o printeightbit +o multibyte -c $'printf %s {\xe8..\xea}' | hd
00000000  5e 28 5e 29 5e 2a                                 |^(^)^*|
00000006

(here in 5.8, hd being the same as hexdump -C on my system).

> 
> Anyway, I will push the patch (included below again) with a test.
> 
> diff --git a/Src/utils.c b/Src/utils.c
> index 62bd3e602..edf5d3df7 100644
> --- a/Src/utils.c
> +++ b/Src/utils.c
> @@ -5519,7 +5519,7 @@ mb_metacharlenconv(const char *s, wint_t *wcp)
>      if (!isset(MULTIBYTE) || STOUC(*s) <= 0x7f) {
>  	/* treat as single byte, possibly metafied */
>  	if (wcp)
> -	    *wcp = (wint_t)(*s == Meta ? s[1] ^ 32 : *s);
> +	    *wcp = (wint_t)STOUC(*s == Meta ? s[1] ^ 32 : *s);
>  	return 1 + (*s == Meta);
>      }
>      /*
[...]

That can't be right. The comment says "treat as single byte", yet the result is
now multibyte characters instead of bytes:

$ ./Src/zsh -o printeightbit +o multibyte -c $'printf %s {\xe8..\xea}' | hd
00000000  c3 a8 c3 a9 c3 aa                                 |......|
00000006

I asked for bytes 0xE8 to 0xEA and got UTF-8 encoded characters U+00E8 to U+00EA.

Though at least now, I can get what I want in this case with:

$ LC_ALL=C ./Src/zsh -o printeightbit +o multibyte -c $'printf %s {\xe8..\xea}' | hd
00000000  e8 e9 ea                                          |...|
00000003

(the control characters are still expanded in ^? fashion, so I
still can't get ranges of bytes with that, but that's as
documented).

-- 
Stephane