Bug: Unicode character of zsh module will become weird characters

Bug: Unicode character of zsh module will become weird characters

[Wu, Zhenyu]

Excuse me:

Reproduce procedures:

```zsh
% git clone --depth=1 https://github.com/zsh-users/zsh
% cd zsh
% sed -i 's/strparam = ztrdup("example");/strparam = ztrdup("你好");/' Src/Modules/example.c
% autoreconf -vif
% ./configure
% make
```

Actual behavior:

```zsh
% zsh -f
laptop% module_path+=( $PWD/Src/Modules )
laptop% zmodload example
The example module has now been set up.
laptop% echo ${(q)exstr}
$'\344'$'\275'\好
laptop%
```

Expected behavior:

```zsh
% zsh -f
laptop% module_path+=( $PWD/Src/Modules )
laptop% zmodload example
The example module has now been set up.
laptop% echo ${(q)exstr}
你好
laptop%
```

Is it a bug? TIA!

-- 
Best Regards
Wu, Zhenyu

Re: Bug: Unicode character of zsh module will become weird characters

[Peter Stephenson]

> On 11/05/2024 14:40 BST Wu, Zhenyu <wuzhenyu@ustc.edu> wrote:
> ```zsh
> % git clone --depth=1 https://github.com/zsh-users/zsh
> % cd zsh
> % sed -i 's/strparam = ztrdup("example");/strparam = ztrdup("你好");/' Src/Modules/example.c
> % autoreconf -vif
> % ./configure
> % make
> ```

I think The problem's actually here.  Zsh has a special encoding of
strings internally to avoid confusion between normal strings and
tokenised strings.  This is because special characters are mixed in with
normal characters in strings inside the code.  This system is known as
"metafication".  This means you have to be careful when creating strings
to be stored inside the system.  In fact, this is usually only a problem
if strings contain byte 0x80, so it can be a bit of a surprise when you
see it.

If you're doing normal input and output this should work automatically,
but you're updating the source code directly.  Try changing the line in
the example file to:

    strparam = metafy("你好", strlen("你好"), META_DUP);

and see if that works --- looks OK to me with a quick test.

cheers
pws