zsh-workers
 help / color / mirror / code / Atom feed
From: "Jason C. Kwan" <jasonckwan@yahoo.com>
To: Bart Schaefer <schaefer@brasslantern.com>
Cc: "zsh-workers@zsh.org" <zsh-workers@zsh.org>
Subject: Re: bug report : printf %.1s outputting more than 1 character
Date: Wed, 15 Mar 2023 04:56:56 +0000 (UTC)	[thread overview]
Message-ID: <478761809.298180.1678856216911@mail.yahoo.com> (raw)
In-Reply-To: <CAH+w=7bkqfUYbQ2DvZ7RPDZpSgZVxMZcT8orAiANZhWMY7fxHg@mail.gmail.com>

[-- Attachment #1: Type: text/plain, Size: 7381 bytes --]

 quote :
============This triggers a branch of the printf code introduced by this comment:
    /*
    * Invalid/incomplete character at this
    * point.  Assume all the rest are a
    * single byte.  That's about the best we
    * can do.
    */============

does the following ( below the "====" line ) behavior look even reasonable at all, regardless of your spec ? Because what the spec ends up doing is treating the rest of the input string as 1 byte and printing everything out, even though there are valid code points further down the input string. 
The behavior is correct when LC_ALL=C is set, meaning zsh already has the codes needed to generate the correct output. My point was that instead of treating the rest of the input string, regardless of size, as 1 byte/character, why not have it behave "as if" LC_ALL=C is in effect whenever it enters this branch :
if (chars < 0) {/* * Invalid/incomplete character at this * point.  Assume all the rest are a * single byte.  That's about the best we * can do. */lchars += lleft;lbytes = (ptr - b) + lleft;break;
and continue in this mode until a locale-valid character is found, then revert back to multi-byte behavior ? wouldn't that be a more logical behavior ?
If that's too complex to implement, then perhaps treat rest of input string as a collection of individual bytes instead of just 1 byte ?
I just find printf '%.3s' outputting a 179 KB string rather odd.
=========================
 zsh --restricted --no-rcs --nologin --verbose -xtrace -f -c '___=$'\''=\343\276\255#\377\210\234\256A\301B\354\210\264_'\''; command printf "%s" "$___" | gwc -lcm; for __ in {1..16}; do builtin printf "%.${__}s" "$___" | gwc -lcm; done '___=$'=\343\276\255#\377\210\234\256A\301B\354\210\264_'; command printf "%s" "$___" | gwc -lcm; for __ in {1..16}; do builtin printf "%.${__}s" "$___" | gwc -lcm; done+zsh:1> ___=$'=㾭#\M-\C-?\M-\C-H\M-\C-\\M-.A\M-AB숴_'+zsh:1> printf %s $'=㾭#\M-\C-?\M-\C-H\M-\C-\\M-.A\M-AB숴_'+zsh:1> gwc -lcm      0       7      16+zsh:1> __=1+zsh:1> printf %.1s $'=㾭#\M-\C-?\M-\C-H\M-\C-\\M-.A\M-AB숴_'+zsh:1> gwc -lcm      0       1       1+zsh:1> __=2+zsh:1> printf %.2s $'=㾭#\M-\C-?\M-\C-H\M-\C-\\M-.A\M-AB숴_'+zsh:1> gwc -lcm      0       2       4+zsh:1> __=3+zsh:1> printf %.3s $'=㾭#\M-\C-?\M-\C-H\M-\C-\\M-.A\M-AB숴_'+zsh:1> gwc -lcm      0       3       5+zsh:1> __=4+zsh:1> printf %.4s $'=㾭#\M-\C-?\M-\C-H\M-\C-\\M-.A\M-AB숴_'+zsh:1> gwc -lcm      0       7      16+zsh:1> __=5+zsh:1> printf %.5s $'=㾭#\M-\C-?\M-\C-H\M-\C-\\M-.A\M-AB숴_'+zsh:1> gwc -lcm      0       7      16+zsh:1> __=6+zsh:1> printf %.6s $'=㾭#\M-\C-?\M-\C-H\M-\C-\\M-.A\M-AB숴_'+zsh:1> gwc -lcm      0       7      16+zsh:1> __=7+zsh:1> printf %.7s $'=㾭#\M-\C-?\M-\C-H\M-\C-\\M-.A\M-AB숴_'+zsh:1> gwc -lcm      0       7      16+zsh:1> __=8+zsh:1> printf %.8s $'=㾭#\M-\C-?\M-\C-H\M-\C-\\M-.A\M-AB숴_'+zsh:1> gwc -lcm      0       7      16+zsh:1> __=9+zsh:1> printf %.9s $'=㾭#\M-\C-?\M-\C-H\M-\C-\\M-.A\M-AB숴_'+zsh:1> gwc -lcm      0       7      16+zsh:1> __=10+zsh:1> printf %.10s $'=㾭#\M-\C-?\M-\C-H\M-\C-\\M-.A\M-AB숴_'+zsh:1> gwc -lcm      0       7      16+zsh:1> __=11+zsh:1> printf %.11s $'=㾭#\M-\C-?\M-\C-H\M-\C-\\M-.A\M-AB숴_'+zsh:1> gwc -lcm      0       7      16+zsh:1> __=12+zsh:1> printf %.12s $'=㾭#\M-\C-?\M-\C-H\M-\C-\\M-.A\M-AB숴_'+zsh:1> gwc -lcm      0       7      16+zsh:1> __=13+zsh:1> printf %.13s $'=㾭#\M-\C-?\M-\C-H\M-\C-\\M-.A\M-AB숴_'+zsh:1> gwc -lcm      0       7      16+zsh:1> __=14+zsh:1> printf %.14s $'=㾭#\M-\C-?\M-\C-H\M-\C-\\M-.A\M-AB숴_'+zsh:1> gwc -lcm      0       7      16+zsh:1> __=15+zsh:1> printf %.15s $'=㾭#\M-\C-?\M-\C-H\M-\C-\\M-.A\M-AB숴_'+zsh:1> gwc -lcm      0       7      16+zsh:1> __=16+zsh:1> printf %.16s $'=㾭#\M-\C-?\M-\C-H\M-\C-\\M-.A\M-AB숴_'+zsh:1> gwc -lcm      0       7      16
+zsh:1> ___=$'=㾭#\M-\C-?\M-\C-H\M-\C-\\M-.A\M-AB숴_'+zsh:1> LC_ALL=C printf %s $'=㾭#\M-\C-?\M-\C-H\M-\C-\\M-.A\M-AB숴_'+zsh:1> gwc -lcm      0       7      16+zsh:1> __=1+zsh:1> LC_ALL=C +zsh:1> printf %.1s '=㾭#????A?B숴_'+zsh:1> gwc -lcm      0       1       1+zsh:1> __=2+zsh:1> LC_ALL=C +zsh:1> printf %.2s '=㾭#????A?B숴_'+zsh:1> gwc -lcm      0       1       2+zsh:1> __=3+zsh:1> LC_ALL=C +zsh:1> printf %.3s '=㾭#????A?B숴_'+zsh:1> gwc -lcm      0       1       3+zsh:1> __=4+zsh:1> LC_ALL=C +zsh:1> printf %.4s '=㾭#????A?B숴_'+zsh:1> gwc -lcm      0       2       4+zsh:1> __=5+zsh:1> LC_ALL=C +zsh:1> printf %.5s '=㾭#????A?B숴_'+zsh:1> gwc -lcm      0       3       5+zsh:1> __=6+zsh:1> LC_ALL=C +zsh:1> printf %.6s '=㾭#????A?B숴_'+zsh:1> gwc -lcm      0       3       6+zsh:1> __=7+zsh:1> LC_ALL=C +zsh:1> printf %.7s '=㾭#????A?B숴_'+zsh:1> gwc -lcm      0       3       7+zsh:1> __=8+zsh:1> LC_ALL=C +zsh:1> printf %.8s '=㾭#????A?B숴_'+zsh:1> gwc -lcm      0       3       8+zsh:1> __=9+zsh:1> LC_ALL=C +zsh:1> printf %.9s '=㾭#????A?B숴_'+zsh:1> gwc -lcm      0       3       9+zsh:1> __=10+zsh:1> LC_ALL=C +zsh:1> printf %.10s '=㾭#????A?B숴_'+zsh:1> gwc -lcm      0       4      10+zsh:1> __=11+zsh:1> LC_ALL=C +zsh:1> printf %.11s '=㾭#????A?B숴_'+zsh:1> gwc -lcm      0       4      11+zsh:1> __=12+zsh:1> LC_ALL=C +zsh:1> printf %.12s '=㾭#????A?B숴_'+zsh:1> gwc -lcm      0       5      12+zsh:1> __=13+zsh:1> LC_ALL=C +zsh:1> printf %.13s '=㾭#????A?B숴_'+zsh:1> gwc -lcm      0       5      13+zsh:1> __=14+zsh:1> LC_ALL=C +zsh:1> printf %.14s '=㾭#????A?B숴_'+zsh:1> gwc -lcm      0       5      14+zsh:1> __=15+zsh:1> LC_ALL=C +zsh:1> printf %.15s '=㾭#????A?B숴_'+zsh:1> gwc -lcm      0       6      15+zsh:1> __=16+zsh:1> LC_ALL=C +zsh:1> printf %.16s '=㾭#????A?B숴_'+zsh:1> gwc -lcm      0       7      16



    On Tuesday, March 14, 2023 at 11:46:14 PM EDT, Bart Schaefer <schaefer@brasslantern.com> wrote:  
 
 On Tue, Mar 14, 2023 at 7:40 PM Jason C. Kwan <jasonckwan@yahoo.com> wrote:
>
> I'm using the macOS 13.2.1 OS-provided zsh, version 5.8.1, which I understand isn't the latest and greatest of 5.9, so perhaps this bug has already been addressed.

A related case been addressed by declaring it an intentional
divergence from POSIX, see
https://www.zsh.org/mla/workers/2022/msg00240.html

However ...

> In the 4-byte sequence as seen below ( defined via explicit octal codes ), under no Unicode scenario should 4 bytes be printed out via a command of printf %.1s, by design.
>
>  - The first byte of \377 \xFF is explicitly invalid under UTF-8 (even allowing up to 7-byte in the oldest of definitions).

This triggers a branch of the printf code introduced by this comment:
    /*
    * Invalid/incomplete character at this
    * point.  Assume all the rest are a
    * single byte.  That's about the best we
    * can do.
    */

Thus, you've deliberately invoked a case where zsh's response to
invalid input is to punt.  This dates back to the original
implementation in workers/23098,
https://www.zsh.org/mla/workers/2007/msg00019.html, January 2007.
  

[-- Attachment #2: Type: text/html, Size: 26365 bytes --]

  reply	other threads:[~2023-03-15  4:57 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <1621619253.265114.1678847919086.ref@mail.yahoo.com>
2023-03-15  2:38 ` Jason C. Kwan
2023-03-15  3:46   ` Bart Schaefer
2023-03-15  4:56     ` Jason C. Kwan [this message]
2023-03-15 15:31       ` Bart Schaefer
2023-03-15 15:50         ` Roman Perepelitsa
2023-03-18 16:56         ` Peter Stephenson

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=478761809.298180.1678856216911@mail.yahoo.com \
    --to=jasonckwan@yahoo.com \
    --cc=schaefer@brasslantern.com \
    --cc=zsh-workers@zsh.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/zsh/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).