From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-3.4 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED, T_SCC_BODY_TEXT_LINE,UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 5913 invoked from network); 4 Apr 2022 14:43:37 -0000 Received: from zero.zsh.org (2a02:898:31:0:48:4558:7a:7368) by inbox.vuxu.org with ESMTPUTF8; 4 Apr 2022 14:43:37 -0000 ARC-Seal: i=1; cv=none; a=rsa-sha256; d=zsh.org; s=rsa-20210803; t=1649083417; b=cp8WWy23vRduNN+UFT0ueH4Qr3SyP4GD/pwSOR5JKfOEUgJFSnVWL+arRDLuAEmnR0XuIjIt5X ZoQePkDP9KJz6LtsXK4VyyqRhAnS5NA/Az2Zca71mEqePxbJNMIuzVQR8KaZWFxV6UTYaILOdO vxVVYry5aLlfcFKvaKHO2Y8f5cLSaNXuD18KvkxS3eJ1Hx/GiQuwBiAEV5QKxWgpMwMdAoyevg mMdgIZ4kFLr1jQ0uPKRLWeAHm5dwC+YJYO2IK4R/pIBYCAYB328Df61bwsPxQ/f/TgKgJpNuYj ZmqCfXSKD8bShyBd6C576aG2AmoHwekUIzuavPFJ1uVS4A==; ARC-Authentication-Results: i=1; zsh.org; iprev=pass (smtpq1.tb.ukmail.iss.as9143.net) smtp.remote-ip=212.54.57.96; dkim=pass header.d=ntlworld.com header.s=meg.feb2017 header.a=rsa-sha256; dmarc=pass header.from=ntlworld.com; arc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed; d=zsh.org; s=rsa-20210803; t=1649083417; bh=TjA+pI1T2BPKiju+Xli7Nj6hvkYEbAL+XM0Q84eVMDk=; h=List-Archive:List-Owner:List-Post:List-Unsubscribe:List-Subscribe:List-Help: List-Id:Sender:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject: References:In-Reply-To:Message-ID:To:From:Date:DKIM-Signature: DKIM-Signature; b=HvTl9Xgcj8jGf3Z83B8/hB0xVlwQ1YzqjK37xTmJppP8oenPiDzkwg+CscgM88U04XUdOPYzXp ndeoj/OlCWSv9cMN3Ug7QUxAsNMaGiANd+8JZa2PngSQwt8h6qpNzdid5QBeOCnx1FO5fBxBTZ sl8rtTQUfm0vBa+YuYXDdXTKqo+PAxq0nsuMsiZqNaujJ0qRbriGv6mftbwspjpbQfxC+hCS5r 4uOCblEi3aO7XlG3ki8VW2rIOj9dMeNXxCIvMfxEBUOHDVFTUDDRDhhseFX2U/+pGvHHpRsHjQ 7ibCwV6+SsD4y65OaVoBs86kD5YMke5oYg4WJX+UrbPcDw==; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=zsh.org; s=rsa-20210803; h=List-Archive:List-Owner:List-Post:List-Unsubscribe: List-Subscribe:List-Help:List-Id:Sender:Content-Transfer-Encoding: Content-Type:MIME-Version:Subject:References:In-Reply-To:Message-ID:To:From: Date:Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID; bh=g5yn9qA5Deypqekm9d6qyAiGqJFRj7UG9JprTXx6RB4=; b=bWrPhmO6KOjN+YWuKvQ0eW1/+B uGora1bBjvT4d1qtBe5X/w5+PwZgXVLcqadYOIlq04m1Rpitvf7VXjtAxCRoEf35dzLY7xLoiCNxs 6ckuTHlHklcJ9ZQ/enJOLd+4PMwjWFEMTd1ThC/+OdkxvIQ3C/0o8YP/jdpvbo2P4wfeZuMCdo7Ar cjt1icMjOPXfKJQkAQcHpgUKKNnBiB2uLi31pDz1y5/bQ98lhtUdhZYWfCEconPDUgpeov26GncUi 3kyFI8YUqH/VWO1cxcjfsCdXrpUzwZOtQC7B/95ho9ctvru+Lw343wvLZD8OsMqWr5Bs3rn9fKW7k 335OUmqQ==; Received: from authenticated user by zero.zsh.org with local id 1nbNvb-0005oa-QU; Mon, 04 Apr 2022 14:43:35 +0000 Authentication-Results: zsh.org; iprev=pass (smtpq1.tb.ukmail.iss.as9143.net) smtp.remote-ip=212.54.57.96; dkim=pass header.d=ntlworld.com header.s=meg.feb2017 header.a=rsa-sha256; dmarc=pass header.from=ntlworld.com; arc=none Received: from smtpq1.tb.ukmail.iss.as9143.net ([212.54.57.96]:59742) by zero.zsh.org with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) id 1nbNvL-0005Tk-AR; Mon, 04 Apr 2022 14:43:20 +0000 Received: from [212.54.57.80] (helo=smtp1.tb.ukmail.iss.as9143.net) by smtpq1.tb.ukmail.iss.as9143.net with esmtp (Exim 4.90_1) (envelope-from ) id 1nbNvK-0004QE-Pe for zsh-workers@zsh.org; Mon, 04 Apr 2022 16:43:18 +0200 Received: from oxbe3.tb.ukmail.iss.as9143.net ([172.25.160.134]) by smtp1.tb.ukmail.iss.as9143.net with ESMTP id bNvKnP0zXBAYNbNvKnYwFR; Mon, 04 Apr 2022 16:43:18 +0200 X-Env-Mailfrom: p.w.stephenson@ntlworld.com X-Env-Rcptto: zsh-workers@zsh.org X-SourceIP: 172.25.160.134 X-CNFS-Analysis: v=2.4 cv=IZSU5Ema c=1 sm=1 tr=0 ts=624b0406 cx=a_exe a=rX2mKgs1IZJU9lF6pltyEQ==:117 a=m8QaGag_sGQA:10 a=IkcTkHD0fZMA:10 a=R2osC4dZ-SQA:10 a=q2GGsy2AAAAA:8 a=hD80L64hAAAA:8 a=fN5zyj8hbCNssJxL1OoA:9 a=QEXdDO2ut3YA:10 a=z9dJwno5l634igLiVhy-:22 X-Authenticated-Sender: p.w.stephenson@ntlworld.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ntlworld.com; s=meg.feb2017; t=1649083398; bh=TjA+pI1T2BPKiju+Xli7Nj6hvkYEbAL+XM0Q84eVMDk=; h=Date:From:To:In-Reply-To:References:Subject; b=V0NbiA1Q/FVKCeGhYs3yIzmAUqS2knAQXi59u/T2UQpdspkKIDcBdNKFtTq1BhqMM 3l358gKBRFjeWCCQGR/rqvp/HtaX+16+gwLDSYjinBXzfy60m/OyhLtDcy4eUnWzEx Fumq3c9kBM7lO0aFHlOyfN8N45IZOkKGznR3kTLA7xa5HdAR5ac/XiDCcVrNRgY3zf HKFRIKeKe5z2o/xjKGKTVgoNey1D8N93HOku2QoN6Z1iqdQj4tDOpa2jBLH4LN1nEI 5yTyEbhWTjyNdJH5ICFRNXGvEZUoiR5CGYSI/4PNTsEojDxirdH3GmNxZxh3iHBhM0 tDzu4MOIV/kqg== Date: Mon, 4 Apr 2022 15:43:18 +0100 (BST) From: Peter Stephenson To: Bart Schaefer , Zsh hackers list Message-ID: <1174195730.504727.1649083398497@mail2.virginmedia.com> In-Reply-To: References: Subject: Re: Lots of test failures when --disable-multibyte MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Priority: 3 Importance: Medium X-Mailer: Open-Xchange Mailer v7.8.4-Rev72 X-Originating-IP: 147.161.167.97 X-Originating-Client: open-xchange-appsuite X-CMAE-Envelope: MS4xfOkB4YsuYhZSPOgHj6zvMAzPAIIw5vjkLL4LtV+fObQsbE0W5AzfVqxKFKW8kcudofGAK1vDqEDmgKI988CMaJwGaBlbMsS3GwxfgSu0VEzOXvT2PX+3 X0YN+EwHG1j4STRKX5K7jQELG8Yy7Dgxu17PIDBE/rnIGzeGkwB3uhTd80a80Tp5wdCMWwOnr+HRm3fR8eW+7Zlcv9q1AMwD9v7w1/1vYCKda6Ihx+wribRe SJkkkX012bJ7i2u0z0CHMrsJX9I7EcyEvWQoe1FX/ec= X-Seq: 49987 Archived-At: X-Loop: zsh-workers@zsh.org Errors-To: zsh-workers-owner@zsh.org Precedence: list Precedence: bulk Sender: zsh-workers-request@zsh.org X-no-archive: yes List-Id: List-Help: List-Subscribe: List-Unsubscribe: List-Post: List-Owner: List-Archive: On 04 April 2022 at 03:03 Bart Schaefer wrote: > Many seem to be because typeset fails to apply $'...' quoting to > strings with embedded metacharacters or NUL bytes, but there are > several other odd ones too. Adding some extra single-byte "nice" quoting isn't so hard; this won't fix everything but seems OK as far as it goes. pws commit 61802c2b2b06cab976d392e145d8db5d372a879d Author: Peter Stephenson Date: Mon Apr 4 15:34:40 2022 +0100 Single byte versions of nice quoting diff --git a/Src/utils.c b/Src/utils.c index f9127c70c..c5ee78db6 100644 --- a/Src/utils.c +++ b/Src/utils.c @@ -438,7 +438,6 @@ putshout(int c) return 0; } -#ifdef MULTIBYTE_SUPPORT /* * Turn a character into a visible representation thereof. The visible * string is put together in a static buffer, and this function returns @@ -517,6 +516,8 @@ nicechar_sel(int c, int quotable) return buf; } +#ifdef MULTIBYTE_SUPPORT + /**/ mod_export char * nicechar(int c) @@ -5793,7 +5794,7 @@ mb_charlenconv(const char *s, int slen, wint_t *wcp) } /**/ -#else +#else /* MULTIBYTE_SUPPORT */ /* Simple replacement for mb_metacharlenconv */ @@ -5833,6 +5834,121 @@ charlenconv(const char *x, int len, int *c) return 1; } +/* + * Non-multibyte version of mb_niceformat() above. Same basic interface. + */ + +/**/ +mod_export size_t +sb_niceformat(const char *s, FILE *stream, char **outstrp, int flags) +{ + size_t l = 0, newl; + int umlen, outalloc, outleft; + char *ums, *ptr, *eptr, *fmt, *outstr, *outptr; + + if (outstrp) { + outleft = outalloc = 2 * strlen(s); + outptr = outstr = zalloc(outalloc); + } else { + outleft = outalloc = 0; + outptr = outstr = NULL; + } + + ums = ztrdup(s); + /* + * is this necessary at this point? niceztrlen does this + * but it's used in lots of places. however, one day this may + * be, too. + */ + untokenize(ums); + ptr = unmetafy(ums, ¨en); + eptr = ptr + umlen; + + while (ptr < eptr) { + int c = STOUC(*ptr); + if (c == '\'' && (flags & NICEFLAG_QUOTE)) { + fmt = "\\'"; + newl = 2; + } + else if (c == '\\' && (flags & NICEFLAG_QUOTE)) { + fmt = "\\\\"; + newl = 2; + } + else { + fmt = nicechar_sel(c, flags & NICEFLAG_QUOTE); + newl = 1; + } + + ++ptr; + l += newl; + + if (stream) + zputs(fmt, stream); + if (outstr) { + /* Append to output string */ + int outlen = strlen(fmt); + if (outlen >= outleft) { + /* Reallocate to twice the length */ + int outoffset = outptr - outstr; + + outleft += outalloc; + outalloc *= 2; + outstr = zrealloc(outstr, outalloc); + outptr = outstr + outoffset; + } + memcpy(outptr, fmt, outlen); + /* Update start position */ + outptr += outlen; + /* Update available bytes */ + outleft -= outlen; + } + } + + free(ums); + if (outstrp) { + *outptr = '\0'; + /* Use more efficient storage for returned string */ + if (flags & NICEFLAG_NODUP) + *outstrp = outstr; + else { + *outstrp = (flags & NICEFLAG_HEAP) ? dupstring(outstr) : + ztrdup(outstr); + free(outstr); + } + } + + return l; +} + +/* + * Return 1 if sb_niceformat() would reformat this string, else 0. + */ + +/**/ +mod_export int +is_sb_niceformat(const char *s) +{ + int umlen, ret = 0; + char *ums, *ptr, *eptr; + + ums = ztrdup(s); + untokenize(ums); + ptr = unmetafy(ums, ¨en); + eptr = ptr + umlen; + + while (ptr < eptr) { + if (is_nicechar(*ptr)) { + ret = 1; + break; + } + ++ptr; + } + + free(ums); + + return ret; +} + /**/ #endif /* MULTIBYTE_SUPPORT */ @@ -6366,6 +6482,22 @@ quotedzputs(char const *s, FILE *stream) return outstr; } } +#else + if (is_sb_niceformat(s)){ + if (stream) { + fputs("$'", stream); + sb_niceformat(s, stream, NULL, NICEFLAG_QUOTE); + fputc('\'', stream); + return NULL; + } else { + char *substr; + sb_niceformat(s, NULL, &substr, NICEFLAG_QUOTE|NICEFLAG_NODUP); + outstr = (char *)zhalloc(4 + strlen(substr)); + sprintf(outstr, "$'%s'", substr); + free(substr); + return outstr; + } + } #endif /* MULTIBYTE_SUPPORT */ if (!hasspecial(s)) { diff --git a/Src/zsh.h b/Src/zsh.h index 641d9c95c..40f9ea537 100644 --- a/Src/zsh.h +++ b/Src/zsh.h @@ -3277,14 +3277,15 @@ enum zexit_t { #define AFTERTRAPHOOK (zshhooks + 2) #define GETCOLORATTR (zshhooks + 3) -#ifdef MULTIBYTE_SUPPORT -/* Final argument to mb_niceformat() */ +/* Final argument to [ms]b_niceformat() */ enum { NICEFLAG_HEAP = 1, /* Heap allocation where needed */ NICEFLAG_QUOTE = 2, /* Result will appear in $'...' */ NICEFLAG_NODUP = 4, /* Leave allocated */ }; +#ifdef MULTIBYTE_SUPPORT + /* Metafied input */ #define nicezputs(str, outs) (void)mb_niceformat((str), (outs), NULL, 0) #define MB_METACHARINIT() mb_charinit()